JP2018197873A - Image processor and image processing program - Google Patents

Abstract

To provide an image processor and an image processing program which can set a region as the focus in an image diversely.SOLUTION: A MPU16 of an image processing engine 14 functions as: a region extraction unit 24 for extracting a plurality of masks from an image acquired from an imaging element 13; a position acquisition unit 25 for acquiring the relative positions of the masks extracted by the region extraction unit 24 in the depth direction of the image; and a focus region setting unit 26 for setting a focus mask to be the target of a focus from the masks extracted by the region extraction unit 24 on the basis of the relative positions of the masks extracted by the position acquisition unit 25 in the depth direction of the image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus and an image processing program.

  Conventionally, the contrast value of the edge of the subject located within the distance measuring area of the image at each position is calculated while moving the focus lens in the optical axis direction, and the lens position where the contrast value is the highest is set as the focus position (focus position). An autofocus device for detection is known (see, for example, Patent Document 1).

JP-A-4-274405

  However, in the conventional autofocus device, when a plurality of subjects are included in the distance measurement area, the subject located on the near side among these subjects is generally set as the subject to be focused. Is. For this reason, there is a problem in that it is impossible to set various subjects to be focused in the image.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image processing apparatus and an image processing program capable of setting various areas to be focused in an image. is there.

  An image processing apparatus that solves the above problem obtains a region extracting unit that extracts a plurality of specific regions from an image acquired from the outside, and a relative position in the depth direction of the image between the specific regions extracted by the region extracting unit. Based on the relative position in the depth direction of the image between the specific areas acquired by the position acquisition unit and the specific areas acquired by the position acquisition unit, the focus to be focused from among the specific areas extracted by the area extraction unit. And a focusing area setting unit for setting a focusing area.

  In the image processing apparatus, the position acquisition unit acquires a distance distribution of the specific region as a relative position in the depth direction of the images between the specific regions, and the focus region setting unit includes the position acquisition unit. It is preferable to set the in-focus area based on the distance distribution of the specific area acquired.

  In the image processing apparatus, the position acquisition unit acquires the distance distribution of the specific area in combination with the arrangement of the specific area constituting the distribution in the image, and the focusing area setting unit Preferably, the in-focus area is set based on a distance distribution of the specific area acquired in combination with the arrangement of the specific area in the image.

  Further, in the image processing apparatus, the database includes a distance distribution of a comparison element to be compared with a distance distribution of the specific area acquired by the position acquisition unit, and the focus area setting unit includes the focus area setting unit. Reference focusing elements set from among the comparison elements constituting the distribution are stored in association with each other as a reference for setting an area, and the focusing area setting unit is acquired by the position acquisition unit Based on the reference focusing element associated with the distribution collated in the database, by collating the distance distribution of the specific region that has been identified against the distance distribution of the comparison element stored in the database It is preferable to set the focusing area.

  In the image processing apparatus, the database stores a combination of the comparison elements with respect to the distance distribution of the comparison elements, and the focusing area setting unit is controlled by the position acquisition unit. By collating the obtained distance distribution of the specific area with the distribution of the comparison element distance and arrangement stored in the database in combination with the arrangement of the specific area in the image, Preferably, the in-focus area is set based on the reference in-focus element associated with the distribution collated in the database.

  In the image processing apparatus, the focusing area setting unit may be configured such that the distance distribution of the specific area acquired by the position acquisition unit is larger in distance than a part of specific areas compared to other specific areas. When the distribution is out of the range, it is preferable to set the in-focus area from the partial specific area.

  In the image processing apparatus, it is preferable that the distance distribution of the specific area includes the distance for each specific area classified into a plurality of distance ranges divided in the depth direction of the image.

  In the image processing apparatus, the position acquisition unit detects the distance of the specific region for each specific region based on the color component amount of the edge of the specific region extracted by the region extraction unit, It is preferable to acquire a relative position in the depth direction of the image between the specific regions.

  Further, in the image processing device, the position acquisition unit detects the distance between the specific regions extracted by the region extraction unit for each specific region by a phase difference detection method, and thereby the depth of the image between the regions. It is preferable to obtain the relative position in the direction.

  Further, in the image processing apparatus, the position acquisition unit detects the distance between the specific regions extracted by the region extraction unit for each specific region using a contrast detection method, and thereby the depth of the image between the specific regions. It is preferable to obtain the relative position in the direction.

  An image processing program that solves the above-described problem is a computer-extracted region extracting step for extracting a plurality of specific regions from an image acquired from the outside, and a depth direction of the image in the specific regions extracted in the region extracting step. A position acquisition step of acquiring a relative position of the image, and a focus from the specific region extracted in the region extraction step based on a relative position in the depth direction of the image between the specific regions acquired in the position acquisition step. And a focusing area setting step for setting a focusing area as a target of the focusing.

  According to the present invention, it is possible to set various regions to be focused in an image.

1 is a block diagram of a digital camera according to a first embodiment. 6 is a flowchart of a focus control processing routine executed by the image processing engine of the embodiment. 6 is a flowchart of a mask separation / integration processing routine executed by the image processing engine of the embodiment. 6 is a flowchart of a distribution matching process routine executed by the image processing engine of the embodiment. The schematic diagram which shows an example of the image content of a through image. The schematic diagram which shows an example of a focus map. The schematic diagram which shows an example of the image content of a through image. (A), (b) is a schematic diagram which shows an example of the evaluation value set to the mask. The schematic diagram which shows an example of distribution of the distance of a comparison element, and a focus map. 10 is a flowchart of a focusing control processing routine executed by the image processing engine of the second embodiment. The schematic diagram which shows an example of the image content of a through image. The graph which shows an example of a focus map. 10 is a flowchart of a focus control processing routine executed by the image processing engine of the third embodiment. (A), (b) is a schematic diagram which shows an example of the image content of a through image. 10 is a flowchart of a focus control processing routine executed by the image processing engine of the fourth embodiment. The schematic diagram which shows an example of the image content of a through image. The schematic diagram which shows an example of a focus map.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.
As shown in FIG. 1, a digital camera (hereinafter referred to as “camera 10”) includes a lens unit 12 (only the focus lens 11 is shown in FIG. 1) and a plurality of lenses such as a focus lens 11 for focus adjustment. And an image sensor 13 that images the light passing through the lens unit 12 by forming an image on the image space side of the lens unit 12. Note that the image sensor 13 is a CMOS (Complementary Metal Oxide Semiconductor) type or CCD (Charge Coupled Device) type image sensor.

  An image processing engine 14 is connected to the output side of the image sensor 13 via an A / D conversion circuit 15. The pixel signal output as an analog signal from the image sensor 13 is converted into a digital signal by the A / D conversion circuit 15 and then input to the image processing engine 14.

  The image processing engine 14 includes an MPU 16 (Micro Processing Unit) that comprehensively controls various operations of the camera 10. The MPU 16 generates a predetermined image by performing image processing such as color interpolation processing, gradation correction, white balance processing, and contour compensation on the pixel signal input from the image sensor 13.

  A non-volatile memory 18, a buffer memory 19, an interface unit (hereinafter referred to as “I / F unit 20”) and a monitor 21 are connected to the image processing engine 14 via a data bus 17.

  The nonvolatile memory 18 stores a program executed by the MPU 16 for operating the image processing engine 14. In the present embodiment, the non-volatile memory 18 stores an image processing program for focusing control shown in the flowchart of FIG. The MPU 16 functions as an area extraction unit 24, a position acquisition unit 25, a focusing area setting unit 26, and a mode setting unit 27 by executing an image processing program for focusing control stored in the nonvolatile memory 18. To do. The nonvolatile memory 18 stores a database 30 that is used when the MPU 16 executes an image processing program for focusing control.

The buffer memory 19 temporarily stores, for example, photographed images, images in the image processing process, images after image processing, images after image compression, and the like.
The I / F unit 20 has a card slot (not shown) in which the memory card 31 is detachably mounted. Then, the I / F unit 20 outputs the image generated by the image processing engine 14 to the memory card 31 attached to the I / F unit 20, and the image stored in the memory card 31 is output to the image processing engine 14. Or has a function to output to

  On the monitor 21, an image temporarily stored in the buffer memory 19 and an image stored in the memory card 31 attached to the I / F unit 20 are output and displayed by the image processing engine 14.

  Further, a release button 32 and an operation unit 33 are connected to the image processing engine 14. The release button 32 inputs an operation signal to the image processing engine 14 when a half-press operation or a full-press operation is performed. The operation unit 33 includes a menu button, a select button, a determination button, a power button, and the like, and inputs an operation signal to the image processing engine 14 when a pressing operation is performed.

Next, an overview of the focus control processing routine executed by the MPU 16 of the image processing engine 14 of the present embodiment will be described with reference to the flowchart of FIG.
Now, when the camera 10 is turned on, the MPU 16 starts a focus control processing routine shown in FIG. In step S <b> 10, the MPU 16 takes in the pixel signal output from the image sensor 13 to generate a through image, and displays the generated through image on the monitor 21.

  Subsequently, in step S11, the area extraction unit 24 of the MPU 16 as an area extraction step, as an example of a specific area that is an area that is expected to receive attention from the through image generated in the previous step S10. Extract the mask. In this case, a mask is extracted from the through image by performing a labeling process on the through image. Here, the labeling technique detects pixel data for each feature amount from a through image, and sequentially detects pixel regions whose positions in the through image are adjacent among pixel regions where the detected pixel data are close to each other. This is a technique for extracting a mask from a through image by grouping. In the present embodiment, the feature amount used for the labeling process includes a motion vector, hue, saturation, brightness, texture (pattern), contrast, and the like.

  Note that the area extraction unit 24 of the MPU 16 may extract a mask from a through image by performing pattern recognition processing on the through image. For example, the region extraction unit 24 of the MPU 16 performs face recognition processing, which is a type of pattern recognition processing, on the through image, thereby recognizing a human face from the through image and extracting the recognized human face as a mask. Also good.

Then, the area extraction unit 24 of the MPU 16 stores the mask image extracted from the through image in the buffer memory 19 in, for example, the Exif (Exchangeable Image File Format) format.
Next, in step S12, the MPU 16 calculates a mask evaluation value of the mask extracted in the previous step S11. In this case, for example, the MPU 16 calculates a score of the mask using the feature amount used in the labeling process in the previous step S11 as an evaluation item, and calculates a mask evaluation value based on the total value of the calculated scores. In the example shown in FIG. 7, three masks a3, b3, and c3 are extracted from the through image. Then, as shown as “measured value 3” in FIG. 8A, numerical values “4”, “5”, and “3” are calculated as mask evaluation values of these masks a3, b3, and c3, respectively. Yes. In the example shown as “measurement value 4” in FIG. 8B, the mask evaluation values of the three masks a4, b4, and c4 included in the through image are numerical values “3”, “4”, and “2”. Are calculated respectively. Then, the MPU 16 stores the calculated mask evaluation value in the buffer memory 19 as additional information of the mask image.

  Subsequently, in step S13, the position acquisition unit 25 of the MPU 16 acquires the mask distance based on the defocus amount of the mask extracted from the through image in the previous step S11 as the position acquisition step. The distance is stored in the buffer memory 19 as additional information of the mask image. The defocus amount of the mask is calculated based on the evaluation value of the longitudinal chromatic aberration at the edge of the mask as described in, for example, Japanese Patent Application Laid-Open No. 2011-186442.

Next, in step S14, the area extracting unit 24 of the MPU 16 performs a mask separation / integration processing routine extracted in the previous step S11.
Details of the mask separation / integration processing routine will be described with reference to the flowchart shown in FIG.

  As shown in FIG. 3, in step S40, the region extracting unit 24 of the MPU 16 determines whether there is a mask including a plurality of small regions having different distances in the mask extracted in the previous step S11. . Specifically, the region extraction unit 24 of the MPU 16 determines whether there is an edge having a different distance from the other edges in the mask edge. If the area extracting unit 24 of the MPU 16 determines that there is a mask including a plurality of small areas with different distances (step S40 = YES), the process proceeds to step S41.

  Next, in step S41, the area extracting unit 24 of the MPU 16 divides the mask determined to include a plurality of small areas in the previous step S40 into a plurality of small areas, and each of the divided small areas is newly set. Extract as a mask. Then, the area extracting unit 24 of the MPU 16 stores the newly extracted mask image in the buffer memory 19 in, for example, the Exif format. Further, the area extracting unit 24 of the MPU 16 acquires the distance of the small area acquired in the previous step S13 as the distance of the new mask to which each corresponds. Further, the area extracting unit 24 of the MPU 16 stores the acquired new mask distance in the buffer memory 19 as additional information of the mask, and then proceeds to step S42.

  On the other hand, if the area extraction unit 24 of the MPU 16 determines that there is no mask including a plurality of small areas with different distances in the previous step S40 (step S40 = NO), the process is performed without performing the process of step S41. Control goes to step S42.

  In step S42, the area extracting unit 24 of the MPU 16 determines whether there is a combination of masks that are adjacent in the through image and close to each other in the mask extracted in the previous step S11. judge. Specifically, the area extracting unit 24 of the MPU 16 first calculates the centroid position of the mask extracted in the previous step S11, and extracts a combination of masks in which the distance between the centroid positions in the through image is less than the threshold value. To do. Then, the region extraction unit 24 of the MPU 16 calculates the difference in the distance between the masks in the extracted combination, and determines whether or not the calculated difference is less than the threshold value.

  If the area extraction unit 24 of the MPU 16 determines that there is a combination of masks that are adjacent to each other in the through image and are close in distance (step S42 = YES), the determined combination of masks is included in the through image. It is determined that the areas are included in the same subject, and the process proceeds to step S44.

  On the other hand, if the region extraction unit 24 of the MPU 16 determines that there is no combination of masks that are adjacent to each other in the through image and are close in distance (step S42 = NO), the process proceeds to step S43.

  In step S43, the region extracting unit 24 of the MPU 16 determines whether there is a combination of masks in which the motion vectors are close and the distances are close in the mask extracted in the previous step S11. Specifically, the area extraction unit 24 of the MPU 16 first detects the position of the mask extracted in the previous step S11 from the through image frame image repeatedly photographed by the image sensor 13 at every photographing cycle. . Subsequently, the area extraction unit 24 of the MPU 16 calculates the amount of change in the position of the mask in the previous and next frame images as a motion vector. Further, the region extraction unit 24 of the MPU 16 extracts a combination of masks whose motion vector difference is less than the threshold value from the masks extracted in the previous step S11. Then, the region extraction unit 24 of the MPU 16 calculates the difference in the distance between the masks in the extracted combination, and determines whether or not the calculated difference is less than the threshold value.

  When the region extraction unit 24 of the MPU 16 determines that there is a combination of masks in which the motion vectors are close and the distances are close (step S43 = YES), the determined combination of masks is applied to the same subject in the through image. It is determined that the area is included, and the process proceeds to step S44.

  In step S44, the area extracting unit 24 of the MPU 16 integrates the combination masks determined to be areas included in the same subject in the through image in the previous step S42 or the previous step S43. A new mask is extracted. Then, the area extracting unit 24 of the MPU 16 stores the newly extracted mask image in the buffer memory 19 in, for example, the Exif format. Further, the area extracting unit 24 of the MPU 16 acquires the average value of the distances of the plurality of masks before integration acquired in the previous step S13 as the integrated new mask distance. Further, the area extracting unit 24 of the MPU 16 stores the acquired new mask distance in the buffer memory 19 as additional information of the mask, and then ends the mask separation / integration processing routine shown in FIG. The processing returns to the focus control processing routine shown in FIG.

  On the other hand, when the region extraction unit 24 of the MPU 16 determines that there is no combination of masks in which the motion vectors are close and the distances are close (step S43 = NO), the processing of the mask shown in FIG. The separation / integration processing routine is terminated, and the processing returns to the focus control processing routine shown in FIG.

  Then, as shown in FIG. 2, in step S15, the position acquisition unit 25 of the MPU 16 calculates a focus map based on the mask distance acquired in the previous step S13, step S41, or step S43. Here, the focus map means a distribution of mask distances calculated based on the defocus amount of each mask included in the through image. The distribution of the mask distance means a variation in the distance of each mask in the entire mask, and is defined by the relative positional relationship of a plurality of masks included in the through image.

  Specifically, the position acquisition unit 25 of the MPU 16 first divides the mask distance from the near side to the infinity side in the depth direction of the through image into a plurality of distance ranges. In the present embodiment, these distance ranges are respectively “proximity area”, “close-range area”, “middle scene area”, “distant view area”, and “infinite distance area” in order from the close side in the depth direction of the through image. Is set. And the position acquisition part 25 of MPU16 classify | categorizes the distance of each mask into these distance ranges divided | segmented in the depth direction of the through image.

  In the example shown in FIG. 5, the distances between the three masks a1, b1, and c1 included in the through image are all relatively small, and all the masks a1, b1, and c1 are close in the depth direction of the through image. It is located on the side. In this case, as shown as “measured value 1” in FIG. 6, the distances of all the masks a1, b1, and c1 are biased with respect to the “close range” that is the closest distance range from the camera 10. It is classified. In the example shown as “measured value 2” in FIG. 6, the distances of all the masks a2, b2, and c2 are biased with respect to the “infinity range” that is the farthest distance range from the camera 10. It is classified.

  Subsequently, in step S16, the mode setting unit 27 of the MPU 16 determines whether or not the image shooting mode can be determined based on the focus map calculated in the previous step S15. When the mode setting unit 27 of the MPU 16 determines that the image shooting mode can be determined based on the focus map calculated from the through image (step S16 = YES), the process proceeds to step S17. .

  Next, in step S17, the mode setting unit 27 of the MPU 16 sets the shooting mode determined in the previous step S16 as the image shooting mode in the camera 10. Then, the MPU 16 controls the setting of the camera 10 related to the focus adjustment (AF) setting, the exposure adjustment (AE) setting, and the white balance adjustment (AWB), and the process proceeds to step S18.

  Specifically, in the example shown as “measurement value 1” in FIG. 6, a focus map in which the distances of all the masks included in the through image are biased toward the “near area” is calculated from the through image. Therefore, the mode setting unit 27 of the MPU 16 determines that the camera 10 is photographing the subject at that time based on the focus map in a state where the distances of all the masks included in the through image are biased to the “close range”. Then, the “macro mode” is set as the image shooting mode in the camera 10.

  In the example shown as “measurement value 2” in FIG. 6, a focus map in which the distances of all the masks included in the through image are biased toward “infinity” is calculated from the through image. For this reason, the mode setting unit 27 of the MPU 16 assumes that the camera 10 is shooting a landscape at that time based on a focus map in which the distances of all the masks included in the through image are biased toward “infinity”. Judgment is made, and “landscape mode” is set as the image capturing mode of the camera 10.

  On the other hand, if the mode setting unit 27 of the MPU 16 determines that the image shooting mode cannot be determined based on the focus map calculated from the through image (step S16 = NO), the process of step S17 is not performed. Then, the process proceeds to step S18.

  In step S <b> 18, the focusing area setting unit 26 of the MPU 16 reads the database 30 stored in the nonvolatile memory 18. In this database 30, the distance distribution of the comparison element to be compared with the focus map calculated from the through image and the focusing mask to be focused from among the masks included in the through image are set. As a reference, a reference focusing element set from comparison elements is stored in association with each other.

  Specifically, in the distribution of the distances of the comparison elements shown as “reference distribution 1” in FIG. 9, the distances of the three comparison elements ra1, rb1, rc1 are “close area”, “closest area”, and “middle scene area”. Is located in each. In this distribution, of the three comparison elements ra1, rb1, and rc1, the comparison element rc1 located in the “middle scene area” that is relatively infinite in the depth direction of the through image is set as the reference focusing element. Has been.

  Further, in the distribution of the distances of the comparison elements shown as “reference distribution 2” in FIG. 9, the distances of the three comparison elements ra2, rb2, and rc2 are respectively positioned in the “closest area”, “middle scene area”, and “far scene area”. doing. Also, in this distribution, of the three comparison elements ra2, rb2, and rc2, the comparison element ra2 located in the “closest area” that is relatively close to the depth direction of the through image is set as the reference focusing element. Yes.

  Further, in the distribution of the distances of the comparison elements shown as “reference distribution 3” in FIG. 9, the distances of the three comparison elements ra3, rb3, and rc3 are “middle scene area”, “far scene area”, and “infinity area”, respectively. positioned. Also, in this distribution, among the three comparison elements ra3, rb3, and rc3, the “middle scene area” that is relatively located in the depth direction of the through image and the “infinity” that is relatively located on the infinity side. The comparison element rb3 positioned in the “far-field area” between the “far-field” is set as the reference focusing element.

  That is, in the present embodiment, the comparison element located closest to the comparison elements is not uniformly set as the reference focusing element, and the reference element among the comparison elements constituting the distribution according to the distribution of the comparison elements Various comparison elements are set as focusing elements. In the example shown in FIG. 9, the number of comparison elements constituting the distribution is three, but any number of comparison elements constituting the distribution can be adopted as long as it is plural.

  Next, in step S19, the focusing area setting unit 26 of the MPU 16 collates the focus map calculated in the previous step S15 with the distance distribution of the comparison element read in the previous step S18. I do.

Here, the details of the distribution matching processing routine will be described with reference to the flowchart shown in FIG.
As shown in FIG. 4, in step S60, the focusing area setting unit 26 of the MPU 16 configures the distance of each mask constituting the focus map calculated in the previous step S15 and the distribution read in the previous step S18. The difference with the distance of each comparison element is calculated.

  In the example shown in FIG. 7, the distances of the three masks a3, b3, and c3 included in the through image are discrete values, and these masks a3, b3, and c3 are spaced apart in the depth direction of the through image. Are arranged. In this case, as shown as “measured value 3” in FIG. 9, the distances of the three masks a3, b3, and c3 are located in the “near area”, “closest area”, and “middle scene area”, respectively. ing. Then, the focusing area setting unit 26 of the MPU 16 compares the focus map shown as “measured value 3” in FIG. 9 with the distance distribution of the comparison element shown as “reference distribution 1” in FIG. In step S60, the following processing is performed. That is, the in-focus area setting unit 26 of the MPU 16 calculates the difference between the mask distance and the comparison element distance for all the one-to-one combinations between the masks a3, b3, and c3 and the comparison elements ra1, rb1, and rc1. calculate.

  In step S61, the focusing area setting unit 26 of the MPU 16 extracts a combination of a mask and a comparison element that minimizes the difference in distance calculated in the previous step S60. In the above example in FIG. 9, the in-focus area setting unit 26 of the MPU 16 calculates the difference in distance between the three comparison elements ra1, rb1, and rc1 for each of the masks a3, b3, and c3. The comparison element that minimizes the difference in distance is extracted as a combination target with each of the masks a3, b3, and c3. In this case, the mask a3 located in the “near area” and the comparison element ra1 located in the “near area” are combined. Further, the mask b3 positioned in the “closest area” and the comparison element rb1 positioned in the “closest area” are combined. Further, the mask c3 located in the “middle scene area” and the comparison element rc1 located in the “middle scene area” are combined.

  Subsequently, in step S62, the focusing area setting unit 26 of the MPU 16 determines whether or not the difference in distance between the mask and the comparison element is less than the threshold for all combinations extracted in the previous step S61. . This threshold value is a reference value for determining whether or not there is a comparison element corresponding to each mask when collating the focus map with the distance distribution of the comparison element. In the above example in FIG. 9, the focus area setting unit 26 of the MPU 16 includes the comparison elements ra1, rb1, and rc1 that match the distance ranges for all the masks a3, b3, and c3. For all combinations extracted in step S61, it is determined that the difference in distance between the masks a3, b3, and c3 and the comparison elements ra1, rb1, and rc1 is less than the threshold value. When the focus area setting unit 26 of the MPU 16 determines that the difference in distance between the mask and the comparison element is less than the threshold value (step S62 = YES) for all combinations extracted in the previous step S61. Shifts the processing to step S63.

  In step S63, the focusing area setting unit 26 of the MPU 16 determines the distance of the comparison element that is the comparison target at that time in the distribution of the distance of the comparison element stored as the database 30 in the nonvolatile memory 18. Is determined to be collable with the focus map calculated in the previous step S15. In the above example in FIG. 9, the focus area setting unit 26 of the MPU 16 can collate the distance distribution of the comparison element indicated as “reference distribution 1” with the focus map indicated as “measured value 3”. Judge that there is. Thereafter, the focusing area setting unit 26 of the MPU 16 ends the distribution matching processing routine shown in FIG. 4 and returns the processing to the focusing control processing routine shown in FIG.

  On the other hand, when the focusing area setting unit 26 of the MPU 16 determines that the distance difference between the mask and the comparison element is not less than the threshold (step S = NO) for all combinations extracted in the previous step S62. Then, the process proceeds to step S64.

  In step S64, the focusing area setting unit 26 of the MPU 16 compares the distance distribution of all comparison elements stored as the database 30 in the nonvolatile memory 18 with the focus map calculated in the previous step S15. It is determined whether or not it has been read for. If the in-focus area setting unit 26 of the MPU 16 determines that the distribution of the distances of all the comparison elements stored in the nonvolatile memory 18 as the database 30 has not been read (step S64 = NO), the process To step S65.

  In step S65, the focusing area setting unit 26 of the MPU 16 compares the comparison element distance distribution stored in the nonvolatile memory 18 as the database 30 with the focus map calculated in the previous step S15. The distribution to be read out is changed. In the example shown in FIG. 9, the focus area setting unit 26 of the MPU 16 changes the distance distribution of the comparison elements read for comparison with the focus map from “reference distribution 1” to “reference distribution 2”. . Thereafter, the focus area setting unit 26 of the MPU 16 returns the process to step S60, and repeats the processes of step S60 to step S62, step S64, and step S65.

  On the other hand, the focusing area setting unit 26 of the MPU 16 reads the distance distributions of all the comparison elements stored as the database 30 in the nonvolatile memory 18 for comparison with the focus map calculated in the previous step S15. If it is determined that it has been issued (step S64 = YES), the process proceeds to step S66.

  Then, in step S66, the focusing area setting unit 26 of the MPU 16 adds the comparison element distance distribution stored as the database 30 in the nonvolatile memory 18 to the focus map calculated in the previous step S15. It is determined that there is no collable distribution.

  In the focus map shown as “measured value 4” in FIG. 9, three masks a4. The distances b4 and c4 are located in the “near area”, “middle scene area”, and “far scene area”, respectively. This distribution is compared with the distance distributions of all the comparison elements stored in the nonvolatile memory 18 (shown as “reference distribution 1”, “reference distribution 2”, and “reference distribution 3” in FIG. 9). Not possible. That is, any distribution stored in the non-volatile memory 18 is a distribution in which there are comparison elements having the same distance ranges for all the masks a4, b4, and c4 in the focus map shown as “measured value 4”. is not. Therefore, the focusing area setting unit 26 of the MPU 16 can collate with the focus map indicated as “measured value 4” in FIG. 9 in the distance distribution of the comparison elements stored as the database 30 in the nonvolatile memory 18. It is determined that there is no distribution. Thereafter, the focusing area setting unit 26 of the MPU 16 ends the distribution matching processing routine shown in FIG. 4 and returns the processing to the focusing control processing routine shown in FIG.

  Then, in step S20, the focusing area setting unit 26 of the MPU 16 is calculated from the through image in the distance distribution of the comparison elements stored as the database 30 in the nonvolatile memory 18 in the previous step S19. It is determined whether or not there is a distribution collated with respect to the focus map. If the in-focus area setting unit 26 of the MPU 16 determines that there is a distribution of distances of comparison elements collated with respect to the focus map calculated from the through image (step S20 = YES), the process is performed. The process proceeds to step S21.

  Next, in step S21, the focusing area setting unit 26 of the MPU 16 reads the reference focusing element from the distribution of the distances of the comparison elements collated with respect to the focus map in the previous step S19. In the focus map shown as “measured value 3” in FIG. 9, the comparison element rc1 is read as the reference focusing element in the distance distribution of the comparison element shown as “reference distribution 1” in FIG.

  Subsequently, in step S22, the focusing area setting unit 26 of the MPU 16 selects an image of a mask corresponding to the reference focusing element read in the previous step S21 from the masks constituting the focus map calculated from the through image. After the flag information is written as additional information, the process proceeds to step S24. That is, the focusing area setting unit 26 of the MPU 16 sets the flag as additional information of the mask image associated with the reference focusing element in the previous step S19 from among the masks constituting the focus map calculated from the through image. Write information. In the focus map indicated as “measured value 3” in FIG. 9, flag information is written in the additional information of the image of the mask c3 associated with the comparison element rc1 in the previous step S19.

  On the other hand, when the in-focus area setting unit 26 of the MPU 16 determines that there is no distribution of comparison element distances compared with the focus map calculated from the through image (step S20 = NO), the process is performed. Control goes to step S23.

  In step S23, the focusing area setting unit 26 of the MPU 16 writes the flag information to the additional information of the mask image based on the relationship between the mask distances included in the through image, and then performs the processing. Control goes to step S24. Specifically, for example, the focusing area setting unit 26 of the MPU 16 has an image of a mask having the shortest distance among masks included in the through image, that is, a mask positioned closest to the through image in the depth direction. Flag information is written to the additional information. In the focus map shown as “measured value 4” in FIG. 9, flag information is written in the additional information of the image of the mask a4 having the shortest distance among the masks a4, b4, and c4 constituting the focus map.

  Then, in step S24, the focusing area setting unit 26 of the MPU 16 adds the distance evaluation value to the mask evaluation value of the mask in which the flag information has been written in the previous step S22 or step S23. In the example shown as “measured value 3” in FIG. 8A, flag information is written in the additional information of the image of the mask c3 among the masks a3, b3, and c3 constituting the focus map. Therefore, a numerical value “5” is added as a distance evaluation value to “3” which is the mask evaluation value of the mask c3 calculated in the previous step S12. As a result, the evaluation value of the mask c3 is a numerical value “8”. In the example shown as “measured value 4” in FIG. 8B, flag information is written in the additional information of the image of the mask a4 out of the masks a4, b4, and c4 constituting the focus map. Therefore, a numerical value “5” is added as a distance evaluation value to “3” which is the mask evaluation value of the mask a4 calculated in the previous step S12. As a result, the evaluation value of the mask a4 is a numerical value “8”.

  Subsequently, in step S25, the focusing area setting unit 26 of the MPU 16 evaluates the mask calculated in the previous step S25 among the masks constituting the focus map calculated from the through image as the focusing area setting step. The mask having the highest value is set as a focusing mask to be focused in the through image. In the example shown as “measured value 3” in FIG. 8A, out of the masks a3, b3, and c3 constituting the focus map, the mask c3 to which the distance evaluation value is added in the previous step S24 is the focusing mask. Set as Further, in the example shown as “measurement value 4” in FIG. 8B, the mask a4 to which the distance evaluation value is added in the previous step S24 among the masks a4, b4, and c4 constituting the focus map is the focusing mask. Set as

  Next, in step S26, the MPU 16 determines whether or not the release button 32 is half-pressed. If the MPU 16 determines that the release button 32 is half-pressed (step S26 = YES), the process proceeds to step S27.

  In step S27, after performing contrast AF for the focus mask set in step S25 of the through image, the process proceeds to step S28. As a result, the through image is focused on the focusing mask.

  On the other hand, if the MPU 16 determines that the release button 32 is not half-pressed (step S26 = NO), the process proceeds to step S28 without passing through the process of step S27.

  In step S28, the MPU 16 determines whether or not a shooting instruction signal is input. This shooting instruction signal is input as an operation signal to the image processing engine 14 when the release button 32 is fully pressed. When the monitor 21 is a touch panel, a shooting instruction signal is input to the image processing engine 14 as an operation signal when the monitor 21 is touched for shooting an image. If the MPU 16 determines that the shooting instruction signal is not input (step S28 = NO), the MPU 16 returns the process to step S10, and repeats the processes of steps S10 to S28 until the shooting instruction signal is input. On the other hand, if the MPU 16 determines that the imaging instruction signal has been input (step S28 = YES), the process proceeds to step S29.

  In step S29, the MPU 16 stores the still image generated based on the pixel signal output from the image sensor 13 at that time in the nonvolatile memory 18 as a captured image.

  Next, the operation of the camera 10 configured as described above will be described below, particularly focusing on the operation when the MPU 16 sets a focusing mask to be focused from among the masks included in the through image. To do.

  In the present embodiment, the MPU 16 calculates a focus map as the distribution of the distances of the masks included in the through image, and the distribution of the distances of the comparison elements stored as the database 30 in the nonvolatile memory 18 as the calculated focus map. Compare with Then, the MPU 16 adds the distance evaluation value to the mask evaluation value of the mask corresponding to the comparison element set as the reference focusing element among the comparison elements in the distribution collated with the focus map.

  That is, the MPU 16 calculates a mask evaluation value as an index of the degree that a person is expected to pay attention to the mask included in the through image, and then mask evaluation values of some masks corresponding to the reference focusing element The distance evaluation value is added to.

  In particular, in the present embodiment, the MPU 16 sets a mask to which a distance evaluation value is added based on a focus map that means variation in the distance of each mask in the entire mask. Therefore, the MPU 16 does not necessarily set the mask having the highest mask evaluation value as the focusing mask, but sets the focusing mask in consideration of the relative positional relationship between the masks in the depth direction of the through image. .

  In the present embodiment, the distance distribution of the comparison elements stored as the database 30 in the nonvolatile memory 18 is set as a reference focusing element among the comparison elements constituting the distribution according to the distribution of the comparison elements. Various comparison elements are set. That is, the comparison element distance distribution stored as the database 30 in the non-volatile memory 18 does not uniformly set the comparison element located closest to the comparison elements as the reference focusing element.

  For this reason, the MPU 16 includes the focus map calculated from the through image in the through image depending on which of the distance distributions of the comparison elements stored as the database 30 in the nonvolatile memory 18 is collated. Various masks to be set as focusing masks are changed. Therefore, even when the MPU 16 extracts a plurality of masks as candidate regions that are expected to be noticed by a person in the through image, the mask located closest to the mask is selected as the focus mask. Is not set uniformly.

According to the first embodiment, the following effects can be obtained.
(1) A focusing mask is set based on the relative position in the depth direction of the through image between the masks included in the through image. Therefore, it is possible to set various areas to be focused on in the through image.

  (2) A focusing mask is set based on variations in the overall distance of the mask included in the through image. For this reason, it is possible to set more various areas to be focused in the through image.

  (3) The focus map calculated from the through image is collated with the distance distribution of the comparison elements stored as the database 30 in the nonvolatile memory 18. Therefore, it is possible to set various regions to be focused on in the through image in a manner suitable for the focus map calculated from the through image.

  (4) Since the distance of the mask is acquired based on the evaluation value of the axial chromatic aberration at the edge of the mask, the depth of the through image between the masks in the depth direction without moving the focus lens 11 to adjust the focus of the through image The relative position can be acquired.

  (5) When it is determined that a plurality of masks extracted from a through image are masks extracted from one subject by combining mask distance data with a mask extraction result from the through image These masks are integrated. Therefore, the accuracy of extracting each subject included in the through image as one mask is improved. Therefore, the relative position of the masks in the depth direction of the through image can be accurately acquired as the relative position of the subjects in the depth direction of the through image.

  (6) When it is determined that a small area extracted from a plurality of subjects is included in one mask extracted from a through image by combining mask distance data with a mask extraction result from the through image The mask is divided into a plurality of small regions. Therefore, the accuracy of extracting each subject included in the through image as one mask is improved. Therefore, the relative position of the masks in the depth direction of the through image can be accurately acquired as the relative position of the subjects in the depth direction of the through image.

(7) A shooting mode suitable for shooting an image can be automatically set based on the focus map calculated from the through image.
(8) When the focus map calculated from the through image shows a state where all the masks included in the through image are biased to the close side in the depth direction of the through image, the camera 10 is in a state of taking a close-up image of the subject. It is determined. Then, the shooting mode of the camera 10 can be automatically set to a macro mode that is a shooting mode suitable for the shooting.

  (9) When the focus map calculated from the live view indicates that all the masks included in the live view are biased toward the infinity side in the depth direction of the live view, the camera 10 is shooting a landscape. It is determined. Then, the shooting mode of the camera 10 can be automatically set to a landscape mode which is a shooting mode suitable for the shooting.

(Second Embodiment)
Next, a second embodiment will be described. The second embodiment is different from the first embodiment in that a focusing mask is set based on a distribution of combinations of mask distances and arrangements. Therefore, in the following description, a configuration that is different from the first embodiment will be mainly described, and a configuration that is the same as or equivalent to that of the first embodiment will be denoted by the same reference numeral, and redundant description will be omitted.

  As shown in FIG. 10, in the focus control processing routine executed by the MPU 16 of this embodiment, the processes in steps S100 to S103 are performed in steps S10, S11, S13, and steps in FIG. 2 in the first embodiment. It is the same as each process of S14.

  In step S104, the position acquisition unit 25 of the MPU 16 calculates the centroid position of the mask extracted in the previous step S101, and then the amount of deviation between the calculated centroid position and the center point of the view angle of the through image. Is acquired as the mask arrangement.

  In the example shown in FIGS. 11 and 12, the shift amounts Ra to Rf between the gravity center positions Ga to Gf of the six masks a21 to f21 included in the through image and the center point P of the view angle of the through image are the masks a21 to f21. Each is acquired as an arrangement. Then, the position acquisition unit 25 of the MPU 16 stores the acquired mask arrangement in the buffer memory 19 as additional information of the mask image.

  Subsequently, in step S105, the position acquisition unit 25 of the MPU 16 calculates a focus map based on the mask distance acquired in the previous step 102 and the mask arrangement acquired in the previous step S104. Here, the focus map of the present embodiment means a distribution of combinations of distances and arrangements of the respective masks included in the through image. In this case, the distribution of the combination of the distance and the arrangement of the mask means the variation of the combination of the distance and the arrangement of each mask in the entire mask. In the example shown in FIG. 12, in the coordinate system in which the horizontal axis is the amount of deviation from the center point of the angle of view and the vertical axis is the mask distance, the combinations of the distance and the arrangement of each mask are plotted in the coordinate system. In this case, the approximate curve S obtained from the plotted coordinates is calculated as the focus map.

  Next, in step S106, the mode setting unit 27 of the MPU 16 performs a mask distance included in the focus map calculated in the previous step S105 in the same manner as in step S16 of FIG. 2 in the first embodiment. Based on the distribution of the image, it is determined whether or not the image shooting mode can be determined. The mode setting unit 27 of the MPU 16 may determine whether or not the image capturing mode can be determined based on the distribution of the mask distance and the arrangement combination calculated as the focus map in the previous step S105. Good.

  When the mode setting unit 27 of the MPU 16 determines that the image shooting mode can be determined based on the focus map calculated from the through image (step S106 = YES), the process proceeds to step S107. .

  Then, in step S107, the mode setting unit 27 of the MPU 16 sets the shooting mode determined in the previous step S106 as the image in the camera 10 in the same manner as the process in step S17 of FIG. 2 in the first embodiment. Is set as the shooting mode, and the process proceeds to step S108.

  On the other hand, if the mode setting unit 27 of the MPU 16 determines that the shooting mode of the image cannot be determined based on the focus map calculated from the through image (step S106 = NO), the process of step S107 is not performed. Then, the process proceeds to step S108.

  In step S108, the focusing area setting unit 26 of the MPU 16 selects the first mask group having a relatively small deviation amount from the center point of the angle of view of the through image among the masks extracted in the previous step S101. A mask group is set as a mask group, and a mask group having a relatively large shift amount from the center point of the angle of view of the through image is set as the second mask group. In the example shown in FIGS. 11 and 12, among the six masks a21 to f21 included in the through image, the masks a21 to c21 are set as the first mask group, and the masks d21 to f21 are the second mask group. Set as

  Then, in step S109, the focusing area setting unit 26 of the MPU 16 calculates the average value T1 and the average value T2 of the distances of the first mask group and the second mask group set in the previous step S108, respectively. In the example shown in FIG. 12, the average distance T1 of the first mask group is calculated as “(La + Lb + Lc) / 3”, and the average distance T2 of the second mask group is “(Ld + Le + Lf) / 3”. Is calculated.

  Subsequently, in step S110, the focusing area setting unit 26 of the MPU 16 determines whether the average distance T1 of the first mask group calculated in the previous step S109 is greater than the average distance T2 of the second mask group. Determine whether or not. When the focusing area setting unit 26 of the MPU 16 determines that the average distance T1 of the first mask group is larger than the average distance T2 of the second mask group (step S110 = YES), The process proceeds to step S111.

  In step S111, the focusing area setting unit 26 of the MPU 16 sets the mask having the longest distance among the first mask groups set in the previous step S108 as the focusing mask to be focused in the through image. Set. That is, when the average distance T1 of the first mask group is larger than the average distance T2 of the second mask group, the focusing area setting unit 26 of the MPU 16 sets the mask arrangement pattern included in the through image. Then, it is determined that the “perspective pattern” is continuously arranged so that the distance of the mask gradually increases from the peripheral side to the center side of the angle of view of the through image. Then, the in-focus area setting unit 26 of the MPU 16 sets, as an in-focus mask, the innermost mask among the masks included in the through image as an area that is expected to be noticed by a person in the “perspective pattern”.

  On the other hand, if the focusing area setting unit 26 of the MPU 16 determines that the average distance T1 of the first mask group is equal to or less than the average distance T2 of the second mask group (step S110 = NO), The process proceeds to step S112.

  In step S112, the focusing area setting unit 26 of the MPU 16 sets the focusing mask based on the relationship between the mask distances included in the through image. Specifically, for example, the focusing area setting unit 26 of the MPU 16 aligns a mask having the smallest distance among the masks included in the through image, that is, a mask positioned closest to the through image in the depth direction. Set as a focus mask.

  Thereafter, in step S113 to step S116, the MPU 16 generates a captured image through the same processing as the processing in step S26 to step S29 in FIG. 2 in the first embodiment.

According to the second embodiment, in addition to the effects (1) and (4) to (9) of the first embodiment, the following effects can be obtained.
(10) A focusing mask is set based on the variation in the overall distance of the mask combined with the arrangement in the through image. For this reason, it is possible to set more various regions to be focused in the through image.

(Third embodiment)
Next, a third embodiment will be described. The third embodiment is different from the second embodiment in the manner of setting a focusing mask based on the distribution of combinations of mask distances and arrangements. Therefore, in the following description, a configuration that is different from the second embodiment will be mainly described, and a configuration that is the same as or equivalent to that of the second embodiment is denoted by the same reference numeral, and redundant description is omitted.

  As shown in FIG. 13, in the focus control processing routine executed by the MPU 16 of this embodiment, each processing in steps S200 to S203 is the same as each processing in steps S100 to S103 in FIG. 10 in the second embodiment. is there.

  In step S204, the position acquisition unit 25 of the MPU 16 calculates a focus map based on the mask distance and the mask arrangement acquired in the previous step 202. In this case, the position acquisition unit 25 of the MPU 16 acquires the centroid position of the mask in the through image as the mask arrangement, and calculates the focus map by combining the acquired centroid position with the mask distance.

  Then, in step S205, the mode setting unit 27 of the MPU 16 performs the mask distance included in the focus map calculated in the previous step S204 in the same manner as the process in step S106 of FIG. 10 in the second embodiment. It is determined whether or not the image shooting mode can be determined based on the distribution of. If the mode setting unit 27 of the MPU 16 determines that the image shooting mode can be determined based on the focus map calculated from the through image (step S205 = YES), the process proceeds to step S206. .

  Subsequently, in step S206, the mode setting unit 27 of the MPU 16 changes the shooting mode determined in the previous step S205 to the image mode in the camera 10 in the same manner as the processing in step S107 in FIG. 10 in the second embodiment. After setting as a photographing mode, the process proceeds to step S207.

  On the other hand, if the mode setting unit 27 of the MPU 16 determines that the shooting mode of the image cannot be determined based on the focus map calculated from the through image (step S205 = NO), the process of step S206 is not performed. Then, the process proceeds to step S207.

  In step S207, the focusing area setting unit 26 of the MPU 16 determines whether or not the number N of the masks arranged in the vertical direction among the masks extracted in the previous step S201 is three or more. In this case, as an example, the focusing area setting unit 26 of the MPU 16 has a deviation amount of the center of gravity position of the masks in the left-right direction that is less than a predetermined threshold value, and the distance between the contours of the masks in the vertical direction is a predetermined value. When it is less than the threshold, it is determined that the determination target masks are arranged in parallel vertically. If the in-focus area setting unit 26 of the MPU 16 determines that the number N of the masks arranged in the vertical direction is three or more (step S207 = YES), the process proceeds to step S208.

Next, in step S208, the focusing area setting unit 26 of the MPU 16 sets N masks arranged in the vertical direction as first to Nth masks in order from the lowermost side.
Subsequently, in step S209, the focusing area setting unit 26 of the MPU 16 sets, as an initial value, a number M indicating how many mask distances from the lowermost side of the masks arranged vertically are to be compared. Set M = “1”.

  Next, in step S210, the in-focus area setting unit 26 of the MPU 16 sets the distance of the M-th mask counted from the lowermost side among the masks arranged in the vertical direction to the (M + 1) th adjacent to the upper side. It is determined whether it is smaller than the mask distance. In other words, in the state where M = “1”, the focus area setting unit 26 of the MPU 16 determines whether the distance of the lowermost mask among the masks arranged in the vertical direction is smaller than the distance of the adjacent masks above it. Determine whether or not. When the focusing area setting unit 26 of the MPU 16 determines that the distance of the Mth mask is smaller than the distance of the adjacent mask above (step S210 = YES), the process proceeds to step S211. Transition.

  In step S211, the in-focus area setting unit 26 of the MPU 16 determines whether or not the relational expression “M <N−1” is satisfied. If the in-focus area setting unit 26 of the MPU 16 determines that the relational expression “M <N−1” is satisfied (step S211 = YES), the process proceeds to step S212.

Subsequently, in step S212, the focusing area setting unit 26 of the MPU 16 increments M to M + 1, and then returns the process to step S210.
Then, the focusing area setting unit 26 of the MPU 16 performs the process of step S <b> 210 again with M = “2”. In this case, the focusing area setting unit 26 of the MPU 16 determines whether or not the distance of the second mask from the lowermost side among the masks arranged in the vertical direction is smaller than the distance of the adjacent mask above it.

  In other words, the focusing area setting unit 26 of the MPU 16 increments M as long as the condition that the distance of the Mth mask is smaller than the distance of the adjacent mask above it is satisfied. Each time M is incremented, the mask combination to be compared among the masks arranged in the vertical direction is shifted upward one by one from the lowest end side.

  Then, the in-focus area setting unit 26 of the MPU 16 does not satisfy the condition “M <N−1” while satisfying the condition in the previous step S210 (step S211 = NO). It is determined that the comparison of the distances for the combination of masks has been completed, and the process proceeds to step S213.

  Next, in step S213, the focusing area setting unit 26 of the MPU 16 sets the Nth mask, which is the uppermost mask among the masks arranged in the vertical direction, as the focusing mask to be focused in the through image. Then, the process proceeds to step S217. That is, if the focus area setting unit 26 of the MPU 16 satisfies the conditions in the previous step S210 for all the masks arranged in parallel vertically, the arrangement pattern of the masks included in the through image is displayed on the pedestal. It is determined that the placement pattern is placed. Then, the focusing area setting unit 26 of the MPU 16 assumes that the uppermost mask among the masks arranged in the vertical direction in this arrangement pattern is an object placed on the pedestal, and aligns the uppermost mask. Set as a focus mask.

  In the example shown in FIG. 14A, three masks a31 to c31 included in the through image are vertically arranged in parallel, and the distance between these masks a31 to c31 is the lowermost mask a31 and the center mask. The size gradually increases in the order of b31 and the uppermost mask c31. Therefore, the arrangement pattern of these masks a31 to c31 is determined to be an arrangement pattern in which an object is placed on the pedestal, and the uppermost mask c31 among these masks a31 to c31 is set as the focusing mask. Is done.

  On the other hand, the in-focus area setting unit 26 of the MPU 16 does not satisfy the condition in the previous step S210 (step S210 = NO) before the condition “M <N−1” in the previous step S211 is not satisfied. Then, the process proceeds to step S214.

  In step S214, the focusing area setting unit 26 of the MPU 16 sets the Mth to Nth masks as candidates for the focusing mask among the masks arranged vertically. That is, the focus area setting unit 26 of the MPU 16 sets the arrangement pattern of the masks included in the through image on the basis that some of the masks arranged in the vertical direction satisfy the condition in the previous step S210. It is estimated that there is a possibility of an arrangement pattern in which an object is placed on a pedestal. Therefore, the in-focus area setting unit 26 of the MPU 16 uses the Mth to Nth masks positioned on the upper end side among the masks arranged vertically as mask candidates that may be objects placed on the pedestal. Set.

  In the example shown in FIG. 14B, the three masks a32 to c32 included in the through image are arranged in parallel vertically, and the distance between these masks a32 to c32 is the lowermost mask a32 and the uppermost mask. The size gradually increases in the order of the mask c32 and the central mask b32. Therefore, although the distance of the lowermost mask a32 is smaller than the central mask b32, the distance of the central mask b32 is larger than the uppermost mask c32. Therefore, among the masks a32 to c32 arranged in parallel vertically, the center mask b32 and the uppermost mask c32 are set as candidates for the focus mask.

  Then, the focusing area setting unit 26 of the MPU 16 determines whether or not a focusing mask is selected from candidate masks. Here, whether or not the focus mask is selected is determined based on whether or not the focus mask is selected and determined through a pressing operation of a select button or a determination button constituting the operation unit 33, for example. Whether or not the focus mask is selected may be determined based on whether or not the focus mask is selected and determined through a touch operation on the monitor 21 when the monitor 21 is a touch panel. If the in-focus area setting unit 26 of the MPU 16 determines that the in-focus mask is not selected (step S214 = NO), the process in step S214 is periodically repeated until the in-focus mask is selected. . On the other hand, when the focusing area setting unit 26 of the MPU 16 determines that the focusing mask has been selected (step S214 = YES), the process proceeds to step S215.

  In step S215, the focusing area setting unit 26 of the MPU 16 sets the mask selected in the previous step S215 as the focusing mask, and then shifts the processing to step S217.

  If the focusing area setting unit 26 of the MPU 16 determines that the number N of masks arranged in parallel vertically is less than three (step S207 = NO), the process proceeds to step S216.

  In step S216, the focusing area setting unit 26 of the MPU 16 sets the focusing mask based on the relationship between the mask distances included in the through image. Specifically, for example, the focusing area setting unit 26 of the MPU 16 aligns a mask having the smallest distance among the masks included in the through image, that is, a mask positioned closest to the through image in the depth direction. Set as a focus mask.

  Thereafter, in step S217 to step S220, the MPU 16 generates a captured image through processes similar to the processes in steps S113 to S116 of FIG. 10 in the second embodiment.

According to the third embodiment, the effects (1) and (4) to (10) of the above embodiments can be obtained.
(Fourth embodiment)
Next, a fourth embodiment will be described. The fourth embodiment is different from the first embodiment in that the focusing mask is set without matching the distance distribution of the mask with the distance distribution of the comparison element. Therefore, in the following description, a configuration that is different from the first embodiment will be mainly described, and a configuration that is the same as or equivalent to that of the first embodiment will be denoted by the same reference numeral, and redundant description will be omitted.

  As shown in FIG. 15, in the focus control processing routine executed by the MPU 16 of this embodiment, each processing of Step S300 to Step S306 is performed in Step S10, Step S11, and Step S13 to Step of FIG. 2 in the first embodiment. It is the same as each process of S17.

  In step S307, the in-focus area setting unit 26 of the MPU 16 determines whether or not the focus map calculated in the previous step S304 includes a mask whose distance is significantly different from other masks. In this case, as an example, the focusing area setting unit 26 of the MPU 16 determines whether or not the masks constituting the focus map include a mask whose distance difference from all other masks is equal to or greater than a threshold value. To do.

  In the example shown in FIGS. 16 and 17, among the eight masks a41 to h41 included in the through image, the distances of the seven masks a41 to g41 are classified as “proximity region” or “closest region”. The distance of the other one mask h41 is classified as a “far view area”. Then, the mask h41 classified as “distant view area” is determined as a mask whose distance is greatly deviated from the other masks a41 to g41 classified as “close area” or “closest area”.

  If the in-focus area setting unit 26 of the MPU 16 determines that it includes a mask whose distance is significantly different from that of other masks (step S307 = YES), the process proceeds to step S308.

  Next, in step S308, the focusing area setting unit 26 of the MPU 16 sets a mask determined to have a large distance compared to other masks in the previous step S307 as a focusing mask. In the example shown in FIGS. 16 and 17, the mask h41 classified as “distant view area” is set as the focusing mask.

  On the other hand, if the in-focus area setting unit 26 of the MPU 16 determines that it does not include a mask whose distance is significantly different from other masks (step S307 = NO), the process proceeds to step S309.

  In step S309, the focusing area setting unit 26 of the MPU 16 sets the focusing mask based on the relationship between the mask distances included in the through image. Specifically, for example, the focusing area setting unit 26 of the MPU 16 aligns a mask having the smallest distance among the masks included in the through image, that is, a mask positioned closest to the through image in the depth direction. Set as a focus mask.

  Thereafter, in step S310 to step S313, the MPU 16 generates a captured image through the same processing as the processing in step S26 to step S29 in FIG. 2 in the first embodiment.

According to the fourth embodiment, in addition to the effects (1), (2), and (4) to (9) of the first embodiment, the following effects can be obtained.
(11) Among the masks extracted from the through image, a mask whose distance is out of comparison with other masks is set as a focusing mask. For this reason, it is possible to suitably set a region that is expected to be noticed by a person in the through image as a region to be focused in the through image.

In addition, you may change said each embodiment into another embodiment as follows.
In the first embodiment, the focusing area setting unit 26 of the MPU 16 approximates the distribution by comparing the focus map calculated from the through image with the distance distribution of the comparison elements stored in the nonvolatile memory 18. The degree may be calculated, and when the calculated degree of approximation is equal to or greater than a predetermined threshold, it may be determined that the focus map and the distance distribution of the comparison element have been collated.

  In the second embodiment, the focusing area setting unit 26 of the MPU 16 may determine whether or not the arrangement pattern of the masks a21 to f21 is a “perspective pattern” as follows.

  That is, as a first modification example, the focusing area setting unit 26 of the MPU 16 determines the distance between masks adjacent to each other in the amount of deviation from the center point of the angle of view of the through image among the masks a21 to f21 included in the through image. May be calculated, and it may be determined whether the calculated difference is constant or not.

  As a second modification, the focusing area setting unit 26 of the MPU 16 compares the focus map calculated from the through image with the distribution of combinations of distances and arrangements of the comparison elements stored in the nonvolatile memory 18. The degree of approximation of the distribution may be calculated using the above, and it may be determined whether or not the calculated degree of approximation is equal to or greater than a predetermined threshold.

  In the second embodiment, the position acquisition unit 25 of the MPU 16 masks the amount of deviation between the barycentric positions Ga to Gf of the masks a21 to f21 included in the through image and the point set at an arbitrary position in the through image. You may acquire as arrangement | positioning of a21-f21, respectively.

  In the third embodiment, the focusing area setting unit 26 of the MPU 16 has a shift amount of the center of gravity position between the masks in the left-right direction that is less than the threshold value, and a shift amount of the center of gravity position between the masks in the vertical direction. If is less than the threshold value, it may be determined that the determination target masks are arranged in parallel vertically.

  In the third embodiment, the MPU 16 is configured such that, with respect to some of the masks arranged in the vertical direction, for some masks, the distance between the lower masks among the masks adjacent in the vertical direction is greater than the distance between the upper masks If the above condition is satisfied, a plurality of captured images may be generated, each focused on a mask candidate that may be an object placed on the pedestal.

  In the third embodiment, the focusing area setting unit 26 of the MPU 16 determines whether the mask arrangement pattern is an arrangement pattern in which an object is placed on the pedestal as follows. Also good. In other words, the focusing area setting unit 26 of the MPU 16 calculates the degree of distribution approximation by comparing the focus map calculated from the through image with the distribution of combinations of distances and arrangements of the comparison elements stored in the nonvolatile memory 18. Then, it may be determined whether the calculated degree of approximation is equal to or greater than a predetermined threshold.

  In each of the above embodiments, the position acquisition unit 25 of the MPU 16 may acquire the mask distance using a detection method such as a phase difference detection method, a contrast detection method, and a TOF (Time Of Flight) method.

In this configuration, when the phase difference detection method is used as the detection method, the mask distance can be quickly acquired.
When the contrast detection method is used as the detection method, the mask distance can be acquired even when the mask is located at an arbitrary position in the through image.

  In each of the above embodiments, the position acquisition unit 25 of the MPU 16 may acquire the mask distance as follows. That is, the position acquisition unit 25 of the MPU 16 first stores in the nonvolatile memory 18 a defocus database in which the degree of blur of the area and the distance of the area are stored in association with each other. Then, the position acquisition unit 25 of the MPU 16 acquires the mask distance by comparing the degree of blur of the mask in the through image with the degree of blur of the area stored in the defocus database.

In this configuration, the distance of the mask can be acquired without moving the focus lens 11 for adjusting the focus of the through image.
In each of the above embodiments, the focus map calculated from the through image is configured by segment data indicating which distance range is included in a plurality of distance ranges, instead of numerical data, for each mask. Also good.

According to this configuration, the data amount of the focus map can be reduced as compared with the case where the focus map is configured by numerical data of distances for each mask.
In each of the above embodiments, the MPU 16 may control the in-focus state of the moving image by executing a process similar to the focusing control processing routine in each embodiment during shooting of the moving image.

  In each of the above-described embodiments, the image processing engine 14 may be mounted as an image processing apparatus that sets an in-focus area in an image with respect to an electronic device other than the camera 10 having an image shooting function. Note that examples of the electronic device having an image shooting function include a video camera, a personal computer, a mobile phone, and a portable game machine.

DESCRIPTION OF SYMBOLS 10 ... Digital camera as an example of an electronic camera, 13 ... Image sensor, 14 ... Image processing engine as an example of an image processing device, 24 ... Area extraction part, 25 ... Position acquisition part, 26 ... Focus area setting part, 27 ... mode setting unit, 30 ... database, a1 to c1, a2 to c2, a3 to c3, a4 to c4, a21 to f21, a31 to c31, a32 to c32, a41 to h41 ... mask as an example of a specific region, ra1 ~ Rc1, ra2-rc2, ra3-rc3 ... comparison elements.

Claims (11)

An area extraction unit that extracts a plurality of specific areas from an image acquired from the outside;
A position acquisition unit that acquires a relative position in the depth direction of the images in the specific regions extracted by the region extraction unit;
Based on the relative positions in the depth direction of the images between the specific areas acquired by the position acquisition unit, an in-focus area to be focused is set from the specific areas extracted by the area extraction unit. An image processing apparatus comprising: a focal region setting unit. The position acquisition unit acquires a distance distribution of the specific area as a relative position in the depth direction of the images between the specific areas,
The image processing apparatus according to claim 1, wherein the focus area setting unit sets the focus area based on a distribution of distances of the specific area acquired by the position acquisition unit. The position acquisition unit acquires the distance distribution of the specific area in combination with the arrangement of the specific area constituting the distribution in the image,
The focus area setting unit sets the focus area based on a distance distribution of the specific area acquired by the position acquisition unit in combination with an arrangement of the specific area in the image. The image processing apparatus according to claim 2. In the database, the distance distribution of the comparison element to be compared with the distance distribution of the specific area acquired by the position acquisition unit, and the reference when the focusing area setting unit sets the focusing area A reference focusing element set from among the comparison elements constituting the distribution is stored in association with each other,
The in-focus area setting unit is collated in the database by collating the distance distribution of the specific area acquired by the position acquisition unit with the distance distribution of the comparison element stored in the database. The image processing apparatus according to claim 2, wherein the focus area is set based on the reference focus element associated with the distribution. In the database, the arrangement of the comparison elements is stored in combination with the distance distribution of the comparison elements, and
The in-focus area setting unit includes the distance and arrangement of the comparison elements stored in the database in combination with the distribution of the distance of the specific area acquired by the position acquisition unit in the arrangement of the specific area in the image. 5. The image according to claim 4, wherein the in-focus area is set based on the reference focus element associated with the distribution collated in the database by collating with a distribution of combinations of Processing equipment.   The focusing area setting unit is configured such that the distribution of the distance of the specific area acquired by the position acquisition unit is a distribution in which some of the specific areas are out of the distance compared to other specific areas. The image processing apparatus according to claim 2, wherein the focus area is set from the partial specific area.   The distance distribution of the specific area includes the distance for each specific area in a manner classified into a plurality of distance ranges divided in the depth direction of the image. The image processing device according to any one of the above. The position acquisition unit detects the distance of the specific region for each specific region based on the color component amount of the edge of the specific region extracted by the region extraction unit, thereby the depth of the image between the specific regions. The image processing apparatus according to claim 1, wherein a relative position in a direction is acquired.   The position acquisition unit acquires a relative position in the depth direction of the image between the specific regions by detecting the distance between the specific regions extracted by the region extraction unit for each specific region using a phase difference detection method. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus.   The position acquisition unit acquires a relative position in the depth direction of the image between the specific regions by detecting the distance between the specific regions extracted by the region extraction unit for each specific region using a contrast detection method. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. On the computer,
An area extraction step for extracting a plurality of specific areas from an image acquired from the outside;
A position acquisition step of acquiring a relative position in the depth direction of the images in the specific areas extracted in the area extraction step;
Based on the relative positions in the depth direction of the images in the specific areas acquired in the position acquisition step, an in-focus area to be focused is set from the specific areas extracted in the area extraction step. An image processing program that executes a focal region setting step.