JP2021180407A - Electronic apparatus - Google Patents

Abstract

To provide an electronic apparatus that allows a user to correct evaluation values easily as intended.SOLUTION: An electronic apparatus of the present invention has: acquisition means that acquires the distribution of evaluation values of an image; selection means that selects a first area corresponding to part of the image in accordance with a first user operation and selects a second area corresponding to part of the image in accordance with a second user operation; and processing means that performs predetermined processing by using a value based on an evaluation value of the second area selected by the selection means as an evaluation value of the first area selected by the selection means.SELECTED DRAWING: Figure 3

Description

The present invention relates to an electronic device, and more particularly to a technique for correcting (correcting) an evaluation value distribution (evaluation value map) such as a distance distribution corresponding to an image.

Image processing is known in which an image is edited (processed) based on a distance distribution recorded together with an image as an evaluation value distribution (evaluation value map). For example, in the background blur function used in smartphones and the like, the background can be blurred by applying a blur filter to the background (area other than the main subject). However, if there is an area where the acquisition of the distance (distance information) has failed, an unnatural image will be generated. Although there are applications that correct (correct) distance information with the brush tool, detailed operations using the brush tool on a small screen such as a smartphone screen are a burden on the user.

Patent Document 1 proposes a device that divides an image region using image information and distance information, and corrects the distance information of a specific region based on the distance information of another region.

JP-A-2017-117030

However, in the apparatus proposed in Patent Document 1, since the region is automatically selected by determining whether or not the main subject exists over a plurality of regions, the distance information is not corrected as intended by the user. Sometimes.

Therefore, the present invention provides an electronic device that allows a user to easily and intentionally correct an evaluation value.

The electronic device of the present invention selects an acquisition means for acquiring an evaluation value distribution of an image and a first region corresponding to a part of the image according to a first user operation, and responds to a second user operation. As an evaluation value of the selection means for selecting the second region corresponding to a part of the image and the first region selected by the selection means, the evaluation value of the second region selected by the selection means. It is characterized by having a processing means for performing a predetermined process using a value based on.

According to the present invention, the user can easily and intentionally correct the evaluation value.

It is a block diagram which shows an example of the functional structure of the camera which concerns on Example 1. FIG. It is a figure which shows an example of the image and the distance map which concerns on Example 1. FIG. It is a flowchart which shows an example of the correction process which concerns on Example 1. FIG. It is a figure which shows the display example which concerns on Example 1. FIG. It is a figure which shows an example of the area extraction result which concerns on Example 1. FIG. It is a flowchart which shows an example of the area extraction which concerns on Example 1. FIG. It is a figure which shows an example of the grouping which concerns on Example 1. FIG. It is a figure which shows an example of the processing which concerns on Example 2. FIG. It is a flowchart which shows an example of the correction process which concerns on Example 2. It is a figure which shows the display example which concerns on Example 2. FIG. It is a flowchart which shows an example of the correction process which concerns on Example 3. FIG. It is a figure which shows the display example which concerns on Example 3. FIG. It is a figure which shows an example of the distance group which concerns on Example 3. FIG.

<Example 1>
Hereinafter, Example 1 of the present invention will be described in detail with reference to the drawings. In the following, a camera (digital camera) which is an example of an electronic device to which the present invention is applied will be described. In the first embodiment, an example in which the evaluation value (for example, distance (distance information)) of a region selected by the user is easily corrected (corrected) based on the evaluation value of an arbitrary region will be described.

FIG. 1 is a block diagram showing an example of the functional configuration of the camera 100 according to the first embodiment. The camera 100 has a system control unit 101, a ROM 102, and a RAM 103. Further, the camera 100 includes an optical system 104, an image pickup unit 105, an A / D conversion unit 106, an image processing unit 107, a recording medium 108, an area extraction unit 109, an image processing unit 110, a display unit 111, an operation unit 112, and the like. It has a bus 113. System control unit 101, ROM 102, RAM 103, image pickup unit 105, A / D conversion unit 106, image processing unit 107, recording medium 108, area extraction unit 109, image processing unit 110, display unit 111, and operation unit 112, respectively. Is connected to bus 113.

The system control unit 101 is a control unit that controls the entire camera 100, and is, for example, a CPU. The system control unit 101 controls the operation of each block included in the camera 100 by reading the operation program of each block included in the camera 100 from the ROM 102, expanding the program into the RAM 103, and executing the program.

The ROM 102 is a non-volatile memory in which stored data (information) can be rewritten, and is, for example, a flash ROM. The ROM 102 stores an operation program of each block included in the camera 100, parameters necessary for the operation of each block, and the like.

The RAM 103 is a volatile memory in which stored data (information) can be rewritten, and is used as a temporary storage area for data output in the operation of each block included in the camera 100. The system control unit 101 and the image processing unit 107 use the RAM 103 as a work memory.

The optical system 104 forms a subject image (optical image of the subject) on the image pickup unit 105. The optical system 104 includes, for example, a fixed lens, a variable magnification lens for changing the focal length, a focus lens for adjusting the focus, and the like. The optical system 104 also includes a diaphragm, and the amount of light at the time of photographing is adjusted by adjusting the aperture diameter of the optical system 104 using the diaphragm.

The image pickup unit 105 is an image pickup element such as a CCD image sensor or a CMOS image sensor. The image pickup unit 105 photoelectrically converts an optical image formed on the image pickup unit 105 by the optical system 104 to obtain an analog image signal. The image pickup unit 105 outputs the obtained analog image signal to the A / D conversion unit 106.

The A / D conversion unit 106 applies an A / D conversion process to the input analog image signal to obtain digital image data. The A / D conversion unit 106 outputs the obtained digital image data to the RAM 103 and records it.

The image processing unit 107 performs image processing on the image data stored in the RAM 103. For example, the image processing unit 107 performs various image processing such as white balance adjustment, color interpolation, and reduction / enlargement. The image processing unit 107 can also acquire (generate) an evaluation value distribution (evaluation value map) of the image from the target image (image data). The image processing unit 107 records the image after image processing in the RAM 103 or the recording medium 108.

The recording medium 108 is, for example, a storage unit such as a memory card that can be attached to and detached from the camera 100. The recording medium 108 records an image processed by the image processing unit 107, an image processed by the image processing unit 110, an image A / D converted by the A / D conversion unit 106, and the like as recorded images. ..

The area extraction unit 109 selects (extracts) an area corresponding to a part of the image processed by the image processing unit 107 according to the user operation. For example, when there is a user operation, the area extraction unit 109 extracts an area based on at least one of the pixel value distribution (color and luminance distribution) of the image and the evaluation value map. Here, the pixel value distribution is a pixel value distribution of an image processed by the image processing unit 107, a pixel value distribution of an image A / D converted by the A / D conversion unit 106, and the like.

The image processing unit 110 processes the image processed by the image processing unit 107 and the image converted to A / D by the A / D conversion unit 106 using the evaluation value map. Image processing is, for example, background blurring, rewriting, and the like. The image processing unit 110 records the processed image in the RAM 103 or the recording medium 108.

The display unit 111 is, for example, a liquid crystal monitor, and displays an image according to a user operation. For example, the display unit 111 may use an image stored in a RAM 103, a recording medium 108, or the like (an image processed by the image processing unit 107, an image processed by the image processing unit 110, or an A / D by the A / D conversion unit 106. The D-converted image, the image recorded on the recording medium 108, and the like are displayed.

The operation unit 112 receives a user operation (instruction from the user) for the camera 100. For example, the operation unit 112 includes a touch panel provided on the display surface of the display unit 111. By operating the operation unit 112, the user can input various settings and various information of the camera 100 to the camera 100.

The bus 113 connects a plurality of blocks of the camera 100 to each other. The plurality of blocks of the camera 100 can exchange signals via the bus 113.

In the first embodiment, the image processing unit 107 generates a defocus amount distribution representing the distribution of focus information on one surface of the image as the evaluation value distribution. Hereinafter, the focus amount distribution will be referred to as a distance map. The method for calculating the defocus amount (focus information; distance information) is not particularly limited, but for example, the method described in Japanese Patent Application Laid-Open No. 2008-15754 (a method for acquiring focus information on one surface of an image from the phase difference of an image) is used. The amount of defocus can be calculated. In the method described in Japanese Patent Application Laid-Open No. 2008-15754, the defocus amount is calculated based on the amount of shift between the two pixel data while relatively shifting the pair of pixel data in the minute block.

FIG. 2A shows an image 201 after processing by the image processing unit 107. In image 201, two people are standing at the same distance in front of the hedge. FIG. 2B shows a distance map 202 corresponding to image 201. Each position (each coordinate) of the distance map 202 corresponds to each position of the image 201, and the distance information of each position of the image 201 calculated by the image processing unit 107 is represented by each position of the distance map 201. .. In FIG. 2B, the distance information (distance; defocus amount) is represented by shading. Here, it is assumed that the distance of the area 203 is originally the distance corresponding to the hedge as in the area 204, but is erroneously calculated as the same distance as the two persons. In the first embodiment, an example in which the distance of the area 203 is easily corrected (corrected) to the same distance as the area 204 will be described.

FIG. 3 is a flowchart showing an example of the distance information correction process according to the first embodiment. The distance information correction process will be described with reference to FIG. The correction process of FIG. 3 is started in response to, for example, an execution of an application for correcting distance information or a user operation instructing the start of correction of a distance map (for example, a part of a function of the application).

In step S301, the system control unit 101 displays the distance map generated by the image processing unit 107 on the display unit 111 (display control). Here, it is assumed that the distance map of FIG. 4A is displayed. The distance map of FIG. 4A is the same as the distance map 202 of FIG.

In step S302, the area extraction unit 109 extracts (selects) a part (first area) of the displayed distance map according to the user operation. Here, the area 402 (designated) of FIG. 4B corresponds to the operation 401 (operation of designating a position in the distance map; operation of touching one place in the distance map) of FIG. 4 (A). The area containing the position) is extracted. Region 402 is the same as region 203 in FIG. 2 (B). The extracted region 402 corresponds to a region having the same position and size in image 201 of FIG. 2 (A). As described above, when there is a user operation, the area extraction unit 109 extracts an area based on at least one of the pixel value distribution (color and luminance distribution) of the image and the evaluation value map. The details of this process will be described later.

In step S303, the system control unit 101 highlights (identifiablely displays) the first area extracted in step S302 on the display unit 111. Here, it is assumed that the area 402 is highlighted by a broken line as shown in FIG. 4 (B). The highlighting method is not limited to this, and the first area may be highlighted by, for example, a stripe pattern.

In step S304, the system control unit 101 determines whether or not the first area highlighted in step S303 is a desired area of the user, depending on the presence or absence of a predetermined operation. If it is not a desired region, the process is returned to step S302, the region (the entire region of the image or the distance map) is finely divided as compared with the previous process, and the first region is extracted again.

The process of step S304 and the process (re-extraction) of step S302 will be specifically described with reference to FIG. For example, in a scene like image 201, it is assumed that the color (hue, etc.) of the clothes of the person on the right and the color of the hedge are similar. When the first region is extracted using the color information of the image 201, the region 501 of FIG. 5, which is different from the region 402 of FIG. 4B, may be extracted when the parameter of the region division is not appropriate. .. In this way, a first region different from the region desired by the user may be extracted. In this case, the user respecifies (tap) the desired position. When this operation is performed, the system control unit 101 determines that the extracted first region is not the region desired by the user. Then, the process is returned to step S302. In step S302, the system control unit 101 resets the parameter of the area division, the area extraction unit 109 performs the area division using the reset parameter, and the user among the plurality of areas obtained by the area division. Extracts the first area containing the position specified by. As a result, the user can select a desired area with a simple operation. Details of the parameters to be reset will be described later.

In step S305, the area extraction unit 109 extracts (selects) another part of the displayed distance map (a second area different from the area 402) according to the user operation. Here, it is assumed that the region 403 of FIG. 4B is extracted. Region 403 is the same as region 204 in FIG. 2 (B). In step S305, it is assumed that the second region is extracted by the same method as in step S302, but it is not necessary. For example, a second area of a predetermined size including the position specified by the user and its periphery may be extracted.

In step S306, the system control unit 101 receives a correction instruction from the user. The correction instruction is a drag-and-drop operation from the position touched (designated) by the user in step S305 to the position in the first area extracted in step S302, as in the operation 404 of FIG. 4C. And. The drag and drop operation can be said to be an operation of moving the touch position while touching. The user operation in steps S305 and S306 is not limited to this. For example, in step S305, the second area may be extracted according to the tap, or the button (display item) for the correction instruction may be displayed on the display unit 111, and the tap for the button may be the correction instruction. ..

In step S307, the system control unit 101 uses the image processing unit 107 so that the distance information of the first region extracted in step S302 becomes a value based on the distance information of the second region extracted in step S305. Correct the generated distance map. Specifically, the distance information of each position in the first region is corrected to the average distance information of the second region. As a result, the distance map of FIG. 4 (A) is corrected to the distance map of FIG. 4 (D). The value used for the correction is not limited to the average value of the plurality of distance information in the second region, and may be other distance information (other representative value) representing the second region. For example, the value used for the correction may be the median value, the mode value, the maximum value, the minimum value, etc. of the plurality of distance information in the second region, or the distance information corresponding to the position of the center of gravity of the second region. And so on.

In step S308, the system control unit 101 determines whether or not there is a user operation (save operation) for saving the corrected distance map in step S307. When there is a save operation, the system control unit 101 saves (records) the corrected distance map in the RAM 103, the recording medium 108, or the like. Then, the correction process of FIG. 3 is completed. If there is no save operation, the process is returned to step S302, and the distance information of other areas is corrected as needed.

The area extraction in steps S302 and S305 will be described in detail. The method of region extraction is not particularly limited, but for example, the method described in JP-A-2014-68279 can be used. In the method described in JP-A-2014-68279, region division is performed using the color information of the image, and the region corresponding to the input position is selected. FIG. 6 is a flowchart showing an example of region extraction (steps S302 and S305) according to the first embodiment. Region extraction will be described with reference to FIG. Here, an example of extracting a part of the image will be described, but since the entire image corresponds to the entire distance map, the process of extracting a part of the image can be said to be the process of extracting a part of the distance map.

In step S601, the area extraction unit 109 divides the image into a plurality of areas (region division).

First, the area extraction unit 109 divides the image into a plurality of blocks. Next, the region extraction unit 109 calculates, for each of the plurality of blocks, the average value of the hues of the plurality of pixels in the block as the hue (representative value) representing the block. The representative value is not limited to the average value, and may be, for example, a median value, a mode value, a maximum value, a minimum value, or the like. The representative value may be a hue corresponding to the position of the center of gravity of the block. Next, the region extraction unit 109 creates a hue histogram with the representative value of each block as an element. At this time, it is preferable to create a histogram using only blocks whose saturation is equal to or higher than the threshold value. This is because blocks with low saturation have low reliability of hue values. Next, the region extraction unit 109 divides the image into a plurality of regions by grouping a plurality of blocks into the same color region using the created histogram (region division).

Region division by grouping will be described with reference to FIGS. 7 (A) to 7 (C).

First, as shown in FIG. 7A, the entire section of the histogram is scanned to detect the highest peak HighestPeek1 (maximum frequency). Next, scanning from the detected mountain toward the valleys on both the left and right sides, the section from the position where the frequency of the histogram is equal to or less than the threshold TH_Freq or the position where the distance of the section from the detected mountain reaches the threshold TH_HueRanger is determined. Detected as a section (range) of the same color area. Then, the area consisting of blocks containing the representative value in the detected section is grouped as the same color area corresponding to the detected section. Further, as shown in FIG. 7B, the detected section is recorded as a grouping-processed section.

Next, the section excluding the section that has been grouped from the entire section of the histogram is scanned again, and the same processing as the above processing is performed. At this time, in scanning from the detected mountain HighestPeek2 in the valley direction, the grouping-processed section may be reached before reaching one of the above two positions based on the threshold values TH_Freq and TH_HueRanger. In that case, the section up to the section that has been grouped is set as the section of the same color area. This series of processing is repeated until all the sections of the histogram are grouped.

Finally, as shown in FIG. 7C, labeling is performed by scanning the entire block and assigning different labels to each of the plurality of identical color regions (multiple identical color regions that are not adjacent to each other) corresponding to the same section of the histogram. Perform processing. As a result, a plurality of same color regions corresponding to the same section can be treated as different groups. That is, it is possible to distinguish a plurality of identical color regions corresponding to the same section.

When the area extraction is performed again, such as when the process is returned from step S304 to step S302, at least one of the threshold value TH_Freq and the threshold value TH_HueRanger is reset. For example, the threshold value TH_Freq may be increased or the threshold value TH_HueRanger may be decreased so that the region is finely divided as compared with the previous processing.

Returning to the description of FIG. In step S602, the area extraction unit 109 extracts the area corresponding to the position designated by the user from the plurality of areas obtained by the area division.

As described above, according to the first embodiment, the evaluation value (distance information) of the first region is the evaluation of the second region by a simple operation such as selecting the first region and the second region. The acquired evaluation value distribution is corrected so that the value is based on the value. As a result, the user can easily and intentionally correct the evaluation value.

Although an example of performing region division using image color information has been described, the method of region division and region extraction is not limited to this. For example, the area may be divided using the distance information. Specifically, the same region division as in FIG. 6 may be performed by using the histogram of the distance information instead of the histogram of the hue. By merging the result of the area division using the color information and the result of the area division using the distance information, the area may be further divided. Luminance information may be used instead of color information. The graph cut method may be used to extract the region with the minimum energy from the input position (position specified by the user).

In addition, although an example using a defocus amount distribution as the evaluation value distribution has been described, the evaluation value distribution is not limited to the defocus amount distribution (the evaluation value is not limited to the defocus amount). The evaluation value may be a value indicating the result of some evaluation. For example, the evaluation value distribution includes a blur degree distribution on one side of the image, a parallax distribution on one side of the image, a depth distribution (distance distribution) indicating the subject distance of each pixel, a normal direction distribution on one side of the image, and an established distribution of semantic region division. May be. The probability distribution is, for example, the distribution of recognition scores for people, plants, behavior detection, etc. on the entire image.

Further, an example of using the defocus amount calculated based on the shift amount between the two pixel data as the distance information has been described, but the defocus amount and the distance information are not limited to this. For example, in the DFD (Depth From Defocus) method acquisition method, the defocus amount may be acquired from the correlation between two images having different focus and aperture values. Information related to the distance distribution obtained from the distance measuring sensor module such as the TOF (Time of Flight) method may be used as the distance information.

Further, although an example of performing the distance information correction process while displaying the distance map on the display unit has been described, the display during the correction process is not limited to this. For example, another display may be performed in which the user can easily recognize the correspondence between the image and the distance map to be corrected. Specifically, the distance map may be superimposed and displayed on the image so that the evaluation value distribution is reflected on a plurality of pixel values of the image and the image is displayed on the display unit. The display of the distance map and the display of the image may be switchable according to the user operation. You may want to display the distance map and display the image instead of the distance map only in the selected area.

<Example 2>
Hereinafter, Example 2 of the present invention will be described in detail with reference to the drawings. The points (configuration, processing, etc.) similar to those in the first embodiment will be omitted or simplified. In the second embodiment, an example of reworking a region selected by the user among the processed images based on the distance map based on the distance information of an arbitrary region will be described. The functional configuration of the camera according to the second embodiment is the same as that of the first embodiment (FIG. 1).

The processing according to the second embodiment will be described with reference to FIGS. 8 (A) to 8 (B). FIG. 8A shows an image 801 after processing by the image processing unit 107. Image 801 is the same as image 201 in FIG. 2 (A). FIG. 8B shows a distance map 802 corresponding to image 801. The distance map 802 is the same as the distance map 202 of FIG. 2 (B), the area 803 is the same as the area 203 of FIG. 2 (B), and the area 804 is the same as the area 204 of FIG. 2 (B).

FIG. 8C is an image 805 (processed image) in which the image processing unit 110 blurs the image 801 based on the distance map 802. Here, it is a desired process to blur the hedge behind the hedge like the area 807 (the area corresponding to the area 804), but since the distance of the area 803 is erroneously calculated as the same distance as the two people. In FIG. 8C, the area 806 (the area corresponding to the area 803) is not blurred. The distance map itself may be corrected as in the first embodiment and rework may be performed using the corrected distance map, but in the second embodiment, the user does not have to be aware of the distance map. An example of processing the region 806 in the same manner as the region 807 according to the operation will be described. The processing is not limited to the blurring processing, and may be another processing based on an evaluation value such as distance information.

FIG. 9 is a flowchart showing an example of the correction process of the machining result according to the second embodiment. The correction processing of the processing result will be described with reference to FIG. 9. The correction process of FIG. 9 is started, for example, in response to an execution of an application for correcting the machining result or a user operation instructing the start of the correction of the machining result (for example, a part of the function of the application).

In step S901, the system control unit 101 displays the image processed by the image processing unit 110 on the display unit 111. Here, it is assumed that the image of FIG. 10A is displayed. The image of FIG. 10 (A) is the same as the image 805 of FIG.

The processing of steps S902 to S906 is the same as the processing of steps S302 to S306 of FIG. 3, except for the difference between the distance map and the processed image. 10 (A) to 10 (C) are the same as FIGS. 4 (A) to 4 (C) except for the difference between the distance map and the processed image. That is, operations 1001 and 1004 correspond to operations 401 and 404, and areas 1002 and 1003 correspond to areas 402 and 403.

In step S907, the system control unit 101 corrects the parameters used for processing the first region based on the parameters used for processing the second region, and the image processing unit 110 re-uses the corrected parameters. Perform processing. Specifically, the parameter of each position in the first region is corrected to the average parameter of the second region. That is, the reworking of the first region is performed using the parameters based on the distance information of the second region. As a result, the image of FIG. 10 (A) is corrected to the image of FIG. 10 (D). The parameters used for processing are not particularly limited, and are, for example, the coefficient of the blur filter, the size of the blur filter, the distance information, and the like. Further, the value used for the correction is not limited to the average value of the plurality of parameters in the second region, and may be another parameter (other representative value) representing the second region. For example, the value used for correction may be the median value, mode value, maximum value, minimum value, etc. of a plurality of parameters in the second region, or the parameter corresponding to the position of the center of gravity of the second region. There may be.

In step S908, the system control unit 101 determines whether or not there is a user operation (save operation) for saving the reprocessed image in step S907. When there is a save operation, the system control unit 101 saves (records) the reprocessed image in the RAM 103, the recording medium 108, or the like. Then, the correction process of FIG. 9 is completed. If there is no save operation, the process is returned to step S902, and other areas are reprocessed as needed.

<Example 3>
Hereinafter, Example 3 of the present invention will be described in detail with reference to the drawings. The points (configuration, processing, etc.) similar to those in the first embodiment will be omitted or simplified. In the third embodiment, an example of switching the second area between a plurality of areas different from the first area according to the user operation will be described. The functional configuration of the camera according to the third embodiment is the same as that of the first embodiment (FIG. 1). In the third embodiment, similarly to the first embodiment, an example in which the distance of the region 203 in FIG. 2B is easily corrected (corrected) to the same distance as the region 204 will be described.

FIG. 11 is a flowchart showing an example of the distance information correction process according to the third embodiment. The distance information correction process will be described with reference to FIG. The correction process of FIG. 11 is started in response to, for example, an execution of an application for correcting distance information or a user operation instructing the start of correction of a distance map (for example, a part of a function of the application).

The processing of steps S1101 to S1104 is the same as the processing of steps S301 to S304 of FIG. 12 (A) and 12 (B) are the same as those of FIGS. 4 (A) and 4 (B) except for the display of the correction button 1203. That is, the operation 1201 corresponds to the operation 401, and the area 1202 corresponds to the area 402.

In S1105, the system control unit 101 determines whether or not there is a correction instruction from the user. It is assumed that the correction instruction of the third embodiment is an operation 1204 for selecting (tapping) the correction button 1203 of FIG. 12B, unlike the first embodiment. If there is a correction instruction, the process proceeds to step S1106, and if there is no correction instruction, the process proceeds to step S1108. The process of step S1108 is the same as the process of step S308 of FIG.

In step S1106, the system control unit 101 selects any of a plurality of regions different from the first region obtained by the region division as the second region. Specifically, the system control unit 101 divides the distance histogram (distance information histogram) into a plurality of distance groups by the same method as the area division described in the first embodiment, and the distance group does not correspond to the first area. Select. For example, the distance histogram of FIG. 13 is divided into three distance groups 1301-1303. Then, assuming that the distance information of the area 1202 belongs to the distance group 1301, the distance group 1302 or the distance group 1303 is selected. The method of selecting the distance group is not particularly limited, but if the distance group is selected in descending order of the distance from the distance information of the area 1202, the distance group 1303 is selected first.

In step S1107, the system control unit 101 is generated by the image processing unit 107 so that the distance information of the first region extracted in step S1102 becomes a value based on the distance information of the distance group selected in step S1106. Correct the distance map. Specifically, the distance information of each position in the first region is corrected to the average distance information of the selected distance group. When the distance group 1303 is selected, the distance map of FIG. 12 (A) is corrected to the distance map of FIG. 12 (C). The value used for the correction is not limited to the average distance information of the selected distance group, and may be other distance information (other representative value) representing the selected distance group. For example, the value used for the correction may be the distance information corresponding to the median value, the mode value, the maximum value, the minimum value, and the like of the selected distance group. The value used for the correction may be distance information corresponding to the position of the center of gravity of the region (second region) corresponding to the selected distance group.

Here, the correction in step S1107 may not be the correction desired by the user, that is, the selected distance group may not be the desired distance group by the user. In that case, as shown in FIG. 12C, the user performs the same operation 1205 (tap of the correction button 1203) as in the operation 1204 again. As a result, the process of step S1106 is performed again. In step S1106, an unselected distance group is selected from the plurality of distance groups that do not correspond to the first region. For example, the distance group 1302 far from the distance group 1301 (distance information of the area 1202) is selected next to the distance group 1303. Then, in step S1107, the distance information in the first region is corrected based on the distance information of the newly selected distance group. When the distance group 1302 is newly selected, the distance map of FIG. 12 (C) is corrected to the distance map of FIG. 12 (D).

As described above, according to the third embodiment, the user does not specify the second area, and the second area is switched and set among the plurality of areas by a simpler operation such as pressing a button. be able to. The same processing as that of Example 3 may be combined with Example 2 for reprocessing the image.

Although the present invention has been described in detail based on the preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various embodiments within the range not deviating from the gist of the present invention are also included in the present invention. included. Further, each of the above-described embodiments is merely an embodiment of the present invention, and each embodiment can be appropriately combined.

Further, although an example in which the present invention is applied to a digital camera has been described, the electronic devices to which the present invention can be applied are not limited to this. The present invention is applicable as long as it is an electronic device capable of performing a predetermined process based on the evaluation value (for example, image processing such as correction of evaluation value distribution and blurring process). For example, the electronic device to which the present invention is applied may be a computer having an image processing unit 107, an area extraction unit 109, and an image processing unit 110 in FIG. 1. That is, the image pickup unit 105, the display unit 111, and the like may be provided in an external device of the electronic device to which the present invention is applied.

It should be noted that the above-mentioned various controls described as those performed by the system control unit 101 may be performed by one hardware, or a plurality of hardware (for example, a plurality of processors and circuits) share the processing to be a device. The whole control may be performed.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100: Camera 101: System control unit 107: Image processing unit 109: Area extraction unit 110: Image processing unit

Claims (19)

An acquisition method for acquiring the evaluation value distribution of an image,
A selection means for selecting a first area corresponding to a part of the image according to a first user operation and selecting a second area corresponding to a part of the image according to a second user operation. ,
An electronic device comprising a processing means for performing a predetermined process by using a value based on the evaluation value of the second region as an evaluation value of the first region. The electronic device according to claim 1, wherein the selection means selects a region corresponding to a part of the image based on at least one of a pixel value distribution and an evaluation value distribution of the image. The electronic device according to claim 2, wherein the selection means selects a region corresponding to a part of the image based on a histogram of at least one of a pixel value and an evaluation value of the image. The selection means divides the area corresponding to the image into smaller parts than the first process according to a predetermined user operation after the first process of selecting the area corresponding to a part of the image. The electronic device according to any one of claims 1 to 3, wherein a second process of selecting an area corresponding to a part of an image is performed. The electron according to any one of claims 1 to 4, wherein the selection means selects the second region including the position in response to the second user operation for designating the position. device. The selection means according to claim 1 to 4, wherein the second area is switched between a plurality of areas different from the first area each time the second user operation is performed. The electronic device according to any one. The predetermined process is a process of correcting the evaluation value distribution acquired by the acquisition means so that the evaluation value of the first region becomes a value based on the evaluation value of the second region. The electronic device according to any one of claims 1 to 6, wherein the electronic device is characterized. The electronic device according to claim 7, further comprising a recording means for recording the evaluation value distribution after the predetermined processing in the storage unit. The processing means processes the image based on the evaluation value distribution acquired by the acquisition means.
The electronic device according to any one of claims 1 to 6, wherein the predetermined processing is processing of the first region. The electronic device according to claim 9, further comprising a recording means for recording the image after the predetermined processing in the storage unit. The electronic device according to claim 9 or 10, further comprising a display control means for displaying an image processed by the processing means on a display unit. The electronic device according to any one of claims 1 to 10, further comprising a display control means for displaying the evaluation value distribution on the display unit. The electronic device according to claim 12, wherein the display control means displays the image on the display unit instead of the evaluation value distribution for the region selected by the selection means. The electron according to any one of claims 1 to 10, further comprising a display control means for displaying the image on a display unit by reflecting the evaluation value distribution on a plurality of pixel values of the image. device. The electronic device according to any one of claims 11 to 14, wherein the display control means highlights an area selected by the selection means on the display unit. Any one of claims 1 to 15, wherein the evaluation value distribution is a defocus amount distribution, a blur degree distribution, a parallax distribution, a distance distribution, a normal direction distribution, or a probability distribution of semantic region division. The electronic equipment described in the section. Steps to get the evaluation value distribution of the image and
A step of selecting a first area corresponding to a part of the image according to a first user operation and selecting a second area corresponding to a part of the image according to a second user operation.
A method for controlling an electronic device, which comprises a step of performing a predetermined process by using a value based on an evaluation value of the second region as an evaluation value of the first region. A program for making a computer function as each means of the electronic device according to any one of claims 1 to 16. A computer-readable storage medium containing a program for causing the computer to function as each means of the electronic device according to any one of claims 1 to 16.

Images