JP2007096437A - Determination of image creator's intention - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine the intention of an image creator from an image. <P>SOLUTION: An image processing apparatus includes an imaging section for using a lens and an imaging element to generate image data indicating colors of a plurality of pixels forming an image; and a distance data generating section for generating distance data indicating a subject distance being a distance between a subject and the imaging section and corresponding to each of the plurality of pixels forming the image. The image processing apparatus also includes with a main subject candidate extracting section for extracting an aggregation of a plurality of pixels as a candidate of the main subject from the image based on the distance data; and a main subject determining section for determining whether the candidate of the main subject is the subject based on predetermined conditions. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for determining an intention of an image creator in an image.

  In recent years, digital still cameras (hereinafter also referred to as “DSC”) that generate image data as digital data by imaging a subject using an image sensor have become widespread. Also, image processing (image correction) such as white balance correction for such image data is generally performed.

  As one of such image processing, a technique of image processing using image data including distance information in each part of a subject is known (for example, Patent Document 1). Patent Document 1 discloses image processing that adds blurring effect to an image using image data including distance information and information on a focus position.

JP2000-207549

  An image creator (imaging person) has a certain intention, such as considering a specific subject in the image as the main subject, or considering all subjects in the image as subjects of the same degree of importance. In many cases, the image is generated. However, there is no known technique for determining the intention of the image creator from the generated image data. For this reason, the above-described prior art does not consider the intention of the image creator, and there is a problem in that image processing is not necessarily performed in accordance with the intention of the image creator.

  The present invention has been made in order to solve the above-described conventional problems, and an object of the present invention is to provide a technique that makes it possible to determine the intention of an image creator from an image.

In order to solve at least a part of the above problems, a first image processing apparatus of the present invention includes:
An imaging unit that generates image data representing colors of a plurality of pixels constituting an image using a lens and an imaging element;
A distance data generating unit that generates distance data corresponding to each of a plurality of pixels constituting the image and representing a subject distance that is a distance between the subject and the imaging unit;
A main subject candidate extraction unit that extracts a set of a plurality of pixels as a main subject candidate from the image based on the distance data;
A main subject determination unit that determines whether or not the main subject candidate is the main subject using a predetermined determination condition.

  In this image processing apparatus, image data and distance data are generated, subject candidates are extracted based on the distance data, and it is determined whether or not the main subject candidate is the main subject using a predetermined determination condition. Therefore, in this image processing apparatus, it is possible to determine the intention of the image creator such as the presence or absence of the main subject in the image and which part of the image is the main subject when the main subject is included in the image.

The image processing apparatus further includes a correction processing unit that corrects the image data,
The correction processing unit corrects image data in consideration of the main subject when the main subject determination unit determines that the main subject candidate is the main subject, and the main subject determination unit If it is determined that the main subject candidate is not the main subject, the image data may be corrected without considering the main subject.

  In this way, it is possible to correct the image data according to the intention of the image creator.

  In the image processing apparatus, when the main subject determination unit determines that the main subject candidate is a main subject, the correction processing unit sets image data on pixels constituting the main subject candidate. On the basis of the above, an index value used for correcting the image data representing the entire image may be set.

  In this way, it is possible to correct the image data so that the correction result on the main subject is good, and it is possible to correct the image data in accordance with the intention of the image creator.

In the above image processing apparatus,
When the main subject determination unit determines that the main subject candidate is the main subject, the main subject determination unit includes a main subject specifying information addition unit that adds information for specifying pixels constituting the main subject candidate to the image data. It is good.

  In this way, image data that can identify the main subject can be generated.

  In the image processing apparatus, the main subject candidate extraction unit extracts a set of pixels whose subject distance is within a predetermined range including the subject distance at the focus position of the image as the main subject candidate. Also good.

  In this way, a subject having a subject distance close to the subject distance at the focus position of the image can be extracted as a main subject candidate, so that the intention of the image creator can be accurately determined.

  In the image processing apparatus, the predetermined range may be set to be narrower as the subject distance at the focus position is smaller.

  In this way, main subject candidates can be accurately extracted.

  In the image processing apparatus, the main subject determination unit may determine the main subject when the ratio of pixels constituting the main subject candidate among all the pixels constituting the image is larger than a predetermined threshold. May be determined to be the main subject.

  In this way, since the main subject is the portion that occupies a large proportion in the image, it is possible to accurately determine the intention of the image creator.

  In the image processing apparatus, the main subject determination unit may determine that a subject distance for the main subject candidate is smaller than a threshold set in accordance with a focal length of the lens at the time of capturing the image. The candidate for the main subject may be determined to be the main subject.

  Even in this case, since the main subject is the portion that occupies a large proportion in the image, it is possible to accurately determine the intention of the image creator.

  In the image processing apparatus, the distance data generation unit measures a subject distance corresponding to a representative position in the divided region for each divided region obtained by dividing the image into a plurality of regions, and measures the measured subject distance. May be set as the subject distance corresponding to the pixels included in the divided area.

  In this way, it is possible to increase the efficiency and speed of generation of distance data.

In order to solve at least a part of the above problem, the second image processing apparatus of the present invention provides:
Image data representing the colors of a plurality of pixels constituting the image, and distance data representing a subject distance, which is a distance between the subject and the imaging unit, corresponding to each of the plurality of pixels constituting the image. A data acquisition unit to acquire;
A main subject candidate extraction unit that extracts a set of a plurality of pixels as a main subject candidate from the image based on the distance data;
A main subject determination unit that determines whether or not the main subject candidate is the main subject using a predetermined determination condition.

  In this image processing apparatus, image data and distance data are acquired, subject candidates are extracted based on the distance data, and it is determined whether or not the main subject candidate is the main subject using predetermined determination conditions. Therefore, in this image processing apparatus, it is possible to determine the intention of the image creator such as the presence or absence of the main subject in the image and which part of the image is the main subject when the main subject is included in the image.

  Note that the present invention can be realized in various modes, for example, an image processing method and apparatus, an image generation method and apparatus, an image correction method and apparatus, and a function of these methods or apparatuses. The present invention can be realized in the form of a computer program, a recording medium recording the computer program, a data signal including the computer program and embodied in a carrier wave, and the like.

Next, embodiments of the present invention will be described in the following order based on examples.
A. First embodiment:
B. Second embodiment:
C. Third embodiment:
D. Variation:

A. First embodiment:
FIG. 1 is an explanatory diagram schematically showing the configuration of a digital still camera 100 as a first embodiment of the present invention. The digital still camera (hereinafter also referred to as “DSC”) 100 according to the first embodiment includes a lens 102, a lens driving unit 104 that drives the lens 102 to adjust the focal length f, and a lens driving that controls the lens driving unit 104. The control unit 106, an image sensor 108 that converts light input to the light receiving surface via the lens 102 into an electrical signal, and an A / D converter 110 that performs A / D conversion on the electrical signal output from the image sensor 108. An interface unit (I / F unit) 112 for exchanging information with an external device, a display unit 114 for displaying images and various setting information, an operation unit 116 for user operation, and each unit of the DSC 100. A CPU 118 for controlling, a distance measuring unit 120 for measuring a subject distance D described later, and an internal memory 200 are provided. Each component of the DSC 100 is connected to each other via a bus 122.

  In the present embodiment, a CCD (Chaged Coupled Device) is used as the image sensor 108. In this embodiment, a liquid crystal monitor is used as the display unit 114. At the time of imaging by the DSC 100, an image of the subject is displayed on a liquid crystal monitor as the display unit 114. In this specification, “subject” means an imaging target such as a person, an object, or a landscape that is captured over the entire image.

  The distance measuring unit 120 measures the distance between the subject and the image sensor 108 (hereinafter referred to as “subject distance D”) corresponding to each of a plurality of predetermined positions on the image generated by the DSC 100. The measurement of the subject distance D by the distance measuring unit 120 can be executed by any known method. For example, the measurement of the subject distance D by the distance measuring unit 120 is executed using triangulation, light waves, or sound waves. Note that a plurality of predetermined positions on the image that are subject to measurement of the subject distance D will be described later. In this embodiment, the subject distance D is defined as the distance between the subject and the image sensor 108, but the subject distance D is defined as the distance between the subject and an arbitrary position in the DSC 100. Also good.

  The internal memory 200 stores a computer program that functions as the image processing unit 210. The image processing unit 210 includes a main subject candidate extraction unit 212, a main subject determination unit 214, and a correction processing unit 216 as modules. The CPU 118 reads out and executes a computer program serving as the image processing unit 210 from the internal memory 200, thereby executing image generation / correction processing described later.

  FIG. 2 is a flowchart showing the flow of image generation / correction processing by the DSC 100. In step S110, the image processing unit 210 (FIG. 1) captures a subject using each unit of the DSC 100, thereby generating image data representing an image (hereinafter referred to as “captured image CI”). Specifically, light projected through the lens 102 onto the light receiving surface of the image sensor 108 is converted into an electrical signal, and the electrical signal is converted into digital data in the A / D converter 110, whereby image data. Is generated. The image data is generated as color data representing the color of each pixel constituting the captured image CI. In this embodiment, RGB data is used as the color data.

  FIG. 3 is an explanatory diagram illustrating an example of the captured image CI represented by the generated image data. In FIG. 3, a position P indicated by a broken line (hereinafter referred to as “focus position P”) indicates a focus position when the captured image CI is captured.

  In step S110 (FIG. 2), the image processing unit 210 also measures the subject distance D using the distance measuring unit 120 (FIG. 1), and generates distance data representing the subject distance D.

  FIG. 4 is an explanatory diagram showing an example of the measured subject distance D. As shown in FIG. FIG. 4 shows the subject distance D (numbers in parentheses (unit: meter)) superimposed on the captured image CI. Here, in the present embodiment, the measurement of the subject distance D by the distance measuring unit 120 is performed for each divided area DA obtained by dividing the captured image CI into a plurality of areas. Specifically, the center pixel of each divided area DA is set as a representative pixel, and the subject distance D corresponding to the position of the representative pixel is measured. That is, the plurality of predetermined positions on the image to be measured for the subject distance D described above are the positions of the central pixels of the divided areas DA in the captured image CI.

  In addition, the image processing unit 210 corresponds to the position of the pixel at the position where the subject distance D is not measured, that is, the subject distance D corresponding to the position of the pixel other than the center pixel in each divided area DA. Set the same distance as the subject distance D. In this way, the subject distance D corresponding to the position of each pixel constituting the captured image CI is set, and distance data representing the subject distance D is generated. When the measured subject distance D is greater than or equal to a predetermined distance, the subject distance D is set as infinity (∞).

  FIG. 5 is an explanatory diagram showing an example of generated image data (color data) and distance data. As shown in FIG. 5, the image data (color data) is generated as data representing the RGB component values of each pixel constituting the captured image CI specified by the coordinates. The distance data is generated as data representing the subject distance D corresponding to the position of each pixel.

In step S120 (FIG. 2), the main subject candidate extraction unit 212 (FIG. 1) of the image processing unit 210 extracts the main subject candidate MSC from the captured image CI. The main subject candidate MSC is a main subject candidate in the captured image CI. In this embodiment, the main subject candidate MSC is defined as an area composed of pixels in which the subject distance D satisfies the following formula (1). The main subject candidate extraction unit 212 extracts a main subject candidate MSC by selecting pixels satisfying the following expression (1) from image data representing the captured image CI.
| D-Dp | ≦ Gt (1)
D: Subject distance (m) at the position of the pixel to be determined
Dp: subject distance (m) at the focus position P
Gt: threshold (m)

  As can be seen from the above equation (1), in this embodiment, the main subject candidate MSC is composed of pixels whose subject distance D is within Gt (m) before and after the subject distance Dp at the focus position P. In general, the main subject candidate MSC is defined as described above in this embodiment because the main subject is often focused during imaging.

  When the focus position P includes a plurality of pixels, the subject distance Dp at the focus position P in the above formula (1) is calculated as the average of the subject distances D at the positions of the plurality of pixels included in the focus position P. Alternatively, it may be calculated as the subject distance D at the position of the representative pixel among the plurality of pixels included in the focus position P. Further, in order to accurately extract the main subject candidate MSC, it is preferable that the measurement of the subject distance D in step S110 is executed at the pixel position included in the focus position P.

  FIG. 6 is an explanatory diagram showing an example of the extracted main subject candidate MSC. In the example of FIG. 6, the subject distance Dp at the focus position P is 5 (m). The threshold Gt is assumed to be 1 (m). At this time, a set of pixels whose subject distance D is 4 (m) or more and 6 (m) or less is extracted as the main subject candidate MSC. FIG. 6 shows the extracted main subject candidate MSC with hatching.

  In the present embodiment, the threshold Gt in the above equation (1) is variably set according to the subject distance Dp at the focus position P in order to accurately extract the main subject candidate MSC. FIG. 7 is a graph showing the threshold Gt according to the subject distance Dp at the focus position P. As shown in FIG. 7, the threshold value Gt is set to increase as the subject distance Dp at the focus position P increases. Accordingly, when the subject distance Dp at the focus position P is small, for example, when performing macro photography, the main subject candidate MSC is configured by pixels having the subject distance D within a narrower range. On the other hand, when the subject distance Dp at the focus position P is large, the main subject candidate MSC is configured by pixels having the subject distance D within a wider range.

  In step S130 (FIG. 2), the main subject determination unit 214 (FIG. 1) performs main subject determination on the extracted main subject candidate MSC. The main subject determination is a determination as to whether or not the main subject candidate MSC is a main subject. In the present embodiment, when the ratio R of the main subject candidate MSC in the captured image CI is greater than the predetermined threshold Rt, the main subject candidate MSC is determined to be the main subject. On the other hand, when the ratio R is equal to or less than the predetermined threshold Rt, it is determined that the main subject candidate MSC is not the main subject. In many cases, a region that shows a relatively large proportion in the captured image CI is a main subject. Therefore, the main subject determination of this embodiment is executed according to the above-described determination conditions.

  For example, it is assumed that the predetermined threshold value Rt in the determination condition is 1/4 (25%). In this case, in the example of FIG. 6, the ratio R of the main subject candidate MSC in the captured image CI is 8/25 (32%), which is larger than the threshold value Rt. Therefore, the main subject candidate MSC is the main subject. Determined.

  FIG. 8 is an explanatory diagram showing the main subject candidate MSC in another captured image CI. The captured image CI illustrated in FIG. 8 is a wider-angle image than the captured image CI illustrated in FIG. In the captured image CI shown in FIG. 8, the ratio R of the main subject candidate MSC in the captured image CI is 3/25 (12%), which is less than the threshold value Rt. It is determined that there is no. In this case, there is no main subject in the captured image CI, and it is determined that all the subjects in the captured image CI are subjects having the same degree of importance.

  If it is determined in step S130 (FIG. 2) that the main subject candidate MSC is the main subject, the process proceeds to step S140, and the correction processing unit 216 (FIG. 1) applies to the image data representing the captured image CI. Then, image correction is performed in consideration of the main subject.

  For example, the correction processing unit 216 performs white balance correction considering the main subject. At this time, the correction processing unit 216 sets a correction index value (parameter) based on the color data of the main subject candidate MSC in the captured image CI, and uses the set parameters for the color data of the entire captured image CI. Perform white balance correction. In this way, it is possible to realize white balance correction for the entire captured image CI so that the white balance in the main subject candidate MSC in the captured image CI is good.

  For example, the correction processing unit 216 performs “blur” correction in consideration of the main subject. At this time, the correction processing unit 216 corrects the blur so that the stronger the blur is added to the pixels whose subject distance D is farther from the subject distance D in the main subject candidate MSC. As the subject distance D in the main subject candidate MSC, the average value of the subject distance D in the pixels constituting the main subject candidate MSC or the subject distance D at the representative position (for example, the focus position P) is used. In this way, the main subject has almost no blur, and the subject that is more distant from the main subject in the depth direction of the captured image CI is corrected to be blurred with respect to the captured image CI. realizable.

  On the other hand, if it is determined in step S130 (FIG. 2) that the main subject candidate MSC is not the main subject, the process proceeds to step S150, and the correction processing unit 216 (FIG. 1) displays image data representing the captured image CI. In contrast, image correction is performed without considering the main subject. For example, the correction processing unit 216 sets an index value (parameter) for white balance correction based on the color data of the entire captured image CI, and performs white balance correction on the color data of the entire captured image CI using the set parameters. Do. In this way, it is possible to realize white balance correction for the entire captured image CI so that the overall white balance is good.

  As described above, in the image generation / correction processing by the DSC 100 of this embodiment, image data is generated and distance data is generated. Based on the distance data, the main subject candidate MSC is extracted. Furthermore, it is determined by main subject determination whether main subject candidate MSC is a main subject. For this reason, in the image generation / correction processing of the present embodiment, whether the main subject exists in the captured image CI, and if it exists, which region in the captured image CI it is, Intent can be determined.

  Further, in the image generation / correction processing of the present embodiment, when it is determined that there is a main subject in the captured image CI, image correction considering the main subject is executed. Accordingly, it is possible to realize image correction in accordance with the intention of the image creator.

B. Second embodiment:
FIG. 9 is a graph for explaining main subject determination in the image generation / correction processing of the second embodiment. The image generation / correction processing of the second embodiment is the same as that of the first embodiment except that the determination criteria for main subject determination are different from those of the first embodiment.

In the main subject determination in the image generation / correction process of the second embodiment, if the following expression (2) is satisfied, it is determined that the main subject candidate MSC is the main subject.
Dp ≦ Dpt (2)
Dp: subject distance (m) at the focus position P
Dpt: threshold (m)

  As can be seen from the above equation (2), in the second embodiment, when the subject distance Dp at the focus position P is equal to or smaller than the threshold value Dpt, the main subject candidate MSC is determined to be the main subject. Here, the threshold value Dpt is variably set according to the focal length f of the lens 102 (FIG. 1) at the time of imaging. FIG. 9 shows an example of the relationship between the focal length f of the lens 102 and the threshold value Dpt during imaging. As shown in FIG. 9, the threshold value Dpt is set so that the threshold value Dpt increases as the focal length f increases.

  When the focal length f is large, the captured image CI is a telephoto image. In this case, even if the subject distance Dp at the focus position P is large to some extent, the proportion of the main subject candidate MSC in the captured image CI is relatively large. On the other hand, when the focal length f is small, the captured image CI is a wide-angle image. In this case, when the subject distance Dp at the focus position P is large, the proportion of the main subject candidate MSC in the captured image CI is relatively small. In the second embodiment, in consideration of such a relationship, the subject at the focus position P such that the proportion of the area of the main subject candidate MSC in the captured image CI becomes a certain size according to the focal length f. The distance Dp is set as the threshold value Dpt.

  Also by the main subject determination in the image generation / correction processing of the second embodiment described above, it is possible to determine whether or not the main subject candidate MSC is the main subject, as in the first embodiment.

C. Third embodiment:
FIG. 10 is an explanatory diagram schematically showing the configuration of an image processing system 10 as a third embodiment of the present invention. The image processing system 10 according to the third embodiment includes a DSC 100a and a computer 300. The third embodiment is different from the first embodiment in that image generation / correction processing (see FIG. 2) is performed by such an image processing system 10.

  The computer 300 includes a CPU 310, a display unit 320 such as a monitor, an operation unit 330 such as a keyboard and a mouse, an external storage device 340 such as a hard disk drive, an interface unit (I / F unit) 350, a ROM, a RAM, and the like. Internal storage device 400. Each component of the computer 300 is connected to each other via a bus 360.

  The interface unit 350 of the computer 300 includes a plurality of input / output terminals, and exchanges information with external devices. For example, the interface unit 350 is connected to the DSC 100 via a cable, and reads data stored in the DSC 100.

  The internal storage device 400 of the computer 300 stores a computer program that functions as the image processing unit 410. The image processing unit 410 is a program similar to the image processing unit 210 (FIG. 1) stored in the DSC 100 in the first embodiment. Similar to the image processing unit 210 of the first embodiment, the image processing unit 410 includes a main subject candidate extraction unit 412, a main subject determination unit 414, and a correction processing unit 416 as modules. The CPU 310 reads out and executes a computer program serving as the image processing unit 410 from the internal storage device 400, thereby executing image generation / correction processing described later.

  The DSC 100a of the image processing system 10 has the same configuration as the DSC 100 of the first embodiment. However, the DSC 100a of the third embodiment does not need to have a computer program as the image processing unit 210 (FIG. 1).

  FIG. 11 is a flowchart showing the flow of image generation / correction processing in the third embodiment. In the third embodiment, the DSC 100a of the image processing system 10 and the computer 300 cooperate to perform image generation / correction processing similar to that in the first embodiment.

  That is, in step S110, the DSC 100a generates image data and distance data (see FIG. 5) as in the first embodiment. In step S115, the computer 300 acquires image data and distance data from the DSC 100a via the interface unit 350 (FIG. 10).

  The processing after step S120 is the same as the processing after step S120 in the image generation / correction processing of the first embodiment, except that the subject is the image processing unit 410 (FIG. 10) of the computer 300.

  As described above, in the image processing system 10 according to the third embodiment, the image generation / correction processing can be executed as in the first embodiment.

D. Variation:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

D1. Modification 1:
The setting mode of the divided area DA in each of the above embodiments is merely an example, and the setting mode of the divided area DA may be another mode. For example, in each of the above embodiments, the captured image CI is divided into 5 parts in the vertical direction and 5 parts in the horizontal direction to set the 5 × 5 divided area DA, but the 10 × 10 divided area DA is set. Alternatively, a 5 × 10 divided area DA may be set. Ultimately, each divided area DA may be constituted by one pixel constituting the captured image CI. Further, the divided area DA may be set so that divided areas DA having different sizes and shapes are mixed.

D2. Modification 2:
In each of the above embodiments, the representative pixel for which the subject distance D is measured is the central pixel of the divided area DA, but the representative pixel may be another pixel in the divided area DA. The representative pixel may be a plurality of pixels in the divided area DA.

D3. Modification 3:
The extraction conditions (the above formula (1)) of the main subject candidate MSC in each of the above embodiments are merely examples, and other conditions may be adopted. For example, the following formula (3) can be adopted as the extraction condition for the main subject candidate MSC. Note that Gt1 and Gt2 may be different values or the same value.
Dp−Gt1 ≦ D <Dp + Gt2 (3)
D: Subject distance (m) at the position of the pixel to be determined
Dp: subject distance (m) at the focus position P
Gt1, Gt2: threshold (m)

D4. Modification 4:
In each of the above embodiments, the threshold Gt is variably set according to the subject distance Dp at the focus position P (see FIG. 7), but the threshold Gt is set to a constant value regardless of the subject distance Dp at the focus position P. It may be set.

D5. Modification 5:
The determination conditions used for main subject determination in the above embodiments are merely examples, and other conditions may be employed. For example, in the first embodiment, the predetermined threshold value Rt in the determination condition is ¼ (25%), but this threshold value Rt may be set to another value.

  Further, as a determination condition used for determining the main subject, a condition relating to a difference between the subject distance D for the main subject candidate MSC and the subject distance D for other regions in the captured image CI may be employed. That is, if the difference is equal to or greater than a predetermined threshold, the main subject candidate MSC is determined to be the main subject, and if the difference is smaller than the predetermined threshold, it may be determined that the main subject candidate MSC is not the main subject. . In this case, the subject distance D for the main subject candidate MSC may be the subject distance Dp for the focus position P, or the subject distance D for the representative pixel in the main subject candidate MSC. Alternatively, an average of the subject distances D for a plurality of representative pixels in the main subject candidate MSC may be employed. Further, the other area in the captured image CI may be an area included within a predetermined distance from the outer peripheral line of the main subject candidate MSC. Further, as the subject distance D for the other area in the captured image CI, the subject distance D for the representative pixel in the other area may be adopted, or a plurality of representatives in the other area may be adopted. You may employ | adopt the average of the object distance D about a pixel.

  In the second embodiment, the main subject determination is performed by comparing the subject distance Dp with respect to the focus position P and the threshold value Dpt, but instead of the subject distance Dp with respect to the focus position P, main subject candidates The subject distance D for the representative pixel in the MSC may be used, or the average of the subject distances D for a plurality of representative pixels in the main subject candidate MSC may be used.

D6. Modification 6:
In each of the above embodiments, white balance correction and blur correction are cited as examples of image correction performed by the correction processing unit 216 in image generation / correction processing. However, the correction processing unit 216 performs contrast correction, sharpness correction, and color correction. Other image corrections such as balance correction may be performed.

D7. Modification 7:
In the third embodiment, in the image generation / correction process, only the image data and distance data generation step (step S110 in FIG. 11) is executed by the DSC 100a, and the remaining steps are executed by the computer 300. . The sharing of processing between the DSC 100a and the computer 300 in the image generation / correction processing may be in another form.

  For example, the DSC 100a may execute the main subject determination (step S130) in the image generation / correction processing (FIG. 11), and the remaining processing may be executed by the computer 300. Alternatively, the main subject candidate extraction (step S120) may be executed by the DSC 100a, and the remaining processing may be executed by the computer 300. In these cases, the DSC 100a may generate data in which information for specifying the main subject candidate MSC or the pixels constituting the main subject (for example, the main subject flag) is added to the image data representing the captured image CI. Good. The computer 300 may acquire this data from the DSC 100a and execute the remaining processing.

D8. Modification 8:
The configuration of the DSC 100 in each of the above embodiments is merely an example, and the configuration of the DSC 100 can be other configurations. In each of the above embodiments, a part of the configuration realized by hardware may be replaced by software, and conversely, a part of the configuration realized by software may be replaced by hardware. Also good.

It is explanatory drawing which shows roughly the structure of the digital still camera as 1st Example of this invention. It is a flowchart which shows the flow of the image production | generation / correction | amendment process by DSC. It is explanatory drawing which shows an example of the captured image CI which the generated image data represents. It is explanatory drawing which shows an example of the measured to-be-photographed object distance D. FIG. It is explanatory drawing which shows an example of the produced | generated image data (color data) and distance data. It is explanatory drawing which shows an example of the extracted main subject candidate MSC. 6 is a graph showing a threshold value Gt corresponding to a subject distance Dp at a focus position P. It is explanatory drawing which shows main subject candidate MSC in the other captured image CI. It is a graph for demonstrating main subject determination in the image production | generation / correction | amendment process of 2nd Example. It is explanatory drawing which shows roughly the structure of the image processing system as 3rd Example of this invention. It is a flowchart which shows the flow of the image production | generation / correction | amendment process in 3rd Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image processing system 100 ... Digital still camera 102 ... Lens 104 ... Lens drive part 106 ... Lens drive control part 108 ... Image sensor 110 ... A / D converter 114 ... Display part 116 ... Operation part 118 ... CPU
DESCRIPTION OF SYMBOLS 120 ... Distance measurement part 122 ... Bus 200 ... Internal memory 210 ... Image processing part 212 ... Main subject candidate extraction part 214 ... Main subject determination part 216 ... Correction processing part 300 ... Computer 310 ... CPU
320 ... Display unit 330 ... Operation unit 340 ... External storage device 350 ... Interface unit 360 ... Bus 400 ... Internal storage device 410 ... Image processing unit 412 ... Main subject Candidate extraction unit 414 ... main subject determination unit 416 ... correction processing unit

Claims (11)

An image processing apparatus,
An imaging unit that generates image data representing colors of a plurality of pixels constituting an image using a lens and an imaging element;
A distance data generating unit that generates distance data corresponding to each of a plurality of pixels constituting the image and representing a subject distance that is a distance between the subject and the imaging unit;
A main subject candidate extraction unit that extracts a set of a plurality of pixels as a main subject candidate from the image based on the distance data;
An image processing apparatus comprising: a main subject determination unit that determines whether or not the main subject candidate is a main subject using a predetermined determination condition. The image processing apparatus according to claim 1, further comprising:
A correction processing unit for correcting image data;
The correction processing unit corrects image data in consideration of the main subject when the main subject determination unit determines that the main subject candidate is the main subject, and the main subject determination unit An image processing apparatus that corrects image data without considering the main subject when it is determined that the main subject candidate is not the main subject. The image processing apparatus according to claim 2,
When the main subject determination unit determines that the main subject candidate is the main subject, the correction processing unit represents the entire image based on image data of pixels constituting the main subject candidate. An image processing apparatus for setting an index value used for correcting image data. The image processing apparatus according to claim 1, further comprising:
When the main subject determination unit determines that the main subject candidate is the main subject, the main subject determination unit includes a main subject specifying information addition unit that adds information for specifying pixels constituting the main subject candidate to the image data. , Image processing device. An image processing apparatus according to any one of claims 1 to 4,
The main subject candidate extraction unit is an image processing apparatus that extracts a set of pixels whose subject distance is within a predetermined range including a subject distance at a focus position of the image as the main subject candidate. The image processing apparatus according to claim 5, wherein
The image processing apparatus, wherein the predetermined range is set to be narrower as the subject distance at the focus position is smaller. An image processing apparatus according to any one of claims 1 to 4,
The main subject determination unit determines that the main subject candidate is the main subject when the ratio of the pixels constituting the main subject candidate among all the pixels constituting the image is larger than a predetermined threshold. An image processing apparatus for determining. An image processing apparatus according to any one of claims 1 to 4,
The main subject determination unit determines that the main subject candidate is a main subject when a subject distance of the main subject candidate is smaller than a threshold set in accordance with a focal length of the lens at the time of capturing the image. An image processing apparatus that determines that The image processing apparatus according to claim 1,
The distance data generation unit measures a subject distance corresponding to a representative position in the divided region for each divided region obtained by dividing the image into a plurality of regions, and includes the measured subject distance in the divided region. An image processing apparatus that sets a subject distance corresponding to a pixel to be recorded. An image processing method comprising:
(A) generating image data representing colors of a plurality of pixels constituting an image using a lens and an image sensor;
(B) generating distance data representing a subject distance, which is a distance between the subject and the imaging unit, corresponding to each of a plurality of pixels constituting the image;
(C) extracting a set of a plurality of pixels as main subject candidates from the image based on the distance data;
(D) determining whether or not the main subject candidate is a main subject using a predetermined determination condition. An image processing apparatus,
Image data representing the colors of a plurality of pixels constituting the image, and distance data representing a subject distance, which is a distance between the subject and the imaging unit, corresponding to each of the plurality of pixels constituting the image. A data acquisition unit to acquire;
A main subject candidate extraction unit that extracts a set of a plurality of pixels as a main subject candidate from the image based on the distance data;
An image processing apparatus comprising: a main subject determination unit that determines whether or not the main subject candidate is a main subject using a predetermined determination condition.

Images