JP5245604B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus having a face detection function and a program used for the image processing apparatus.

  2. Description of the Related Art Conventionally, an imaging device such as a digital camera or a digital video camera that captures an object using a CCD or CMOS type solid-state imaging device is known.

  A person image captured using this imaging apparatus is often used as an ID photo. In this case, since it is desirable that the background of the person is plain, it is necessary to select an imaging place where the background is plain when imaging. Therefore, in order to relax the conditions that can be an imaging location, an image processing technique that can extract an area indicating a person excluding a background portion from a captured image is desired.

  By the way, what has the function to detect a person's face has appeared in the imaging device. Such an imaging device can detect the approximate position and size of the face by detecting parts such as eyes, nose and mouth. However, since it is difficult to discriminate parts and contours other than the person's face, it is not suitable as a means for extracting an area indicating a person.

Under such circumstances, for example, Patent Document 1 discloses a method of separating and extracting a region indicating a person by quantizing pixels in image data in several stages based on brightness. ing.
JP 2005-223523 A

  However, since the method of Patent Document 1 requires a clear brightness difference between a pixel indicating a person and a pixel indicating a background, the brightness difference is small or the background is complicated. In some cases, the extraction accuracy becomes unstable. Therefore, depending on the imaging location, it is difficult to extract a region indicating a person excluding the background portion from the captured image.

  Accordingly, an object of the present invention is to provide an image processing apparatus that can easily extract a region indicating a person excluding a background portion from a captured image.

The image processing apparatus according to the invention of claim 1 includes an acquisition unit that acquires unrealized and the first image and the second image different backgrounds of the same subject, in the first image and the second image the acquired Based on the relative displacement, the detecting means for detecting the face portion of the subject, the displacement measuring means for measuring the relative displacement between the face portion of the first image and the face portion of the second image, The association means for associating the pixels constituting the image with the pixels constituting the second image and the associated pixels are compared, and the difference between the pixel values of the pixels is equal to or greater than a first predetermined value Includes a determining unit that sets the pixel as a background portion and sets the pixel as a non-background portion when the difference between the pixel values of the pixels is less than a first predetermined value.

The image processing apparatus according to the invention of claim 2 includes a selection means for selecting one of the first image or the second image, the background portion the discriminating means with the selected image by the selection means and further comprising a setting means for setting the pixel value of the determined pixel to be the same value.

  The image processing apparatus according to the invention of claim 3 determines the center of the detected face part, and sets a pixel whose distance from the center of the face part is a third predetermined value or more as a background part, The discriminator further includes a second discriminating unit that uses a pixel whose distance from the center of the face portion is a fourth predetermined value or less as a non-background portion, and for the region excluding the discrimination target of the second discriminating unit. It is characterized by performing.

In the image processing apparatus according to the fourth aspect of the present invention, the second determination unit has a predetermined angle of view with respect to a neck direction viewed from the center of the face part of the subject based on an orientation of the face part. A range that is assumed to include a torso portion of a human body that is an angle range is excluded from the discrimination target.

  The image processing apparatus according to claim 5 further includes warning means for outputting a warning when the feature amount of the face portion of the first image and the feature amount of the face portion of the second image are greatly different. It is characterized by providing.

  In the image processing apparatus according to the sixth aspect of the present invention, when the warning means has a feature quantity of the face portion of the first image greatly different from a feature quantity of the face portion of the second image, the second means Based on the feature amount of the face portion of the image, the position and size of the face portion of the second image for approximating the feature amount of the face portion of the second image to the feature amount of the face portion of the first image; And at least one of the directions is notified.

  The image processing apparatus according to claim 7, wherein the acquisition unit newly creates an image when the feature quantity of the face portion of the first image and the feature quantity of the face portion of the second image do not approximate. The new image is acquired and used as the second image.

The image processing apparatus according to an eighth aspect of the invention is characterized in that the acquisition unit includes a subject as a first image group and includes a plurality of similar background images and a subject as a second image group and a background different from the first image group. A plurality of images, selecting one image from each of the acquired image groups, generating a plurality of combinations, for each of the plurality of combinations, the combination having the highest correlation of the face part, The image processing apparatus further includes a determining unit that determines the first image and the second image.

The image processing apparatus according to claim 9 further includes a measurement unit that measures a degree of focus on the face portion of the subject, and the detection unit uses the degree of focus measured by the measurement unit as the degree of focus. It is detected as a feature amount of the face portion of the first image and the second image .

The image processing apparatus according to the invention of claim 10, the estimation means on the basis of the feature amount of the detected face parts, to estimate the non-background portion of the first image and the second image above acquired, before Kikao And aligning the non-background portion other than the face portion of the first image and the non-background portion other than the face portion of the second image based on the estimation result of the estimating means. The image forming apparatus further includes alignment means for associating the pixels constituting the first image with the pixels constituting the second image based on the two alignment results.

  The image processing apparatus according to claim 11 stores a non-background portion of the selected image when the user selects a non-background portion of the first image or a non-background portion of the second image. It further comprises storage means for performing.

Program according to the invention of claim 12, computer, same subject unrealized and acquisition means for acquiring the first image and the second image different background, in the first image and the second image the acquired From the detection means for detecting the face portion of the subject, the displacement measuring means for measuring the relative displacement between the face portion of the first image and the face portion of the second image, the first image is based on the relative displacement. An associating means for associating the constituting pixels with the pixels constituting the second image, comparing the associated pixels with each other, and when the difference between the pixel values of the pixels is equal to or greater than a first predetermined value, When the pixel is the background portion and the difference between the pixel values of the pixel is less than the first predetermined value, the pixel functions as a non-background portion.

  According to the present invention, it is possible to easily extract a region indicating a person excluding a background portion from a captured image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description of the embodiments, the same constituent elements are denoted by the same reference numerals, and the description thereof is omitted or simplified.

[First Embodiment]
FIG. 1 is a block diagram showing a schematic configuration of a digital camera 1 as an image processing apparatus according to the present embodiment. The digital camera 1 has a function of detecting a human face portion in image data, and mainly includes the following units.

  That is, the digital camera 1 includes a photographing unit 2, a CPU 11, a key input unit 12, a face detection unit 13, a flash memory 14, a display unit 15, an external memory 16, a resolution conversion unit 17, a JPEG conversion unit 18, and an SDRAM 19.

  Here, the photographing unit 2 is an acquisition unit of the present invention that converts an optical image including a subject that has passed through a lens into a digital signal and stores the digital image in the SDRAM 19 as image data (first image, second image). The photographing unit 2 includes a timing generator (TG) 21, a CCD driving unit 22, a CCD 23, and an A / D conversion unit 24.

  The CPU 11 controls the above-described units of the digital camera 1, operates the data stored in the flash memory 14, the external memory 16, the SDRAM 19, and the like, and replaces the background portion in the image data according to the present embodiment with a predetermined image. Perform processing.

  That is, the CPU 11 constitutes the second image with the displacement measuring means of the present invention for measuring the relative displacement between the face portion of the first image and the face portion of the second image, and the pixels constituting the first image based on the relative displacement. The association means of the present invention for associating pixels, the determination means of the present invention for comparing the associated pixels to determine the background portion and the non-background portion, the background portion and the non-background without comparing pixel values A second discriminating means of the present invention for discriminating a portion; a setting means of the present invention for generating a new image by setting a pixel value of a pixel discriminated as a background portion to a predetermined value (second predetermined value); Warning means of the present invention for outputting a warning when the feature amount of the face portion of the first image and the feature amount of the face portion of the second image are greatly different, and a book for storing the non-background portion of the image selected by the user It is a storage means of the invention (details of processing will be described later) .

  The CPU 11 also functions as a measurement unit of the present invention that measures the degree of focus on the face portion of the subject, and further, as an estimation unit of the present invention that estimates a non-background portion based on the feature amount of the face portion. Function.

  The key input unit 12 includes a power key, a mode switching key for switching between a recording mode and a playback mode, which are basic operation modes of the digital camera 1, or an ID photo mode according to the present embodiment, a MENU key, a shutter key, and the like. Includes multiple keys. The shutter key has a half shutter function that can be operated in two steps: a half-pressed position for a user to make a shooting advance notice and a full-pressed position for instructing an actual shooting operation. The operation state of each key of the key input unit 12 is scanned by the CPU 11 as needed.

  The face detection unit 13 is a detection unit of the present invention that detects a human face portion from image data stored in the external memory 16 or the SDRAM 19. Specifically, the face detection unit 13 detects at least one of the position, size, and orientation of the face part as a face parameter (feature amount).

  The flash memory 14 is a nonvolatile memory capable of rewriting stored data, and stores various programs executed by the CPU 11 and various data necessary for processing by the CPU 11. The flash memory 14 also stores setting information regarding various functions of the digital camera 1 set by the user.

  The display unit 15 includes a liquid crystal display and a driving circuit for the liquid crystal display. In the recording mode, every time image data for one frame is stored in the SDRAM 19, the image data is converted into a video signal and displayed on the display unit 15 as a through image. When shooting is performed in the recording mode, the image data temporarily stored in the SDRAM 19 is compressed and encoded by the JPEG conversion unit 18 using the JPEG method, and then is statically stored in the external memory 16 including, for example, various memory cards. It is recorded as a picture file.

  In the reproduction mode, the still image file recorded in the external memory 16 is appropriately read according to the user's selection operation, decoded by the JPEG conversion unit 18 and expanded as image data in the SDRAM 19, and displayed on the display unit 15. Displayed as a still image.

  In the ID photo mode, two image data (first image and second image) are stored in the SDRAM 19 by the photographing operation, and the CPU 11 determines the pixel of the background portion for at least one of the two image data. New composite image data in which the pixel value is set to a predetermined value (second predetermined value) is generated. Then, the generated new composite image data is compressed and encoded by the JPEG conversion unit 18 by the JPEG method, and then recorded in the external memory 16 as a still image file.

  Here, the display unit 15 outputs a warning based on a command from the CPU 11 when the feature amount of the face portion of the first image and the feature amount of the face portion of the second image are greatly different from each other. It is.

  The resolution conversion unit 17 converts the resolution of the image data stored in the SDRAM 19. The image data whose resolution has been converted is converted into a video signal, whereby the image displayed on the display unit 15 is enlarged or reduced.

  The timing generator 21 generates a timing signal, and the CCD driving unit 22 generates a driving signal based on the timing signal. The CCD 23 photoelectrically converts the optical image of the subject based on the timing signal output from the CCD drive unit 22 and outputs it as an imaging signal. The A / D conversion unit 24 converts the imaging signal output from the CCD 2 into a digital signal and stores it as image data in the SDRAM 19.

  Hereinafter, details of the processing of the digital camera 1 executed under the control of the CPU 11 will be described.

  FIG. 2 is a diagram showing an outline of processing according to the present embodiment. First, the first image (a) and the second image (b) including a person's face are photographed by the digital camera 1. At this time, it is assumed that the background of the person is different from each other, and the size and orientation of the person on the image do not change.

  The CPU 11 determines the face part of the first image (a) and the face part of the second image (b) based on the face parameters of the face part detected by the face detection unit 13 from the image data of both of the two images. And align. Thus, the CPU 11 calculates the relative displacement amount between the two pieces of image data, and associates the pixels constituting the first image with the pixels constituting the second image by the relative displacement amount.

  Subsequently, the CPU 11 compares the pixels associated with each other as described above. When the difference between the pixel values of the pixels is equal to or greater than a predetermined value (first predetermined value), the CPU 11 determines the pixel as a background portion. When the difference between the pixel values of this pixel is less than a predetermined value (first predetermined value), this pixel is determined as a non-background portion (person portion). In other words, if two pieces of image data having different backgrounds are used, the values of the corresponding pixels in the background portion are different and the same or similar in the human portion. Therefore, the pixel values of the corresponding pixels are compared in this way. Thus, the background and non-background regions can be separated.

  Then, the CPU 11 sets the pixel value of the pixel determined to be the background portion to at least one of the first image (a) and the second image (b) to a predetermined value (second predetermined value), A new image (c) is generated. Thereby, the background portion of the image (c) is filled with a predetermined color.

  3 and 4 are flowcharts showing the flow of processing according to the present embodiment. When the ID photo mode is set by operating a mode switching key provided in the key input unit 12, the CPU 11 reads a program for executing the processing of the flowcharts shown in FIGS. 3 and 4 from the flash memory 14. . The CPU 11 executes various processes according to this program.

First, the photographing processing shown in FIG. 3, based on the user, the control of the CPU 11, the first image _{P 1} and the second image _{P 2} is recorded.

In step S <b> 101, the CPU 11 controls the photographing unit 2 in response to the user pressing the shutter key, acquires the first image P <b> ₁ , and records it in the SDRAM 19.

In step S102, CPU 11, to the first image _{P 1} recorded in step S101, it performs detection of a face part by the face detection unit 13, obtains the position, the size, the face parameters orientation, and the like.

  In step S103, the CPU 11 detects a face part by the face detection unit 13 for the through image that is a candidate for the second image that is continuously acquired from the photographing unit 2, as in step S102. Get face parameters.

  In step S104, the CPU 11 determines whether or not the face parameter of the through image acquired in step S103 and the face parameter of the first image acquired in step S102 are greatly different and deviate from a predetermined condition. To do. If this determination is YES, the process proceeds to step S105, and if the determination is NO, the process proceeds to step S106.

  In step S105, since any of the face position, size, orientation, etc. is greatly different between the first image and the second image candidate, the CPU 11 uses the display unit 15 to prompt the user to make adjustments. For example, a warning such as “the first image does not match the face portion” is output. Further, the CPU 11 notifies at least one change of the face parameter of the second image in order to approximate the face parameter of the second image to the face parameter of the first image. Specifically, the CPU 11 outputs, for example, a warning such as “face is too close” or “turn to the left” as the change point of the face parameter to satisfy the predetermined condition in step S104.

  When the user adjusts according to these warning displays, the deviation of the face parameters is improved. The warning output is, instead of or in addition to the display output by the display unit 15, an audio output by a speaker (not shown), a light emitting output of an LED (Light Emitting Diode) indicating a predetermined warning content, or the like. It may be.

  In step S106, the CPU 11 determines whether or not the face parameter of the through image acquired in step S103 approximates the face parameter of the first image acquired in step S102 with a predetermined degree of similarity or more. If this determination is YES, since the face portion can be aligned between the first image and the through image, the process proceeds to step S107. On the other hand, if this determination is NO, it is difficult to align the face portion between the first image and the through image, and the background portion cannot be accurately separated. A new through image as a candidate is acquired.

In step S <b> 107, the CPU 11 determines the through image acquired in step S <b> 103 as the second image P <b> ₂ and records it in the SDRAM 19.

  Next, the image composition process shown in FIG. 4 is executed following the photographing process shown in FIG. 3, and a new image in which the background portion of the person is set to a predetermined value is generated under the control of the CPU 11. Hereinafter, description will be made with reference to the explanatory diagram of FIG.

In step S201, CPU 11 has the face detecting unit 13 from the first image _{P 1} detects the face portion in the person (non-background portion) C, the center _P F (m face portion based on the detected position of the face portion , N) (see FIG. 5).

  In step S202, the CPU 11 detects a face portion by the face detection unit 13 from the second image P2.

In step S203, CPU 11 is calculated by performing positioning for the detected face parameters, the relative displacement of the second image P2 with respect to the first image _{P 1} of the (a, b) by the face detection in step S201 and step S202 To do.

In step S204, CPU 11, based on the size of the detected face portion at step S201, a circle with a radius L ₁ showing the outside of the face portion, and a circle of radius L ₂ included in the face portion, Is set (see FIG. 5).

In step S205, CPU 11 has a _{D 1} is the pixel value of coordinates (x, y) in the first image _{P 1,} the coordinates in the second image _{P 2 (x + a, y} + b) and _{D 2} is a pixel value of the get. Here, the coordinates (x + a, y + b ) is the coordinates (x, y) relative displacement amount calculated in step S203 from the (a, b) are displaced by coordinates, the first image _{P 1} and the second image P Each pixel of ₂ is matched by this relative displacement amount (a, b).

In step S206, the CPU 11 determines whether P ₁ (x, y) is outside the circle with the radius L ₁ . Specifically, if the condition [(x−m) ² + (y−n) ² <L ₁ ² ] is satisfied, the determination is YES, and the process proceeds to step S207. On the other hand, if this condition is not satisfied, the determination is NO and the process proceeds to step S208.

In step S207, since P ₁ (x, y) is within the circle with the radius L ₁ , the CPU 11 determines whether or not P ₁ (x, y) is outside the circle with the radius L ₂ . Specifically, if the condition [(x−m) ² + (y−n) ² > L ₂ ² ] is satisfied, the determination is YES, and the process proceeds to step S209. On the other hand, if this condition is not satisfied, the determination is NO, and it can be determined that it is a non-background part in the region B (see FIG. 5), and the process proceeds to step S210.

In step S208, since P ₁ (x, y) is outside the circle having the radius L ₁ , the CPU 11 determines the direction of the neck when P ₁ (x, y) is viewed from the center position P _F (m, n) of the face. And whether it is within a range of a predetermined angle θ (see FIG. 5). Here, as the direction of the neck, the direction in which the neck will exist is estimated from the arrangement of each component of the detected face part. Note that this range is a range that is assumed to include a human torso portion. For example, the expected angle with respect to the neck direction viewed from the center of the face portion is appropriately designed as a range such as 180 degrees or 120 degrees. Good. If this determination is YES, since P ₁ (x, y) may be a non-background portion such as a neck, shoulder, or trunk, the process proceeds to step S209. On the other hand, if this determination is NO, it can be determined that P ₁ (x, y) is in the region A (see FIG. 5) and is the background portion, and thus the process proceeds to step S211.

In step S209, since P ₁ (x, y) is not in the area A or the area B and is a coordinate that cannot be immediately discriminated as the background portion or the non-background portion, the CPU 11 determines the pixel values D ₁ and D ₂ as To determine whether or not the difference is less than the threshold value R. If this determination is YES, the process proceeds to step S210, and if the determination is NO, the process proceeds to step S211.

In step S210, the CPU 11 determines that P ₁ (x, y) is a non-background portion, and records the pixel values D ₁ and D ₂ in the SDRAM 19 as the pixel values of the composite images 1 and 2, respectively.

In step S211, the CPU 11 determines that P ₁ (x, y) is the background portion, replaces the pixel values D ₁ and D ₂ with the pixel value D ₃ for background, and the pixels of the composite images 1 and 2 respectively. The value is recorded in the SDRAM 19.

In step S212, the CPU 11 determines whether all the pixels are the background portion or the non-background portion, and determines whether the pixel values of the composite images 1 and 2 are recorded. The relative displacement (a, b) for pixels which are not associated with is determined to be a background portion may pixel value D ₃ for the background as a composite image 1 or 2 pixel values. If this determination is YES, the process moves to step S213, and if the determination is NO, the process returns to step S205 to repeat the process for other coordinates.

  In step S213, the CPU 11 displays the composite images 1 and 2 recorded in the SDRAM 19 on the display unit 15, and receives a selection input from the user as to which non-background portion of the composite image is used.

  In step S214, the CPU 11 JPEG encodes the composite image 1 or 2 selected by the user in step S213 using the JPEG conversion unit 18.

In step S215, the CPU 11 records the image data encoded in JPEG in step S214 in the external memory 16. Thus, new image data by replacing the background portion of the person in the first image P ₁ and the second image P ₂ to a predetermined value are generated and recorded.

  As described above, according to the present embodiment, alignment is performed using the facial parameters of the face portions in the two images, so that the background and the non-background can be easily separated regardless of the background pattern. . Furthermore, in face detection, only the approximate shape of the face can be known, but by comparing the corresponding pixel values in the two images after alignment, for example, other than the face such as hair, neck or shoulder The non-background part of can be separated.

  In addition, the region A and the region B are set from the center position and size of the face, and the background and the non-background are discriminated without comparing the pixel values, so that the processing time required for generating the composite image is shortened. The

  Further, the direction of the neck (torso portion) is determined from the face direction, and a predetermined region in this direction is excluded from the region A (region not including a person), so the torso portion other than the face is within the region A. It is possible to suppress erroneous detection of the background portion due to the occurrence of the error.

  Further, since the warning is output when the face parameter is greatly different between the two images, it is possible to prevent a situation in which the background cannot be separated and the background cannot be separated.

  In addition, two composite images are generated and can be selected by the user. When generating an image that is stored for a long time such as an ID photo, the user wants to generate an image with the best possible expression. Therefore, according to the present embodiment, the user can select one of the two favorite ones. it can.

  In this embodiment, the warning output or the determination of the second image is performed by comparing face parameters by face detection. However, the present invention is not limited to this. For example, the CPU 11 measures the degree of focus and the focal length with respect to the face portion that is the subject, outputs a warning when the degree of focus and the focal length deviate, and determines the second image when approximated. Good.

  In the present embodiment, the second image is automatically determined due to similarity of the face parameters, but the present invention is not limited to this. For example, after performing alignment using face parameters, the difference in pixel values in the background portion (for example, region A) is greater than or equal to a predetermined value, and the difference in pixel values in the non-background portion (for example, region B) is less than or equal to the predetermined value. In this case, it may be determined as the second image (shutter is released). As a result, the image values of the background portion deviate from each other, and two pieces of image data that approximate the pixel values of the non-background portion are selected, so that the background and the non-background can be accurately separated.

  In the present embodiment, the position adjustment is performed using the face parameters, but the present invention is not limited to this. For example, you may further align non-background parts (torso part) other than a face. Specifically, based on the face parameters of the face portion detected by face detection, a region having a predetermined positional relationship is estimated as a non-background portion, and pixels such as block matching are estimated for the region estimated as the non-background portion. Perform alignment by comparing each other. In this way, by using a plurality of alignment methods in combination, the CPU 11 can perform alignment with high accuracy.

  In this embodiment, two composite images are generated as new image data. However, the present invention is not limited to this. Three or more images may be acquired, and three or more composite images may be generated. As a result, the user can select one favorite image from many face image candidates.

  In the present embodiment, the background or the non-background is determined based on the pixel value difference. However, the present invention is not limited to this, and the determination result may be corrected. Specifically, when a region of a pixel determined to be a non-background portion is within a predetermined area surrounded by pixels determined to be a background portion, the pixel determined to be a non-background portion within the predetermined area Is corrected as a background portion. In addition, when the area of the pixel determined to be the background portion is within a predetermined area surrounded by the pixels determined to be the non-background portion, the pixel determined to be the background portion within the predetermined area is replaced with the non-background portion. Correct as part. As a result, there is a possibility that a pixel in which the background and the non-background are erroneously determined is corrected and a composite image can be generated with high accuracy.

[Second Embodiment]
Next, 2nd Embodiment of this invention is described based on drawing. In the present embodiment, the operation of the digital camera 1 is different from that of the first embodiment. That is, in the first embodiment, new composite image data is generated based on two pieces of image data recorded in the SDRAM 19. On the other hand, in this embodiment, a plurality of candidate images are recorded for each of the first image and the second image.

  Here, the CPU 11 extracts two images from the acquired three or more images, generates a plurality of combinations of the first image and the second image, and the correlation of the face part for each of the plurality of combinations. It functions as a determination unit that determines the highest combination as the first image and the second image.

  FIG. 6 is a flowchart showing the flow of the photographing process according to the present embodiment. This process is executed instead of the shooting process (FIG. 3) of the first embodiment, and continues to the image composition process (FIG. 4).

  In step S301, the CPU 11 determines whether or not the shutter key has been pressed by the user. The CPU 11 waits until this determination is YES. When the determination is YES, the CPU 11 proceeds to step S302 to acquire the first image.

  In step S <b> 302, the CPU 11 controls the photographing unit 2 to acquire a candidate image that is a candidate for the first image and record it in the SDRAM 19.

  In step S303, the CPU 11 detects a face portion by the face detection unit 13 with respect to the candidate image recorded in step S302, and acquires face parameters such as position, size, and orientation.

  In step S304, the CPU 11 determines whether or not a predetermined number of sheets have been recorded as candidates for the first image. If this determination is YES, acquisition of candidate images for the first image is terminated, and the process proceeds to step S305. On the other hand, if this determination is NO, the process returns to step S302 to record the next candidate image.

  In step S305, the CPU 11 determines whether the shutter key has been pressed by the user. The CPU 11 waits until this determination is YES, and when the determination is YES, the CPU 11 proceeds to step S306 to acquire the second image. In order to prompt the user to press the shutter key for the second time, the CPU 11 preferably performs display output using a display unit such as the display unit 15 or an LED, or performs sound output using a speaker.

  In step S <b> 306, the CPU 11 controls the photographing unit 2 to acquire a candidate image that is a candidate for the second image and record it in the SDRAM 19.

  In step S307, the CPU 11 detects the face portion of the candidate image recorded in step S306 by the face detection unit 13, and acquires face parameters such as position, size, and orientation.

  In step S308, the CPU 11 determines whether or not a predetermined number of sheets has been recorded as candidates for the second image. If this determination is YES, acquisition of candidate images for the second image is terminated, and the process proceeds to step S309. On the other hand, if this determination is NO, the process returns to step S306 to record the next candidate image.

In step S309, the CPU 11 selects one image from the first image candidate image group and one image from the second image candidate image group. At this time, CPU 11 will, for all combinations, to assess the similarity of the face parameters, the similarity is the highest combination is determined as the first image P ₁ and the second image P _2. In addition, it is not limited to the similarity of the face parameter, for example, after performing the alignment by the face parameter, a combination having the highest correlation of the face part or a combination having a low correlation of the part estimated to be the background part. The first image and the second image may be determined.

  As described above, according to the present embodiment, two images are automatically selected from a plurality of candidate images, and the background and the non-background are separated. As a result, an image that can accurately separate the background and the non-background can be selected without bothering the user, so that a composite image can be efficiently generated.

  It should be noted that the present invention is not limited to the first embodiment or the second embodiment, and modifications, improvements, etc. within a scope that can achieve the object of the present invention are included in the present invention.

  For example, in the first embodiment and the second embodiment, the image pickup device has been described as the CCD 23, but another image pickup device such as a CMOS sensor may be used. Further, a part of the function as the displacement measuring means, the associating means, the discriminating means, the setting means, the second discriminating means, the warning means, the determining means, the measuring means, the estimating means, and the storage means is controlled by a control unit different from the CPU 11. It may be operated.

  The present invention is not limited to a digital camera. For example, the present invention may be applied to an imaging device having a still image imaging function such as a mobile phone terminal with a camera, or another image processing device having a function of generating a new image from a plurality of images. Can also be applied. Note that such an image processing apparatus includes a PC (Personal Computer), a PDA (Personal Digital Assistant), and the like that realize the above-described functions by operating based on a predetermined program.

1 is a block diagram illustrating a schematic configuration of a digital camera according to a first embodiment. It is a figure which shows the outline of the process which concerns on 1st Embodiment. It is a flowchart which shows the flow of the imaging | photography process which concerns on 1st Embodiment. It is a flowchart which shows the flow of the image composition process which concerns on 1st Embodiment. It is a figure where it uses for description of the method which discriminate | determines the background part and non-background part which concern on 1st Embodiment. It is a flowchart which shows the flow of the imaging | photography process which concerns on 2nd Embodiment.

Explanation of symbols

C Non-background portion D ₁ , D ₂ Pixel value D ₃ Background pixel value (second predetermined value)
L ₁ predetermined distance (third predetermined value)
L ₂ predetermined distance (fourth predetermined value)
P ₁ first image _{P 2} second image _{P F} face portion of the center R threshold (first predetermined value)
1 Digital camera (image processing device)
2 Shooting unit (acquisition means)
11 CPU (displacement measurement means, association means, discrimination means, setting means, second discrimination means, warning means, determination means, measurement means, estimation means, storage means)
13 Face detection unit (detection means)
15 Display (Warning means)

Claims (12)

Acquisition means for acquiring a first image and a second image that include the same subject and have different backgrounds;
Detecting means for detecting a face portion of the subject from the acquired first image and second image;
A displacement measuring means for measuring a relative displacement between the face portion of the first image and the face portion of the second image;
Association means for associating pixels constituting the first image and pixels constituting the second image based on the relative displacement;
The associated pixels are compared with each other, and if the pixel value difference of the pixel is equal to or greater than a first predetermined value, the pixel is set as a background portion, and the pixel value difference of the pixel is the first predetermined value. If the value is less than the value, a determination unit that sets the pixel as a non-background portion;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
Selecting means for selecting one of the first image and the second image;
An image processing apparatus, further comprising: a setting unit that sets a pixel value of a pixel determined to be a background portion by the determination unit to the same value for the image selected by the selection unit. The image processing apparatus according to claim 2,
The center of the detected face part is determined, a pixel whose distance from the center of the face part is a third predetermined value or more is set as a background part, and the distance from the center of the face part is a fourth predetermined value. A second discriminating unit that uses the following pixel as a non-background portion;
An image processing apparatus that executes the determination unit on an area excluding the determination target of the second determination unit. The image processing apparatus according to claim 3.
The second determining means is based on the orientation of the face part, and it is assumed that a torso part of a human body in which a prospective angle with respect to a neck direction viewed from the center of the face part of the subject is within a predetermined angle range is included. An image processing apparatus that excludes a range to be determined from a discrimination target. The image processing apparatus according to any one of claims 1 to 4,
An image processing apparatus, further comprising: a warning unit that outputs a warning when a feature amount of the face portion of the first image and a feature amount of the face portion of the second image are greatly different. The image processing apparatus according to claim 5.
When the feature amount of the face portion of the first image and the feature amount of the face portion of the second image are greatly different from each other, the warning means determines the first image based on the feature amount of the face portion of the second image. At least one of the position, size, and orientation of the face portion of the second image for approximating the feature amount of the face portion of the two images to the feature amount of the face portion of the first image; An image processing apparatus. The image processing apparatus according to any one of claims 1 to 6,
The acquisition means acquires a new image when the feature amount of the face portion of the first image and the feature amount of the face portion of the second image do not approximate, and sets the new image as the second image. An image processing apparatus. The image processing apparatus according to claim 1,
The acquisition means acquires a plurality of images including a subject as a first image group and a similar background and a plurality of images including a subject as a second image group and a background different from the first image group,
Select one from each of the acquired image groups, generate a plurality of combinations, and for each of the plurality of combinations, the combination having the highest correlation of the face part is selected as the first image and the second image. An image processing apparatus, further comprising: a determination unit that determines The image processing apparatus according to any one of claims 1 to 8,
Measuring means for measuring the degree of focus on the face portion of the subject,
The image processing apparatus characterized in that the detection means detects the degree of focus measured by the measurement means as a feature amount of a face portion of the first image and the second image. The image processing apparatus according to any one of claims 1 to 9,
Estimating means for estimating a non-background portion of the acquired first image and second image based on the detected feature amount of the face portion ;
Together before aligning the Kikao portion, based on the estimation result of the estimating means, the position and the non-background portion other than the face portion of the first image, and a non-background portion other than the face portion of the second image, the In addition, alignment means for associating the pixels constituting the first image and the pixels constituting the second image based on the two alignment results ,
An image processing apparatus further comprising: The image processing apparatus according to claim 1,
An image processing apparatus further comprising a storage unit that stores a non-background portion of the selected image when the user selects a non-background portion of the first image or a non-background portion of the second image. . Computer
Acquisition means for acquiring a first image and a second image that include the same subject and have different backgrounds;
Detecting means for detecting a face portion of the subject from the acquired first image and second image;
A displacement measuring means for measuring a relative displacement between the face portion of the first image and the face portion of the second image;
Association means for associating pixels constituting the first image and pixels constituting the second image based on the relative displacement;
The associated pixels are compared with each other, and if the pixel value difference of the pixel is equal to or greater than a first predetermined value, the pixel is set as a background portion, and the pixel value difference of the pixel is the first predetermined value. If the value is less than the value, a determination unit that sets the pixel as a non-background part,
Program to function as.

Images