JP2007299297A - Image composition device and control method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a subject image so that all persons and a target image are made to be the best shot when the subject image includes a plurality of persons and a target object image. <P>SOLUTION: Three frames of subject images I1, I2 and I3 are obtained by performing consecutive photographing. A face image 71 with the best expressions is obtained among face images 61, 71 and 81 of the same subject. A face image 62 with the best expressions is similarly obtained among face images 62, 72 and 82 of the same subject. A face image 83 with the best expressions is obtained among face images 63, 73 and 83 of the same subject. The obtained face images 71, 62 and 83 are combined with a background image 101 of the subject image I2 of the second frame to obtain a composite image. Therefore, the composite images are mostly an image with good expressions. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image composition device and a control method thereof.

When a large number of people are included in the subject as in a group photo, it is difficult to improve the facial expression of all the people because there are people who close their eyes. For this reason, when taking a group photo, there is one in which each person is individually captured and synthesized (Patent Document 1). However, a good image of each person is not always obtained just because the images are taken individually.
JP 2001-251550 A

In addition, there are those that perform continuous shooting, evaluate the face orientation, eye-browsing, etc. for a specific person image in the obtained multiple-frame subject image, and select a subject image including a highly evaluated person image (Patent Documents 2 and 3).
JP-A-2005-45600 JP 2005-45601 A

  However, the best shot subject image of one person is simply selected.

  An object of the present invention is to obtain a subject image so that all the people and the target image become the best shot when the subject image includes a plurality of persons and target images.

  An image composition device according to a first aspect of the present invention is an image pickup device that picks up a subject and outputs image data representing the subject image, and two or more different face images from the subject image represented by the image data output from the image pickup device. Face detecting means for detecting a part, facial expression evaluation value calculating means for calculating facial expression evaluation values of two or more different face image parts detected by the face detecting means, imaging in the imaging device, detection in the face detecting means Obtained by the control of the imaging device, the face detecting means, and the evaluation value calculating means, and the control means so that the processing and the calculation in the facial expression evaluation value calculating means are repeated a plurality of times continuously. A process for determining a face image portion having a high expression evaluation value from a plurality of identical face image portions detected in a plurality of subject images is performed by two or more different processes. The face image determining means for each of the face image portions, and two or more different face image portions determined by the face image determining means and the background image portion excluding the face image portion are combined to form a single composite subject image. It is characterized by comprising generating means for generating.

  The first invention also provides a suitable control method for the image composition apparatus. That is, in this method, the imaging device images a subject, outputs image data representing the subject image, and the face detection means differs from the subject image represented by the image data output from the imaging device by two or more. The facial image portion is detected, the facial expression evaluation value calculation means calculates facial expression evaluation values of two or more different facial image portions detected by the face detection means, and the control means captures images in the imaging device. The face image determining means controls the imaging device, the face detecting means, and the evaluation value calculating means so that the detection processing in the face detecting means and the calculation in the facial expression evaluation value sending means are continuously repeated a plurality of times. , A process of determining a face image portion having a high facial expression evaluation value from a plurality of identical face image portions detected in a plurality of frame subject images obtained by control of the control means , For each of two or more different face image portions, and the generating means combines two or more different face image portions determined by the face image determining means and a background image portion excluding the face image portion to produce one frame. A composite subject image is generated.

  According to the first invention, the subject is imaged, and image data representing the subject image is obtained. Two or more different face image portions are detected from the subject image represented by the obtained image data. A facial expression evaluation value is calculated for each of the detected two or more different face image portions. Such imaging, face image portion detection processing, and facial expression evaluation value calculation processing are repeated a plurality of times. A process of determining a face image portion having a high expression evaluation value in a plurality of identical face image portions detected in a plurality of subject images is performed for each of two or more face image portions. The determined two or more face image portions and the background image portion excluding the face image portion are combined to generate a single subject image.

  According to the first invention, by obtaining a plurality of frame subject images, a plurality of identical face image portions can be obtained from the plurality of frame subject images. Since the facial image part with a high facial expression evaluation value determined from the same facial image part is combined with the background image part, all facial expression parts included in the composite image have a high facial expression evaluation value It will be a thing. Even if a plurality of face image portions are included, all of the plurality of face image portions have a high expression evaluation value (for example, a good expression).

  The facial expression evaluation value calculating means may further comprise a specifying means for specifying to calculate one evaluation value among a smile, an angry face, a staring face and a crying face.

  Warning means is further provided for warning when a facial image portion whose facial expression evaluation value is greater than or equal to a threshold value among two or more different facial image portions of the composite image generated by the generating device is less than a predetermined value. Also good.

  The relative deviation from the background image portion is eliminated based on the relative deviation amount between each of the two or more different face image portions determined by the face image determination means and the background image portion. Further, a deviation correction means for correcting a deviation of each face image portion may be further provided. In this case, the generation means will synthesize the face image portion corrected by the shift correction means and the background image portion.

  According to a second aspect of the present invention, there is provided an image synthesizing apparatus that images a subject and outputs image data representing the subject image, and two or more different target image portions from the subject image represented by the image data output from the imaging device. Target image detection means for detecting pose evaluation value calculation means for calculating each pose evaluation value of two or more different target image portions detected by the face detection means, imaging in the imaging device, in the target image detection means Obtained by the control of the imaging device, the target image detection means, the evaluation value calculation means, and the control means so that the detection process and the calculation in the pose evaluation value calculation means are repeated a plurality of times in succession. Target image portion having a high pose evaluation value from among a plurality of the same target image portions detected in a plurality of frame subject images A target image determination unit that performs processing for determining each of two or more different target image portions, two or more different target image portions determined by the target image determination unit, a background image portion excluding the target image portion, and And generating means for generating a single composite subject image.

  The second invention also provides a method suitable for the image composition apparatus. That is, in this method, the imaging device images a subject, outputs image data representing the subject image, and the target image detection means detects two or more subject images represented by the image data output from the imaging device. Different target image portions are detected, the pose evaluation value calculating means calculates pose evaluation values for each of two or more different target image portions detected by the target image detecting means, and the control means is used for imaging in the imaging apparatus. , Controlling the imaging device, the target image detecting means, and the evaluation value calculating means so that the detection processing in the target image detecting means and the calculation in the pause evaluation value sending means are repeated a plurality of times in succession, The deciding means includes a plurality of identical target image portions detected in the plurality of subject images obtained by the control of the control means. A process for determining a target image portion having a high evaluation value is performed for each of two or more different target image portions, and the generation means includes two or more different target image portions determined by the target image determination means, and a target image portion. The background image portion except for is combined to generate a single framed subject image.

  According to the second invention, the subject is imaged, and image data representing the subject image is obtained. Two or more different target image portions are detected from the subject image represented by the obtained image data. A pose evaluation value is calculated for each of the two or more detected different target image portions. Such imaging, target image portion detection processing, and pose evaluation value calculation processing are repeated a plurality of times. A process for determining a target image portion having a high pose evaluation value in a plurality of identical target image portions detected in a plurality of subject images is performed for each of two or more target image portions. The two or more determined target image portions and the background image portion excluding the target image portion are combined to generate a single subject image.

  According to the second invention, by obtaining a plurality of frame subject images, a plurality of identical target image portions can be obtained from the plurality of subject images. Since the target image part having a high pose evaluation value determined from among the same target image parts is combined with the background image part, all facial expressions of the target image part included in the composite image have a high pose evaluation value. It will be a thing. Even if a plurality of target image portions are included, all of the plurality of target image portions have a high pose expression evaluation value (substantially the same pose).

  You may further provide the designation | designated means to designate the kind of pose evaluation value calculated in the said pose evaluation value calculation means.

  The image processing apparatus may further include a warning unit that warns when the number of target image portions having a pose evaluation value equal to or greater than a threshold value among two or more different target image portions of the composite image generated by the generation unit is less than a predetermined number. Good.

  The relative deviation from the background image portion is eliminated based on the relative deviation amount between the target image portion and the background image portion of each of the two or more different target image portions determined by the target image determination means. Further, a deviation correction unit for correcting a deviation of each target image portion may be further provided. In this case, the generation means will synthesize the target image portion corrected by the shift correction means and the background image portion.

  The synthesizing means performs, for example, the synthesizing process using the least blurred background image portion or the least noisy background image portion among the plurality of background image portions obtained from the plurality of subject images.

  FIG. 1 shows an embodiment of the present invention and is a block diagram showing an electrical configuration of a digital still camera.

  The entire operation of the digital still camera is controlled by the CPU 1.

  The digital still camera includes an operation device 2 including a power button, a shutter release button, a mode setting dial, a menu button, various buttons such as an OK button, up / down / left / right buttons, and switches. Yes. An operation signal output from the operation device 2 is input to the CPU 1. Modes that can be set by the mode setting dial include various modes such as a normal imaging mode, a composite imaging mode, and a playback mode. The digital still camera is provided with a strobe light emitting device 3 whose light emission is controlled by the CPU 1.

  The lens driving circuit 4 controls the position of the imaging lens 7, and the diaphragm driving circuit 5 controls the aperture value of the diaphragm 8.

  When the image pickup mode is set (the normal image pickup mode and the composite image pickup mode are the same), the light bundle condensed by the image pickup lens 7 is converted into an aperture 8, an infrared cut filter 9, and an optical low pass filter. The light is incident on the light receiving surface of the CCD 11 through 10 and a subject image is formed. When the CCD 11 is driven by the drive circuit 6, a video signal representing a subject image is output from the CCD 11. The video signal is input to the analog front end 12.

  The video signal is subjected to predetermined analog signal processing such as CDS (correlated double sampling), signal amplification, and analog / digital conversion at the analog front end 12. The image data output from the analog front end 12 is given to the main memory 13 and temporarily stored.

  The image data is read from the main memory 13 and given to the display device 21. The subject image obtained by imaging is displayed on the display screen of the display device 21 (so-called through image).

  When the shutter release button is pressed halfway, the image data obtained by imaging is temporarily stored in the main memory 13 as described above. Image data is read from the main memory 13 and input to the integrating circuit 18. In the integrating circuit 18, the luminance components of the image data are integrated. Data representing the obtained integrated value is input to the CPU 1, and the aperture value of the aperture 8 and the shutter speed of the CCD 11 are determined.

  In the normal imaging mode, after that, when the shutter release button is fully pressed, the image data obtained by imaging is temporarily stored in the main memory 13 as described above. Image data is read from the main memory 13 and input to the digital signal processing circuit 14. In the digital signal processing circuit 14, luminance data and color difference data are generated from the input image data. The generated luminance data and color difference data are compressed by the compression / decompression processing circuit 19. The compressed luminance data and color difference data are recorded on the memory card by the external recording device 20.

  In the composite imaging mode, imaging of a plurality of frames is performed, and synthesis is performed using the obtained subject images of the plurality of frames. Therefore, when the shutter release button is pressed, imaging for a predetermined number of frames input by the user is repeated. The first frame of image data obtained by imaging is given to the main memory 13 and temporarily stored in the same manner as described above. The stored image data is input to the face detection / expression recognition circuit 15.

  The face detection / expression recognition circuit 15 detects a face image portion from the subject image represented by the input image data, recognizes the expression of the detected face image portion, and evaluates the evaluation value (expression evaluation). Value). Expression data for calculating the expression evaluation value is stored in the sub memory 17. The sub memory 17 stores facial expression data representing a facial expression of a smile, a facial expression of an angry face, a facial expression of an eyebrows, a facial expression of a crying face, and the like. Each of these facial expressions has generalized features, and data representing these features is stored in the sub memory 17 as facial expression data. For example, a smiley expression has features such as a mouth that opens sideways and a wrinkle that runs outside the nose and beyond both ends of the lips. The characteristics of facial expressions other than smiles can be considered in the same way. It will be understood that the facial expression evaluation value is calculated by using such facial expression data. In addition, data representing facial features such as image parts of eyes, nose, mouth, ears, and cheeks are also stored in the sub-memory 17, and data representing these facial features can be used from the subject image. The image portion of the face can be detected.

  Thus, when face detection processing and facial expression recognition (expression evaluation value calculation) are performed on a subject image for one frame, the next frame is imaged. Similarly, face detection processing and facial expression recognition are performed on image data obtained by imaging the next frame. The imaging process is repeated until the predetermined number of frames input by the user is completed, and the face detection process and facial expression recognition are also repeated. Of course, the face detection process and facial expression recognition may be performed after all of the predetermined number of frames have been imaged even if the next frame subject is not imaged after the face detection process and facial expression recognition is completed. Good.

  In this way, when imaging of a predetermined number of frames, face detection processing, and facial expression recognition are completed, a facial image to be synthesized is determined from among many facial images based on the facial expression evaluation value. Image synthesis is performed using the determined face image or the like. Details of this image composition will be described later. Image data representing the synthesized image is compressed by the compression / decompression processing circuit 19. The compressed composite image data is recorded on the memory card 20 by the external recording device 20.

  Further, in the digital still camera in this embodiment, a person detection / pose recognition circuit 16 may be provided. The person detection / pose recognition circuit 16 recognizes a person image (target image) and its pose from the subject image, and calculates a pose evaluation value. Similar to the data of face detection / expression recognition performed in the face detection / expression recognition circuit 15, data for detecting a human image and pose are recognized, and data for calculating a pose evaluation value is stored in the sub-memory 17. Is done.

  When the reproduction mode is set, the image data recorded on the memory card is read by the external recording device 20 and input to the compression / decompression processing circuit 19. The compression / decompression processing circuit 19 decompresses the data. By applying the decompressed image data to the display device 21, an image represented by the image data recorded on the memory card is displayed.

  2 and 3 are flowcharts showing the processing procedure of the composite imaging mode.

  This image composition imaging mode is included in multiple frame subject images obtained by continuously shooting the same subject multiple times (the shooting interval is the interval at which the obtained multiple frame subject images are considered to be approximately the same). The image with the desired expression is found out from the face images that are present and synthesized into a single frame image.

  First, the number of frames for continuous shooting is input (step 31), and a desired facial expression is selected (step 32). Further, a desired number of people is input (step 33). In this embodiment, a warning is issued when the number of face images having a desired expression among the face images included in the composite image is less than the desired number. The number of people is entered. The number of frames for continuous shooting, selection of a desired facial expression, and input of a desired number of people are all performed by the user. For example, when a menu button is pressed, a menu is displayed on the display screen, and a selection item for the number of continuously shot frames, a selection item for facial expression, and a selection item for the number of people appear in the menu. From the items that appear, the desired number of frames, facial expressions, number of people, etc. are selected with the up / down / left / right buttons, etc., and the number of frames is determined by pressing the enter button. Among the facial expressions, for example, there are good facial expressions (such as a smile) and bad facial expressions (such as an eyelid), and a desired facial expression is selected by the user. Here, it is assumed that three frames are input as the continuous number. It is assumed that a good facial expression is selected as a desired facial expression.

  When the user presses the shutter release button, the number of shots is initialized and continuous shooting is started (steps 34 and 35). When a subject image for the first frame is obtained by shooting, a face image is detected from the subject image.

  FIG. 4 is an example of a subject image I1 obtained by photographing the first frame.

  The subject image I1 of the first frame includes person images 51, 52, and 53. By performing face image detection processing on the subject image I1, face image portions 61, 62, and 63 of the person images 51, 52, and 53 are detected. Of the face image portions 61, 62, and 63, the face image portions 61 and 62 have good expressions, and the face image portion 63 has bad expressions. A background image 91 is formed in the background (around) of these human images 51, 52 and 53.

  Returning to FIG. 2, when the face detection process ends, facial expression recognition / evaluation processing is performed on the detected face image portion, and a facial expression evaluation value is calculated (step 37). Here, since a good facial expression is selected as a desired facial expression, a facial image with a good facial expression is recognized from among the facial image portions 61, 62 and 63 detected from the subject image I1 of the first frame, and The evaluation value of the good facial expression is calculated. As described above, facial expression recognition processing can be performed based on the degree of opening of the mouth and the degree of heel. It can be understood that the facial expression evaluation value is obtained by the facial expression recognition processing of the detected face image portion. When the facial expression evaluation value is obtained for the face image portion of the first frame, the position of the facial image portion from which the facial expression evaluation value is obtained and the facial expression evaluation value are stored.

  Steps 35 to 38 are repeated until shooting of the continuous frame number input by the user is completed. If the continuous frame number has not been shot (NO in step 39), the number of shot frames is incremented (step 40), and the second frame is shot (step 35).

  FIG. 5 is an example of the subject image I2 of the second frame.

  Since the subject image I2 of the second frame is also the same subject as the subject image I1 of the first frame, person images 51, 52, and 53 included in the subject image I1 of the first frame are included. By performing face detection processing on the second subject image I2, face image portions 71, 72, and 73 of the person images 51, 52, and 53 are detected. In the subject image I2 of the second frame, of the face image portions 71, 72 and 73, the face image portion 71 has a good expression, but the face image portions 72 and 73 have a bad expression. The background image 101 is also included in the subject image I2 of the second frame.

  In addition, the third frame is shot.

  FIG. 6 is an example of the subject image I3 of the third frame.

  The third frame subject image I2 is also the same subject as the first frame subject image I1 and the second frame subject image I2, so that the first frame subject image I1 and the second frame subject image I2 respectively. The same person images 51, 52 and 53 as the included person images are included. By performing face detection processing on the subject image I3 of the third frame, the face image portions 81, 82, and 83 of the person images 51, 52, and 53 are detected. In the subject image I3 of the third frame, the face image portion 53 has a good expression, but the face image portion 52 has a bad expression. The background image 102 is also included in the subject image I3 of the third frame. Although the background images 91, 101, and 102 of the three-frame subject images I1, I2, and I3 are substantially the same, the signs are changed unlike the human image.

  Returning to FIG. 2, when shooting for the number of consecutive frames is completed (YES in step 39), a face image having a high facial expression evaluation value is determined among a plurality of face images of the same person (step 41). The three frames of the subject images I1, I2 and I3 include person images 51, 52 and 53, respectively. The three frames of the subject images I1, I2, and I3 are images of the same person, and a plurality of face images 61, 71, and 81 of the same person are obtained by photographing the three frames. Of these face images 61, 71 and 81, it is assumed that the facial expression 71 obtained from the subject image I2 of the second frame has the highest expression evaluation value. Then, the face image 71 is determined as a face image having a high expression evaluation value among the plurality of face images 61, 71 and 81 of the same person.

  Similarly, if the facial expression 62 of the face image 62 obtained from the subject image I1 of the first frame is the highest among the facial images 62, 72 and 82 for the human image 52, the facial expression evaluation value is high. A face image 62 is determined as the face image. Of the face images 63, 73, and 83 for the human image 53, if the facial image 83 obtained from the third frame subject image I3 has the highest facial expression evaluation value, the facial image having a high facial expression evaluation value As a result, the face image 83 is determined.

  When the face image is determined in this way, a background image obtained by removing the face image portion from the subject image is determined (step 42). The background image may be determined in advance, or the background image with the least noise or the background image with the least blur may be selected from the three background images 91, 101, and 102 obtained. If there are many small areas where the level exceeds a predetermined threshold, it is considered that there is a lot of noise, and if there are few such small areas, the noise is considered to be low. A subject image with many high frequency components is considered to have the least blur. In this embodiment, it is assumed that the second background image is used.

  FIG. 7 is an example of the background image 101. As described above, the background image 101 is the portion from which the face image is removed.

  Returning to FIG. 2, when the background image is determined, the amount of deviation between the determined background image 101 and the determined face images 71, 62 and 83 is calculated (step 43). In accordance with the calculated amount of deviation, the deviation of the face images 71, 62 and 83 is corrected so as to coincide with the background image 101 (step 43).

  FIG. 8 shows the deviation between the background image and the face image.

  As described above, the shooting intervals of the three subject images I1, I2 and I3 are short, but there may be a deviation between these three subject images I1, I2 and I3. For example, the background image 101 is obtained from the subject image I2 of the second frame, and the subject image I2 of the second frame includes face image portions 71, 72, and 73. Of the determined face images 71, 62 and 83, the face image 71 is the face image 71 of the second frame, so there is no deviation from the background image 101 of the second frame. Since the face image 62 is the face image 62 of the first frame, there is a positional deviation of Δ12 with respect to the background image 101 of the second frame. Similarly, since the face image 83 is the face image 83 of the third frame, there is a positional deviation of Δ23 from the background image 101 of the second frame. The face image is corrected (position correction, size adjustment, etc.) so that the positional deviation between the background images 101 is eliminated.

  Returning again to FIG. 2, when the face image is corrected, the corrected face image and the background image are combined to obtain a combined image (step 45).

  FIG. 9 shows how to generate a composite image.

  As described above, the face image 71 of the second frame, the face image 62 of the first frame, the face image 83 of the third frame, and the background image 101 of the second frame are combined to obtain a combined image I4. The obtained composite image I4 is an image with many good expressions.

  Referring to FIG. 2, it is confirmed whether the number of face image portions having facial expression evaluation values equal to or greater than the threshold among the face image portions in the composite image is greater than or equal to the number of persons input in step 33 (see FIG. 2). Step 46). If it is not more than the number of people (NO in step 46), a warning is issued to prompt the user to retake (step 47). If it is greater than or equal to the number of persons (YES in step 46), the process of step 47 is skipped.

  Predetermined digital signal processing is performed on the obtained image data representing the synthesized image (step 48), and the synthesized image data is recorded on the memory card (step 49).

  In the above-described embodiment, misalignment correction and the like are performed. However, it is not always necessary to perform misalignment correction when the misalignment correction can be ignored.

  10 to 16 show another embodiment. In the above-described embodiments, facial expression evaluation values of faces are used to increase the number of facial images with good facial expressions. However, in the following embodiments, the number of human images (target images) in the same pose increases. It is what you want to do.

  10 and 11 are flowcharts showing the processing procedure in the composite imaging mode.

  Similarly to the above, the number of continuously shot frames is input (step 111), and a desired pose is selected (step 112). In the embodiment described below, the desired pose is raised both hands, but it is needless to say that any other pose may be used. This pose is selected using a menu in the same manner as the expression is selected. When the desired number of people is input (step 113), the number of frames is initialized and continuous shooting is started (step 114). In this embodiment, if three frames are taken, it goes without saying that any number of frames may be used.

  The first frame is shot and a subject image is obtained (step 115). A person image is detected from the obtained subject image (step 116). The detected pose of the person image is recognized, and a pose evaluation value is calculated (step 117). The position of the detected person image and the calculated pose evaluation value are stored (step 118). When the first frame has been shot (NO in step 119), the number of shots is incremented (step 120), and the second frame and the third frame are shot. The detection of the person image and the calculation of the pose evaluation value are also performed in the subject image of the second frame and the subject image of the third frame.

  FIG. 12 shows an example of the subject image I11 for the first frame.

  The subject image I11 includes person images 131, 132, and 133 and a background image 171. By performing the human image detection process, the image portions 141, 142, and 143 of the human images 131, 132, and 133 are detected from the subject image I11. Pause evaluation values of the detected image portions 141, 142 and 143 are calculated.

  FIG. 13 shows an example of the subject image I12 of the second frame.

  The subject image I12 also includes person images 131, 132, and 133 and a background image 172, and these image portions 151, 152, and 153 are detected. Pause evaluation values of the detected image portions 151, 152, and 153 are calculated.

  FIG. 14 shows an example of the subject image I13 for the third frame.

  The subject image I13 also includes person images 131, 132 and 133 and a background image 173, and these image portions 161, 162 and 163 are detected. Pause evaluation values of the detected image portions 161, 162 and 163 are calculated.

  FIG. 15 is an example of a background image.

  This background image is the background image 172 obtained from the subject image I12 of the second frame. A portion excluding the detected image portions 151, 152, and 153 is a background image 172.

  For the person image 131, the image portion 161 of the third frame subject image I13, for the person image 132, the image portion 142 of the first frame subject image I11, and for the person image 133, the second frame subject image I12. Assume that each image portion 153 has a high pose evaluation value.

  Returning to FIG. 2, when shooting is completed for the number of continuously shot frames input by the user (YES in step 119), a face image having a high pose evaluation value is determined from the same person image (step 121). As described above, the image portions 161, 142, and 153 are determined as face images having a high pose evaluation value among the same person image.

  Thereafter, a background image is determined (step 122). A shift amount between the determined background image and the determined person image is calculated (step 123), and correction is performed so that the calculated shift amount is eliminated (step 124). The corrected person image and the background image are combined to obtain a combined image (step 125).

  FIG. 16 shows a method of generating a composite image.

  As described above, the image portion 161 of the subject image I13 for the third frame, the image portion 142 of the subject image I11 for the first frame, the image portion 153 of the subject image I12 for the second frame, and the background image 172 are synthesized. A composite image I14 is obtained.

  Referring to FIG. 11, it is confirmed whether or not the obtained composite image I14 has an image portion having a pose evaluation value equal to or greater than a threshold value, which is greater than or equal to the input number of people (step 126). If it is not more than the number of persons (NO in step 126), a warning is given to the photographer (step 127). If the number is greater than or equal to the number of persons, the process of step 127 is skipped.

  Thereafter, digital signal processing is performed (step 128), and the composite image data is recorded on the memory card (step 129).

  In the above-described embodiment, a human image is detected and a composite image is generated so that the number of human images in the same pose increases. However, not only a human image but also other target images such as an animal image may be used. Good.

It is a block diagram which shows the electric constitution of a digital still camera. It is a flowchart which shows the process sequence of composite imaging mode. It is a flowchart which shows the process sequence of composite imaging mode. It is an example of a to-be-photographed image. It is an example of a to-be-photographed image. It is an example of a to-be-photographed image. It is an example of a background image. The deviation from the background image is shown. It shows how to compose an image. It is a flowchart which shows the process sequence of composite imaging mode. It is a flowchart which shows the process sequence of composite imaging mode. It is an example of a to-be-photographed image. It is an example of a to-be-photographed image. It is an example of a to-be-photographed image. It is an example of a background image. It shows how to compose an image.

Explanation of symbols

1 CPU
11 CCD
15 Face detection / expression recognition circuit
16 Human detection / pause recognition circuit

Claims (11)

An imaging device for imaging a subject and outputting image data representing the subject image;
Face detection means for detecting image portions of two or more different faces from a subject image represented by image data output from the imaging device;
Facial expression evaluation value calculation means for calculating facial expression evaluation values of two or more different face image portions detected by the face detection means;
The imaging device, the face detection unit, and the evaluation value calculation unit are controlled so that imaging in the imaging device, detection processing in the face detection unit, and calculation in the facial expression evaluation value calculation unit are repeated a plurality of times in succession. Control means,
A process for determining a face image portion having a high expression evaluation value from a plurality of identical face image portions detected in a plurality of subject images obtained by the control of the control means is performed for each of two or more different face image portions. A face image determining means for generating a composite image of a single frame by combining two or more different face image portions determined by the face image determining means and a background image portion excluding the face image portion;
An image synthesizing apparatus.   2. The image synthesizing apparatus according to claim 1, wherein the expression evaluation value calculating means further comprises a specifying means for specifying to calculate one evaluation value among a smile, an angry face, a staring face and a crying face.   Warning means for warning when a facial image portion whose facial expression evaluation value is greater than or equal to a threshold value among two or more different facial image portions of the composite image generated by the generation device is less than a predetermined value is further provided. The image composition device according to claim 1. The relative deviation from the background image portion is eliminated based on the relative deviation amount between each of the two or more different face image portions determined by the face image determination means and the background image portion. Is further provided with a deviation correction means for correcting the deviation of each face image portion,
The generating means synthesizes each face image part corrected by the deviation correcting means and the background image part,
The image composition device according to claim 1. An imaging device for imaging a subject and outputting image data representing the subject image;
Target image detection means for detecting two or more different target image portions from the subject image represented by the image data output from the imaging device;
Pose evaluation value calculating means for calculating pose evaluation values of two or more different target image portions detected by the target image detecting means;
The imaging apparatus, the target image detection means, and the pose evaluation value calculation means so that the imaging in the imaging apparatus, the detection processing in the target image detection means, and the calculation in the pose evaluation value calculation means are repeated a plurality of times in succession. Control means for controlling
A process for determining a target image portion having a high expression evaluation value from a plurality of identical target image portions detected in a plurality of subject images obtained by the control of the control means is performed for each of two or more different target image portions. A target image determining means for generating a single frame composite subject image by combining two or more different target image portions determined by the target image determining means and a background image portion excluding the target image portion;
An image synthesizing apparatus.   6. The image synthesizing apparatus according to claim 5, further comprising designation means for designating a type of a pose evaluation value calculated by the pose evaluation value calculation means.   Claim means further comprising warning means for warning when the number of target image parts having a pose evaluation value equal to or greater than a threshold value among two or more different target image parts of the composite image generated by the generating means is less than a predetermined number. Item 6. The image composition device according to Item 5. The relative deviation from the background image portion is eliminated based on the relative deviation amount between the target image portion and the background image portion of each of the two or more different target image portions determined by the target image determination means. Is further provided with a deviation correction means for correcting the deviation of each target image portion,
The generating means synthesizes each target image portion and background image portion corrected by the deviation correcting means,
The image composition device according to claim 5. The synthesizing means performs the synthesizing process using the least blurred background image part or the least noise background image part among the plurality of background image parts obtained from the subject images of the plurality of frames.
The image composition device according to claim 1 or 5. The imaging device images the subject and outputs image data representing the subject image.
A face detecting means for detecting two or more different face image portions from the subject image represented by the image data output from the imaging device;
Facial expression evaluation value calculating means calculates facial expression evaluation values for each of two or more different face image portions detected by the face detection means;
The imaging means, the face detection means, and the evaluation value calculation so that the control means continuously repeats the imaging in the imaging apparatus, the detection process in the face detection means, and the calculation in the facial expression evaluation value calculation means a plurality of times. Control means,
The face image determining means determines a face image part having a high expression evaluation value from a plurality of identical face image parts detected in a plurality of subject images obtained by the control of the control means. For each different face image part,
A generating unit that combines two or more different face image parts determined by the face image determining unit and a background image part excluding the face image part to generate a single composite subject image;
A method for controlling an image composition apparatus. The imaging device images the subject and outputs image data representing the subject image.
A target image detecting means detects two or more different target image portions from the subject image represented by the image data output from the imaging device;
A pose evaluation value calculating unit calculates a pose evaluation value of each of two or more different target image portions detected by the target image detecting unit;
The imaging means, the target image detection means, and the evaluation are such that the control means continuously repeats the imaging in the imaging apparatus, the detection processing in the target image detection means, and the calculation in the pose evaluation value calculation means a plurality of times. Control the value calculation means,
The target image determining means determines a target image part having a high pose evaluation value from a plurality of identical target image parts detected in a plurality of subject images obtained by the control of the control means. For each different target image part,
A generating unit that combines two or more different target image portions determined by the target image determining unit and a background image portion excluding the target image portion to generate a single composite subject image;
A method for controlling an image composition apparatus.

Images