JP2006121661A - Synthesized image generation method and apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for generating a synthesized image which can generate synthesized image data in which face images of persons which have constant relationship are arranged collectively. <P>SOLUTION: Adjacent synthesis positions are combined, grouping of the synthesis positions which are established on a template is performed, and order of the groups is determined according to the number of synthesis positions belonging to each of the groups. Face images of the persons contained in the image data such a group photography are detected, metric between adjacent person faces is measured, and face images of nearer persons are combined and distributed to each group by the determined order. The distributed face images are stuck on a synthesis position of each of the groups, and synthesized image data is generated. As a result, synthesized image data in which face images of persons which have constant relationship are arranged collectively can be generated easily. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a composite image creation method, a composite image creation apparatus, and a program. For example, a face image of a person is cut out from a group photo and is synthesized with a predetermined template to generate a composite photo. The present invention relates to an image creating apparatus and a program.

  As a method for providing interesting photographic prints, a method of overlaying a decorative frame on a photographed person image and combining and printing is known (for example, see Patent Documents 1 and 2).

  Further, a method of cutting out only a human image portion from an image and synthesizing the image with another background such as a CG or a landscape image has been conventionally known (Patent Document 3).

Furthermore, when an image of a person is closed with eyes closed, a method of generating an image with open eyes by extracting the eye part from the image with open eyes and combining them is also proposed. (Patent Document 4).
JP 2000-22929 A JP 2004-178163 A JP-A-10-210340 JP 2004-54376 A

  By the way, in recent years, as a new service in a sightseeing spot, a service has been proposed in which a face image of a person is cut out from a group photo taken in a sightseeing spot, and is combined with a book split template as shown in FIG. 5 and printed. . In such a book split template, a face image cut out from a group photo is pasted at a pre-set synthesis position on the template. However, the synthesis position is a group of groups scattered on the template. In such a case, if face images are randomly pasted, family members, friends, and couples may be pasted into different groups.

  The present invention has been made in view of such circumstances, and a composite image creation method and composite image creation capable of creating composite image data in which face images of persons having a certain relationship are arranged together are arranged. An object is to provide a device and a program.

  According to an aspect of the present invention, a facial image of each person included in the image data is extracted, and the extracted facial image is synthesized with the template at a synthesis position set in advance on the template. In the composite image data generation method to be generated, the composite positions set on the template are grouped by combining adjacent composite positions, the rank of the group is determined according to the number of composite positions to which the composite positions belong, and included in the image data A person's face image is detected, the relationship between the detected individual face images is detected, and the face images are distributed to the respective groups in the order according to the relationship between the face images. The present invention relates to a composite image creation method characterized in that composite image data is generated by combining with the template at a composite position of each group.

  According to this aspect, the composite positions set on the template are grouped by combining adjacent composite positions, and the rank of the group is determined according to the number of composite positions belonging to each group. A face image of a person included in the image data is detected, a relationship between the detected individual faces is detected, and face images of persons with a short distance are combined and distributed to each group in the order determined above. Then, the distributed face image is combined with the combined position of each group to generate combined image data. That is, in this aspect, persons whose faces are close to each other are assumed to be in a certain relationship, for example, family, friends, and lovers, and are arranged so as to be combined as one unit. Accordingly, it is possible to easily generate composite image data in which face images of persons having a certain relationship are arranged and arranged.

  Note that the “image data” herein includes all of the images that form the image, such as photographs using negatives and prints, slides including a plurality of images, digitized negative images and DSC images, etc. including. The “image data” is intended not only for images captured instantaneously such as photographs, but also for a plurality of images captured continuously in time such as moving images. In the case of a plurality of images captured continuously in time, the shooting time interval of the shooting frames is not particularly limited. In addition, “detecting a relationship between detected individual faces” includes, for example, a case where a distance between faces of adjacent persons is measured.

  According to another aspect of the present invention, the face image of each person included in the image data is extracted, and the extracted face image is combined with the template at a combination position set in advance on the template, thereby combining the image data. In the synthetic image data generation method for generating image data, a human face image included in the image data is detected, a relationship between the detected individual face images is detected, and the face images are grouped according to the relationship between the face images. Then, the face images of the persons belonging to the same group are combined with the template at the combination position set on the template so as to generate the combined image data so that the face images of the persons are arranged at close combination positions. The present invention relates to a composite image creation method characterized by this.

  According to this aspect, the face image of the person included in the image data is detected, and the relationship between the individual face images is detected. Then, the face images are combined and divided into groups, and the face images of the persons belonging to the same group are combined at the combined position set on the template so that the face images of the persons belonging to the same group are arranged at close combined positions. Data is generated. That is, also in this aspect, face images having a certain relationship are arranged so as to be combined as one unit. Accordingly, it is possible to easily generate composite image data in which face images of persons having a certain relationship are arranged and arranged.

  The relationship between face images may be weighted according to the face direction of the person to be measured. In this case, weighting is performed according to the face direction of the person to be measured. For example, if the distance in the horizontal direction is short, it is presumed that the relationship between the two is stronger, and the distance in the horizontal direction is given the greatest weight. Thereby, it is possible to perform a faithful arrangement according to the relationship between the persons included in the image data.

  It is assumed that there is a strong relationship between the faces facing each other, and weighting is performed according to the face direction of the person to be measured so that the face images of those persons are placed close to each other. May be.

  You may weight the relationship between face images according to the size of the face of the person to be measured. In this case, weighting is performed according to the size of the face of the person to be measured. For example, in a relationship between a person with a large face and a person with a small face, the person is assumed to be a parent and child and is given a large weight. Thereby, it is possible to perform a faithful arrangement according to the relationship between the persons included in the image data.

  In this case, weighting may be performed based on the compatibility between the size of the face of the person to be measured (face image size) and the size that can be combined at the combining position on the template. For example, when the size of the face image and the synthesizable size of the synthesis position are substantially the same or close to each other, such a face image may be preferentially synthesized with the template at the synthesis position. This makes it possible to obtain composite image data that has been synthesized in a natural state.

  You may weight the relationship between face images according to the facial expression of the person to be measured. In this case, weighting is performed according to the facial expression of the person to be measured. For example, it is determined that there is a strong relationship between persons with laughing facial expressions, and a large weight is given. Thereby, it is possible to perform a faithful arrangement according to the relationship between the persons included in the image data.

  According to another aspect of the present invention, the face image of each person included in the image data is extracted, and the extracted face image is combined with the template at a combination position set in advance on the template, thereby combining the image data. In the composite image data generation method for generating image data, the image data is captured in two steps, a face image is detected from the two captured image data, the moving direction and the moving amount of each detected face image are measured, Face images with a measured movement amount greater than or equal to a threshold value are detected, and the detected face images are grouped based on the measured movement direction and at least one position before and after the movement, and the face images of each person belonging to the same group are close The face image of each person is synthesized with the template at the synthesis position set on the template so as to be arranged at the synthesis position to generate synthesized image data. For the synthesis image generating method comprising and.

  According to this aspect, the image data is captured in two steps, and a face image is detected from the two captured image data. Then, the movement direction and movement amount of each detected face image are measured, face images whose measured movement amount is equal to or greater than a threshold value are detected, and grouped based on the movement direction and at least one position before and after the movement. The Then, the face image of each person is pasted at the composition position set on the template so that the face images of the persons belonging to the same group are arranged at the close composition position, and the composite image data is generated. That is, in this aspect, when the image data is photographed twice, the second photographing is more in shape than the first photographing, and those having an intimate relationship are photographed close to each other (second photographing) Is taken with close friends), and the face image of each person is measured by measuring the moving direction and moving amount of each person's face image in the first and second shooting. Are grouped from the degree of concentration in the moving direction, and the groups are combined so that they are arranged together on the template. Accordingly, it is possible to easily generate composite image data in which face images of persons having a certain relationship are arranged and arranged.

  Furthermore, according to the template, the hue of the face image to be combined with the template may be adjusted. In this case, since the hue of the face image is adjusted according to the template theme or the like, more interesting composite image data can be obtained.

  Furthermore, a plurality of image data including the composite image data may be sequentially output together with music. In this case, the output timing of the composite image data may be determined according to the music. As a result, the synthesized image data can be output in synchronization with the music, which increases the fun.

  According to another aspect of the present invention, the face image of each person included in the image data is extracted, and the extracted face image is combined with the template at a combination position set in advance on the template, thereby combining the image data. Combined image grouping means for grouping synthesized positions set on the template by combining adjacent synthesized positions, and a synthesized position of each group grouped by the grouping means. A group order setting unit that sets the group order according to the number, a face image detection unit that detects a face image of a person included in the image data, and a relationship between individual face images detected by the face image detection unit A combination of face images according to the relationship between the face images detected by the relationship detection means and the relationship detection means A combined position adjusting unit that assigns face images combined in the order set by the group order setting unit to a group of combined positions grouped by the grouping unit, and a combined position assigned by the combined position adjusting unit. The present invention relates to a composite image creating apparatus comprising: an image compositing unit that composites each face image with a template to generate composite image data.

  According to this aspect, it is possible to easily generate composite image data in which face images of persons having a certain relationship are arranged and arranged.

  According to another aspect of the present invention, the face image of each person included in the image data is extracted, and the extracted face image is combined with the template at a combination position set in advance on the template, thereby combining the image data. A face image detecting means for detecting a face image of a person included in the image data, and a relationship detecting means for detecting a relationship between individual face images detected by the face image detecting means. The grouping means for combining and grouping the face images according to the relationship between the face images detected by the relationship detecting means, and each person so that the face images of the persons belonging to the same group are arranged at a close combined position. Position adjustment means for assigning the face image to the composition position set on the template, and the composition position assigned by the composition position adjustment means. An image synthesizing means for synthesizing the respective face images and templates have to generate combined image data, relating to the composite image generating apparatus, characterized in that it comprises a.

  According to this aspect, it is possible to easily generate composite image data in which face images of persons having a certain relationship are arranged and arranged.

  The relationship detecting means may weight the relationship between the face images according to the direction of the face image to be measured. In this case, it is possible to perform a faithful arrangement according to the relationship between persons included in the image data.

The relationship detection means may weight the relationship between the face images according to the size of the face image to be measured.
In this case, it is possible to perform a faithful arrangement according to the relationship between persons included in the image data.

  The relationship detection means may weight the relationship between the face images according to the facial expression of the person to be measured. In this case, it is possible to perform a faithful arrangement according to the relationship between persons included in the image data.

  According to another aspect of the present invention, the face image of each person included in the image data is extracted, and the extracted face image is combined with the template at a combination position set in advance on the template, thereby combining the image data. A face image detecting means for detecting a face image from two pieces of image data taken in two steps, and measuring a moving direction of each face image detected by the face image detecting means. A moving direction measuring unit that detects a moving amount measuring unit that measures a moving amount of each face image detected by the face image detecting unit, and a face image in which the moving amount measured by the moving amount measuring unit is a threshold value or more, Face image grouping means for grouping detected face images based on the moving direction and at least one position before and after the movement, and the face images of each person belonging to the same group are close A synthesis position adjusting means for assigning each person's face image to a synthesis position set on the template so as to be arranged at the composition position; and each face image is synthesized with the template at the synthesis position assigned by the synthesis position adjustment means. And a composite image creating apparatus. The composite image creating apparatus includes: a composite image generating unit configured to generate composite image data.

  According to this aspect, it is possible to easily generate composite image data in which face images of persons having a certain relationship are arranged and arranged.

  The image synthesizing unit may further adjust the hue of the face image synthesized with the template according to the template.

  Furthermore, an image output means for sequentially outputting a plurality of image data including the composite image data together with music may be provided. In this case, the image output means may determine the output timing of the composite image data according to the music.

  According to another aspect of the present invention, a face image of each person included in image data is extracted, and the extracted face image is combined with the template at a combination position set in advance on the template to generate combined image data. According to the number of combined positions of each group grouped by the grouping means, the combined position grouping means for grouping the combined positions set on the template by combining adjacent combined positions Group order setting means for setting the order of groups, face image detection means for detecting a face image of a person included in image data, and relationship detection means for detecting the relationship between individual face images detected by the face image detection means And combining the face images according to the relationship between the face images detected by the relationship detecting means, A combined position adjusting unit that assigns the face images combined in the order set by the group order setting unit to the group of combined positions grouped in step, a cutout unit that cuts out the face image of each person from the image data, and the cutout A face image of each person clipped by the means is synthesized with the template at a synthesis position on the template assigned by the synthesis position adjustment means, and functions as image synthesis means for generating synthesized image data. The present invention relates to a composite image data generation program.

  According to this aspect, it is possible to easily generate composite image data in which face images of persons having a certain relationship are arranged and arranged.

  According to another aspect of the present invention, a face image of each person included in image data is extracted, and the extracted face image is combined with the template at a combination position set in advance on the template to generate combined image data. A face image detecting means for detecting a face image of a person included in image data, a relationship detecting means for detecting a relationship between individual face images detected by the face image detecting means, and the relationship detection Grouping means for combining and grouping the face images according to the relationship between the face images detected by the means, and the face images of the persons are arranged at close synthetic positions so that the face images of the persons belonging to the same group are arranged close together. A combining position adjusting unit to be assigned to a combining position set on the template, a cutting unit for cutting each person's face image from the image data, and the cutting unit A face image of each person cut out by the take-out means is combined with the template at the composition position on the template assigned by the composition position adjustment means, and functions as image composition means for generating composite image data. This relates to a composite image data generation program.

  According to this aspect, it is possible to easily generate composite image data in which face images of persons having a certain relationship are arranged and arranged.

  According to another aspect of the present invention, a face image of each person included in image data is extracted, and the extracted face image is combined with the template at a combination position set in advance on the template to generate combined image data. A face image detecting means for detecting a face image from two image data photographed in two steps, and a moving direction measurement for measuring the moving direction of each face image detected by the face image detecting means. Means, a movement amount measuring means for measuring a movement amount of each face image detected by the face image detection means, a face image whose movement amount measured by the movement amount measurement means is equal to or greater than a threshold value, and moving the detected face image Face image grouping means for grouping based on the direction and at least one position before and after the movement, so that the face images of the persons belonging to the same group are arranged at close synthetic positions A combination position adjusting unit that assigns a face image combined with a face image of each person to a combination position set on the template, a cutting unit that cuts out the face image of each person from one or both image data, and the cutting unit A composition that functions as an image composition unit that synthesizes the cut face image of each person with the template at a composition position on the template assigned by the composition position adjustment unit to generate composite image data. The present invention relates to an image data generation program.

  According to this aspect, it is possible to easily generate composite image data in which face images of persons having a certain relationship are arranged and arranged.

  Another aspect of the present invention is a composite photo generation method for generating a composite photo by cutting out a face image of each person photographed in a group photo and pasting it on a composite position set in advance on a template. Combining the composite positions together to group the composite positions set on the template, determining the rank of the group according to the number of composite positions to which it belongs, detecting the face image of the person shown in the group photo, Measure the distance between the faces of adjacent people, combine face images of those who are close to each other, distribute them to each group in the order, and paste the distributed face images at the combined position of each group A method for generating a composite photograph is provided.

  According to this aspect, the composite positions set on the template are grouped by combining adjacent composite positions, and the rank of the group is determined according to the number of composite positions belonging to each group. A face image of a person photographed in a group photo is detected, a distance between faces of adjacent persons is measured, and face images of persons with a short distance are combined and distributed to each group in the order determined above. Then, the allocated face image is pasted at the combination position of each group to generate a composite photograph. That is, according to the present invention, persons whose faces are close to each other are assumed to have a certain relationship, for example, family, friends, and lovers, and are arranged so as to be combined into one unit. Thereby, it is possible to easily generate a composite photograph in which face images of persons having a certain relationship are arranged and arranged.

  Another aspect of the present invention is a composite photo generation method for generating a composite photo by cutting out a face image of each person photographed in a group photo and pasting it at a composite position set in advance on a template. Detect face images of people in the group photo, measure the distance between the faces of adjacent people, combine the face images of those who are close to each other, and group them together. Provided is a composite photo generation method characterized in that a composite photo is generated by pasting each person's face image at a composite position set on the template so that the face images are close to the composite position.

  According to this aspect, the face image of the person shown in the group photo is detected, and the distance between the faces of adjacent persons is measured. Then, the face images of persons who are close to each other are combined and grouped, and the face images of the persons in the same group are arranged at close composite positions so that each person's face image is set on the template. To create a composite photo. That is, also in the present invention, persons whose faces are close to each other are presumed to be persons having a certain relationship and are arranged so as to be combined as one unit. Thereby, it is possible to easily generate a composite photograph in which face images of persons having a certain relationship are arranged and arranged.

  Another aspect of the present invention provides a composite photo generation method characterized in that the measured distance is weighted according to the face direction of a person to be measured.

  According to this aspect, the measured distance is weighted according to the face direction of the person to be measured. For example, if the distance in the horizontal direction is short, it is assumed that the relationship between the two is stronger, and the largest weight is assigned to the distance in the horizontal direction. As a result, it is possible to perform a more faithful arrangement according to the relationship of the persons shown in the group photo.

  According to another aspect of the present invention, there is provided a composite photo generation method characterized in that the measured distance is weighted according to the size of the face of the person to be measured.

  According to this aspect, the measured distance is weighted according to the size of the face of the person to be measured. For example, in the relationship between a person with a large face and a person with a small face, the person is assumed to be a parent and child and is given a large weight. As a result, it is possible to perform a more faithful arrangement according to the relationship of the persons shown in the group photo.

Another aspect of the present invention is a composite photo generation method for generating a composite photo by cutting out a face image of each person photographed in a group photo and pasting it at a composite position set in advance on a template. A group photo is taken in two steps, a face image is detected from the two group photos taken, the movement direction and the amount of movement of each detected face image are measured, and a face whose measured movement amount is equal to or greater than a threshold value Each person is detected so that the detected face image is grouped based on the measured moving direction and at least one position before and after the movement, and the face images of the persons belonging to the same group are close to each other. A composite photo generation method is provided in which a composite photo is generated by pasting the face image of the face to a composite position set on the template.

  According to this aspect, the group photo is taken in two steps, and the face image is detected from the two group photos taken. Then, the movement direction and movement amount of each detected face image are measured, a face image whose measured movement amount is equal to or greater than a threshold value is detected, and grouped based on the movement direction and at least one position before and after the movement. . Then, the face images of the persons belonging to the same group are pasted at the composition position set on the template so that the face images of the persons belonging to the same group are arranged at close composition positions. In other words, in the present invention, when a group photo is taken twice, the second time shooting is more like the first time shooting, and those having an intimate relationship are taken closer to each other (second shooting) Is taken with close friends), and the face image of each person is measured by measuring the moving direction and moving amount of each person's face image in the first and second shooting. Are grouped from the degree of concentration in the moving direction, and the groups are combined so that they are arranged together on the template. Thereby, it is possible to easily generate a composite photograph in which face images of persons having a certain relationship are arranged and arranged.

  Another aspect of the present invention is a composite photo generation apparatus that generates a composite photo by cutting out a face image of each person photographed in a group photo and pasting it on a composite position set in advance on a template. A combination position grouping unit that combines adjacent combination positions to group the combination positions set on the template, and sets the group rank according to the number of combination positions of each group grouped by the grouping unit Measuring the distance between the adjacent face images of the face image detected by the face image detecting means and the face image detecting means for detecting the face image of the person photographed in the group photo A combination of the measurement unit and face images measured by the measurement unit are combined and grouped by the grouping unit. Combining position adjusting means for assigning to the group in the order set by the group order setting means, and generating a composite photograph by pasting each face image to the combining position assigned by the combining position adjusting means. Provided is a composite photo generating apparatus.

  According to this aspect, it is possible to easily generate a composite photograph in which face images of persons having a certain relationship are arranged and arranged.

  Another aspect of the present invention is a composite photo generation apparatus that generates a composite photo by cutting out a face image of each person photographed in a group photo and pasting it on a composite position set in advance on a template. Face image detecting means for detecting a face image of a person shown in a group photo, measuring means for measuring a distance between adjacent face images of the face image detected by the face image detecting means, and the measuring means Grouping means for grouping face images with close distances measured in the above and grouping them, and the face images of each person are set on the template so that the face images of the persons belonging to the same group are close to each other. Combining position adjusting means for assigning to the combining position, and generating a composite photograph by pasting each face image to the combining position assigned by the combining position adjusting means. To provide a composite photograph generation device.

  According to this aspect, it is possible to easily generate a composite photo in which face images of persons having a certain relationship are arranged and arranged.

  According to another aspect of the present invention, there is provided a composite photograph generating apparatus characterized in that the measuring unit weights a distance measured according to a direction of a face image to be measured.

  According to this aspect, it is possible to perform a faithful arrangement according to the relationship of the persons shown in the group photo.

  According to another aspect of the present invention, there is provided a composite photograph generating apparatus characterized in that the measuring unit weights a distance measured according to a size of a face image to be measured.

  According to this aspect, it is possible to perform a faithful arrangement according to the relationship of the persons shown in the group photo.

  Another aspect of the present invention is a composite photo generation apparatus that generates a composite photo by cutting out a face image of each person photographed in a group photo and pasting it on a composite position set in advance on a template. A face image detecting means for detecting a face image from two group photographs taken in two steps, a moving direction measuring means for measuring the moving direction of each face image detected by the face image detecting means, A movement amount measuring means for measuring a movement amount of each face image detected by the face image detecting means; a face image whose movement amount measured by the movement amount measuring means is equal to or greater than a threshold; and detecting the detected face image as a movement direction. The face image grouping means for grouping based on at least one position before and after the movement, and the face images of the persons so that the face images of the persons belonging to the same group are close to the combined position. And a composite position adjusting means for assigning to the composite position set above, and generating a composite photograph by pasting each face image to the composite position assigned by the composite position adjusting means Providing equipment.

  According to this aspect, it is possible to easily generate a composite photo in which face images of persons having a certain relationship are arranged and arranged.

  In another aspect of the present invention, a face image of each person photographed in a group photo is cut out and pasted at a synthesis position set in advance on a template, and a computer is used to generate a composite photo. A combination position grouping unit that groups the combination positions that are combined to group the combination positions set on the template, and a group that sets the rank of the group according to the number of combination positions of each group grouped by the grouping unit Rank setting means, face image detection means for detecting a face image of a person photographed in a group photo, measurement means for measuring a distance between adjacent face images of the face image detected by the face image detection means, Combining face images with close distances measured by the measurement means, the grouping means grouped by the grouping means into the combined position group Composite position adjusting means for assigning in the order set by the loop order setting means, cropping means for cutting out each person's face image from the group photo, and assigning the face image of each person cut out by the cutting means by the composite position adjusting means There is provided a composite photo generation program which functions as an image composition means for generating a composite photo by pasting at a composite position on the template.

  According to this aspect, it is possible to easily generate a composite photo in which face images of persons having a certain relationship are arranged and arranged.

Another aspect of the present invention provides a computer for generating a composite photo by cutting out a face image of each person photographed in a group photo and pasting it at a composite position set in advance on a template. Face image detection means for detecting a face image of a person shown in a photograph, measurement means for measuring a distance between adjacent face images of the face image detected by the face image detection means, and measurement by the measurement means Grouping means for combining and grouping face images that are close to each other, and assigning the face images of each person to the combination position set on the template so that the face images of the persons belonging to the same group are close to each other. Compositing position adjusting means, cutting means for cutting out the face image of each person from the group photo, and the face position image of each person cut out by the cutting means as the combining position adjusting hand Be in the image synthesizing means affixed to the synthesis position on the assigned the template to generate a composite photograph function as to provide a composite photograph creation program according to claim.

  According to this aspect, it is possible to easily generate a composite photo in which face images of persons having a certain relationship are arranged and arranged.

  According to another aspect of the present invention, a computer is used to generate a composite photo by cutting out a face image of each person photographed in a group photo and pasting it on a composite position set in advance on a template. Face image detecting means for detecting face images from two group photographs taken in divided times, movement direction measuring means for measuring the moving direction of each face image detected by the face image detecting means, and face image detection A movement amount measuring means for measuring a movement amount of each face image detected by the means; detecting a face image whose movement amount measured by the movement amount measurement means is equal to or greater than a threshold; and detecting the detected face image at least before and after movement. Face image grouping means for grouping based on one position, and the face images of each person are set on the template so that the face images of the persons belonging to the same group are close to each other. The composite position adjusting means to be assigned to the composite position, the cutout means for cutting out the face image of each person from one or both group photographs, and the face image of each person cut out by the cutout means is assigned by the composite position adjusting means. Provided is a composite photo generation program that functions as an image composition unit that generates a composite photo by pasting at a composite position on a template.

  According to this aspect, it is possible to easily generate a composite photo in which face images of persons having a certain relationship are arranged and arranged.

  Note that when the template and the face image are combined, various processes may be performed so that more natural image data can be obtained. For example, it is possible to apply a known technique in which the combined portion of the edge portion of the face image and the template has a more natural color, or to change the inclination of the face image or the like according to the combining position of the template.

  According to the present invention, it is possible to create composite image data in which images of persons having a certain relationship are arranged and arranged.

  First, the overall flow of creating composite image data according to the present invention will be described. FIG. 1 is a diagram showing an example of the overall flow of creating composite image data. First, image data based on a negative, a normal DSC (digital still camera), a DSC attached to a mobile phone or the like, a DV (digital video), or the like is input to the composite image creating apparatus (S1 in FIG. 1). Examples of the image data input at this time include a normal photographed photograph and a group photograph, and may be image data in which a plurality of faces are photographed. The number of image data may be singular or plural.

  In the composite image creation device, a face image is detected from the input image data, and each face image is extracted (S2). The extracted face images are grouped (grouped) based on predetermined face information that suggests the relevance of the face images (S3). Then, a template to be combined with the face image is selected by the user or the like (S4). There may be a single template or a plurality of templates. Then, the composition position on the template relating to the face image is adjusted, and the template and the extracted face image are synthesized at the adjusted composition position (S5), and the composite image data is completed (S6).

  The adjustment of the face image combining position is performed based on the grouping information, and the face image and the template are combined at an appropriate position. Various processes are performed so that the face image and the template are synthesized in a natural state. For example, image processing such as so-called anti-aliasing processing and blur processing is performed on the peripheral portion of the frame of the face image. The completed composite image data may be in any form, for example, one image such as a postcard or a photo, an album in which a plurality of images are arranged together, and one image over time. It is also possible to arrange it in a form such as a slide show that is displayed one by one.

  After the composite image data is completed (S6), the completed composite image data is output as required (S7). The completed composite image data can be distributed and output to any place such as a personal computer (PC), a mobile device (for example, a mobile phone), a laboratory, or the like as required after the composite image data is created. The composite image data is output by any method. For example, the composite image data is output by printing on paper, displaying on a screen, recording on a computer-readable medium such as a hard disk or flash memory, etc. Is done.

  Hereinafter, specific embodiments of a composite image creation method, a composite image creation apparatus, and a program according to the present invention will be described in detail.

(First embodiment)
FIG. 2 is a block diagram showing an embodiment of a composite photo generation apparatus according to the present invention. As shown in the figure, the composite photo generation apparatus 10 of the present embodiment mainly includes an image input unit 12, a face detection unit 14, a template storage unit 16, a template information storage unit 18, an inter-face distance detection unit 20, and a composite position. The adjustment unit 22, the image composition unit 24, the image output unit 26, and the like are included.

  The image input unit 12 takes in image data taken as a group photo. The capturing method is not particularly limited, and may be directly captured from a digital camera or may be captured from a memory card storing image data via a memory card reader. Alternatively, image data transmitted via a network may be received and captured.

  Further, the image data used here is not limited to the one taken with a digital camera, but may be one obtained by reading a photographic print with a scanner.

The face detection unit 14 detects a human face from the image data of the group photograph taken from the image input unit 12. Since a method for detecting a face from an image obtained by photographing a person is a known technique, the detailed detection method is omitted here. As an example, for example, a pixel having a color close to the color designated as the skin color is extracted from the original image, and the extracted region is detected as a face region. In this process, for example, in the color space for distinguishing the skin color from other colors, the range of the skin color in the color space is determined from the information of the skin color sampled in advance, and whether the color of each pixel is in the determined range or not. This is done by determining whether or not.

  The face-to-face distance detection unit 20 detects the distance between the faces of each person detected by the face detection unit 14 with the adjacent person. For example, the center of gravity of the face is used as the detection position. In addition, face parts (eyes, eyes, centers of both eyes, nose, mouth, ears, head, chin, etc.) may be used. Further, the face parts may be the left and right ends of adjacent persons that are different. For example, the right eye of one person and the left eye of the other person, the right ear of one person, the left ear of the other person, or the like may be used.

  The template storage unit 16 stores a plurality of template data used for image composition. In the template information storage unit 18, template information of each template stored in the template storage unit 16, that is, “position information (position information to which a face image is pasted)” of a composite position set in each template is stored. , “Group information (information of grouped composite positions)”, “Distance information between composite positions (distance information between adjacent composite positions)”, and the like are individually stored.

  The composition position adjustment unit 22 determines the arrangement of the face image to be pasted on the template based on the inter-face distance of each person detected by the inter-face distance detection unit 20 and the template information. This will be described in detail later.

  The image composition unit 24 pastes the face image of the person cut out from the image data of the group photo based on the face image arrangement information determined by the composition position adjustment unit 22 at the composition position set on the template, and then composes the composite photo. Is generated.

  The image output unit 26 outputs the composite photograph image data generated by the image composition unit 24 to the monitor. Further, it is output to a printer as necessary, and the generated composite photograph image is printed on a photographic paper.

  FIG. 3 is a flowchart showing a composite photo generation procedure in the composite photo generation apparatus 10 of the present embodiment.

  First, group photo image data used for image composition is input to the image input unit 12 (step S10). The image data input from the image input unit 12 is added to the face detection unit 14, and the face shown in the image is detected (step S11). Based on the detected face position information, the face-to-face distance between adjacent faces is detected by the face-to-face distance detection unit 20 (step S12). The detected distance information between the faces is added to the composite position adjustment unit 22.

  The composition position adjustment unit 22 determines the arrangement of each face image to be pasted on the template based on the distance information between the faces detected by the inter-face distance detection unit 20 and the template information of the template to be used (step S13). ).

FIG. 4 is a flowchart showing a procedure for determining the arrangement of face images. As shown in the figure, first, the composite position adjustment unit 22 sorts the composite position groups according to the number of composite positions belonging to each group, based on the composite position group information included in the template information (step). S20). For example, when there are four groups, three groups, two groups, and a single group as the group of the combination positions, the four groups have the first combination position. The group of the synthesis position is sorted such as two groups, two groups, and a single group. In the case of the template shown in FIG. 5, there are four groups A, three groups B and C, two groups D, and single groups E, F, G, H, and I in the template shown in FIG. When sorted, group A → (B, C) → D → (E, F, G, H, I). In the case of the same number, rearrangement is performed with the advantage that the distance between the combined positions in the group is shorter.

  Next, face images are assigned to each group according to the sorted order (step S21). At this time, face images are assigned by combining persons having a short face-to-face distance. For example, when the face image of the group photo shown in FIG. 6 is combined with the template shown in FIG. 5, the order of the group at the combining position is as follows: group A → (B, C) → D → (E, F, G, H, I), so first assign to group A. Since there are four composite positions belonging to group A, four sets of face images having the closest face-to-face distance are assigned. In FIG. 6, since the face images b, c, d, and e have the shortest face-to-face distance, the face images b, c, d, and e are assigned to the group A. Next, it is assigned to group B. Since there are three composite positions belonging to group B, three sets of face images having the closest face-to-face distance are assigned from the remaining face images excluding the face images b, c, d, and e previously selected. In FIG. 6, the face images g, h, and i are next to the face images b, c, d, and e, and therefore the face images g, h, and i are assigned to the group B.

  In this way, face images are assigned to each group in the sorted order, the face image layout is determined, and the determined information is output to the image composition unit 24 (return to step S14).

  The image compositing unit 24 cuts out the image of each face from the image data of the group photo, and pastes the cut out face image on the combining position on the template based on the determined arrangement information to generate a composite photo (step S14). Then, the generated composite photograph is output to the image output unit 26 (step S15).

  The image output unit 26 outputs the composite photograph generated by the image composition unit 24 to a monitor or a printer and causes the monitor to reproduce and display or print on the photographic paper.

  As described above, in the composite photo generation apparatus 10 according to the present embodiment, persons having a close face-to-face distance in each person photographed in the group photo are combined as a group. Generally, close friends, such as family members and friends, take close-up shots at the time of shooting. Therefore, as shown in this embodiment, arrange close-up faces so that close faces belong to the same group. Can be combined into the same group.

  In the present embodiment, the persons whose adjacent face distances between adjacent persons are simply combined are combined, but the distance value is weighted according to the detected face distance direction or face size. You may do it. For example, as an example of weighting according to the direction, weighting may be performed in the order of the horizontal direction, the diagonal direction, and the vertical direction. As an example of weighting according to the size of the face, a large weight is given to the distance value between the large face (adult face) and the small face (child's face). Note that the weighted distance value may be used as it is, or may be converted into a rank value using a table. By weighting the distance value in accordance with the direction of face distance or the size of the face detected in this way, the face image arrangement is more faithful to the relationship of intimacy between persons photographed in the group photo. be able to.

  Further, in this embodiment, when sorting the groups at the synthesis position, the sort positions are sorted in the order of the number of the synthesis positions belonging to each group, but may be sorted in the order of the synthesis positions in the template. In addition, when there is a face image that is close to the adjacent distance, priority may be given to a face that is close to the position in the group photo and the position in the template.

  Further, when sorting in the order of the composite positions belonging to the group, it may be sorted in the order of decreasing the number of composite positions belonging to each group, contrary to the above embodiment. In the case of the same number, the distance between the combined positions in the group may be considered, and the group having a short distance may be set to be dominant.

  Further, in the present embodiment, the combination positions are grouped and assigned to each group by combining persons having a short distance between faces. First, the persons having a close distance between faces are combined and grouped. Alternatively, the face images of each person may be allocated on the template so that the face images of the persons belonging to the same group are arranged at close synthetic positions. In this case, it is preferable that the groups are sorted according to the number of faces of persons belonging to the group, and assigned to the group at the combination position set on the template in the sorted order.

(Second Embodiment)
Next, a second embodiment of the composite photo generation apparatus according to the present invention will be described.

  The composite photo generation apparatus according to the second embodiment measures the moving direction and moving amount of each face image of the group photo taken in two steps, and determines the degree of concentration of the moving direction of the face image moved over a certain distance. Based on this, the face images are grouped, and the groups are combined so as to be arranged together on the template.

  FIG. 7 is a block diagram illustrating a configuration of the composite photo generation apparatus 30 according to the second embodiment. As shown in the figure, the composite photo generation device 30 of the second embodiment includes a change amount detection unit 32 instead of the inter-face distance detection unit 20 of the composite photo generation device 10 of the first embodiment. It has been. Other configurations are the same as those of the composite photo generation apparatus 10 of the first embodiment described above. Therefore, the same components as those in the composite photo generation apparatus 10 of the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  The change amount detection unit 32 detects the movement amount and the movement direction of each face image from the face positions of the two group photographs detected by the face detection unit 14. That is, based on the position information of each person's face detected from the first group photo and the position information of each person's face detected from the second group photo, the face position of each person is It is detected in which direction and how much it has moved in the second shooting with respect to the first shooting. Information on the movement amount and movement direction of each person's face detected by the change amount detection unit 32 is added to the composite position adjustment unit 22. The composition position adjustment unit 22 determines the arrangement of each face image to be pasted on the template based on the information on the movement amount and movement direction of each person's face detected by the change amount detection unit 32 and the template information.

  FIG. 8 is a flowchart showing a composite photograph generation procedure in the composite image generation apparatus of the present embodiment.

  First, image data of two group photographs used for image composition is input to the image input unit 12 (step S30).

  As described above, since the composite photo generation apparatus 30 according to the present embodiment uses two group photographs, it is assumed that the group photographs are taken twice as a premise for image composition. At this time, it is preferable that the first shooting is performed in a formal arrangement, and the second shooting is performed in a complicated shape.

  The two pieces of image data input from the image input unit 12 are added to the face detection unit 14, and the position of the face shown in each image is individually detected (step S31).

Position information of each face in each image detected by the face detection unit 14 is added to the change amount detection unit 32, and the change amount detection unit 32 detects the face of each face in each image detected by the face detection unit 14. The amount and direction of movement of each face are detected from the position information of the position (step S32). That is, it is detected how much each face has moved in what direction in the second shooting with respect to the first shooting.

  Information on the movement amount and movement direction of each face detected by the change amount detection unit 32 is added to the synthesis position adjustment unit 22, and the synthesis position adjustment unit 22 detects each face detected by the change amount detection unit 32. Face images are grouped based on the information of the movement amount and the movement direction (step S33).

  FIG. 9 is a flowchart showing a procedure for grouping face images.

  First, a face with a small movement amount is checked based on the detected movement amount information of each face (step S40). That is, a face whose movement amount is less than a preset threshold value is checked. Then, the checked face is set at the center of the group (step S41). For example, as shown in FIG. 10, when a group photo of 13 people is taken in two steps, the movement amount of the persons a, c, f, g, h, i, j, l is small. Check c, f, g, h, i, j, l, and set the face position to the center of the group.

  Next, it is determined whether or not there is an unchecked face (step S42). That is, it is determined whether or not there is an image whose movement amount is equal to or greater than a threshold. Here, if it is determined as an unchecked face ant, based on the movement direction of the unchecked face and the position information before (or after) the movement, it is grouped with the central face ahead of the change direction. (Step S43). For example, in the example shown in FIG. 10, since the faces of the persons b, d, e, k, and m are unchecked faces, the faces of the persons b, d, e, k, and m are grouped. At this time, for example, since the face of the person c exists in the direction of the change, the person b is grouped with the face of the person c set at the center. Further, since the face of the person c exists in the direction of the change, the person d is also grouped with the face of the person c set at the center. Also, since the face of the person f exists in the direction of the change, the person e is grouped with the face of the person f set at the center. As described above, based on the movement direction of the unchecked face and the position information of the face before the movement, the face is grouped with the center face in the direction of the change (step S43).

  As a result, in the example shown in FIG. 10, the faces of the persons (b, c, d) and (k, l, m) are set to a group of three persons, and the faces of the persons (e, f) are two persons. Set to group. Then, a, g, h, i, and j are set to a single group.

  Even if the movement amount is equal to or greater than the threshold, if there is no person's face in the direction of change, the movement amount is set to a single group.

  As described above, grouping is performed, and when there is no unchecked face, the grouping process is completed (return to step S34). Then, the groups are sorted according to the number of faces belonging to each group (step S34).

  In the example shown in FIG. 10, since the groups b, c, d and the groups k, l, m are three, the groups e, f are two, and the groups a, g, h, i, j are one, the group (b , C, d) → group (k, l, m) → group (e, f) → group (a) → group (g) → group (h) → group (i) → group (j) Is done.

Next, each group of faces is assigned in the sorted order to each composition position set in the template to be used based on the template information stored in the template information storage unit 18 (step S35). That is, for example, a group with three faces is assigned to a group with three combined positions, and a group with two faces is assigned to a group with two combined positions. In this way, the arrangement of the face images is adjusted so that the groups of the faces are arranged in the group at the same synthesis position. Then, the determined face image arrangement information is output to the image composition unit 24.

  The image compositing unit 24 cuts out each face image from the image data of the second group photo, and pastes each cut out face image at the combining position on the template based on the determined arrangement information to generate a composite photo. (Step S36). Then, the generated composite photograph is output to the image output unit 26 (step S37).

  The image output unit 26 outputs the composite photograph generated by the image composition unit 24 to a monitor or a printer and causes the monitor to reproduce and display or print on the photographic paper.

  As described above, the composite photo generating apparatus 30 according to the present embodiment measures the moving direction and moving amount of each face image of the group photo taken in two steps, and determines the moving direction of the face image moved by a certain distance or more. The face images are grouped based on the degree of concentration, and the group images are combined and arranged on the template. This makes it possible to combine good friends together in the same group.

  In this embodiment, the face image to be pasted on the template is cut out from the image of the second group photo, but it may be cut out from the image of the first group photo and pasted. Alternatively, it may be cut out and pasted from both group photo images.

  In addition, a face image may be cut out from each group photo image to generate two composite photos. When generating two composite photographs, different templates may be used. When the same template is used, the combination position may be changed.

  Further, in this embodiment, when sorting face image groups, the face images are sorted in descending order of the number of faces of persons belonging to the group. However, they may be sorted in ascending order. In addition, when sorting in descending order of the number of faces of persons belonging to a group, if the number of faces of persons belonging to each group is the same, the distance between the faces of persons belonging to each group is taken into consideration. You may make it set so that the group with a short distance may become dominant.

  Further, the two group photographs used for composition do not necessarily have to be taken continuously, and any two of the pictures taken a plurality of times can be selected and used. For example, in the case of shooting five times, the first shot and the third shot may be used, or the second shot and the fifth shot may be used.

(Third embodiment)
In the present embodiment, an example of combining moving image data and a template will be described. The configuration of the present embodiment is substantially the same as the configuration of the first embodiment described above, and the same parts as those of the first embodiment are denoted by the same reference numerals and detailed description thereof is omitted.

FIG. 11 is a diagram illustrating an example in which the present invention is applied to moving image data. In general, a moving image is a display in which a plurality of pieces of image data taken over time at predetermined time intervals are continuously displayed in the order of shooting order. Video shown in FIG. 11, the time T ₀ through T _n n + 1 sheets image data taken in is intended to continuously display the image data at time T ₀ toward the image data at time T _n. Even in the case of such a moving image composed of a plurality of image data, as in the first embodiment described above, a face image is extracted from each image data, and the extracted face image and template are combined, Composite image data can be created. Note that the face image and template combining method and the like are performed by substantially the same method as in the first embodiment described above.

When composite image data is created based on a moving image as shown in FIG. 11, a face image that is copied to the target image data from a plurality of pieces of image data constituting the moving image as a target. By combining with a single template, a single composite image data can be created. In addition, a plurality of image data constituting a moving image is targeted for all or a part of a plurality of image data, and a plurality of synthesized images are synthesized with one or a plurality of templates. Image data can be created. In particular, when a plurality of templates are used, the template to be combined with the face image can be changed according to the image data shooting times T _{0 to} T _n .

  In addition, when a single template and multiple image data that make up a movie are to be combined, the face image changes over time on the template by combining the multiple combined image data created over time. It is also possible to create a new video.

  In this way, a moving image including a plurality of image data is set as a target of the combined image data, thereby creating one combined image data, generating a plurality of combined image data, generating a combined image data of a still image, and Composite image data can be created, and the degree of freedom in creating composite image data is improved.

(Fourth embodiment)
In this embodiment, an example in which a slide show is created by combining composite image data and non-composite image data will be described. Here, the “slide show” refers to one in which a plurality of image data is sequentially reproduced at a predetermined time interval, and predetermined music may be played during reproduction. Note that the predetermined time interval at the time of reproduction may be constant or not constant. The configuration of the present embodiment is substantially the same as the configuration of the first embodiment described above, and the same parts as those of the first embodiment are denoted by the same reference numerals and detailed description thereof is omitted.

  In the present embodiment, composite image data is created using the composite image creation apparatus and composite image creation method of the first embodiment described above, and the image composition unit 24 uses the composite image data and the non-composite image data. An image book is created by arranging and in the order of playback. The image collection created in this way is written by an image output unit 26 onto a hard disk, CD, DVD, flash memory, or other computer-readable medium. Then, each image data of the image collection is sequentially reproduced according to the reproduction order by the image output unit 26 or in cooperation with the image output unit 26 and other devices.

  The composite image data created in the present embodiment is created by synthesizing single or plural image data and single or plural templates. In particular, when creating composite image data based on a plurality of image data, for example, (1) creating composite image data of all image data, (2) every predetermined order based on the playback order of the slide show (3) Create composite image data of designated image data. (4) Create composite image data of image data specified based on music played during a slide show. It is possible to create.

  FIG. 12 is a diagram illustrating an example of a case where composite image data used for a slide show is created based on music played during a slide show. The left side of FIG. 12 is a diagram showing the order in which a plurality of image data of the image collection is reproduced, and the right side of FIG. 12 is a diagram showing the relationship between the degree of music excitement (vertical axis) and time (horizontal axis). It is. The “music excitement level” is determined based on an arbitrary factor. For example, the degree of music excitement depends on musical factors such as the pitch, volume, tempo, rust, etc. Determined based on. In addition, the timing at which music is most excited is determined based on an arbitrary factor. For example, the pitch, volume, or tempo can be converted into frequency components, and the peak point of the frequency component element and the vicinity of the peak point can be set as “the most exciting music”. In the example shown in FIG. 12, at the timing when the music is most exciting (see “A” in FIG. 12), the composite image data using the template (see the black portion in FIG. 12) is reproduced. This makes the slide show more interesting.

  If the image data to be the target of the composite image data is determined in advance, the composite image data is created from the image data and the template, and then the composite image data and the non-composite image data are combined in the order of reproduction. Also good. Prior to creating composite image data, non-synthesized image data is arranged in the order of reproduction, and composite image data is created based on a part or all of the arranged non-synthesized image data and a template. The synthesized image data created in the reproduction order may be inserted.

  In the present embodiment, an example in which an image collection is configured by a plurality of image data obtained by combining composite image data and non-composite image data has been described. However, an image collection may be configured by only composite image data.

  In addition, the image collection including the composite image data can be used for purposes other than the slide show. For example, not only when sequentially reproducing single image data as in a slide show, but also when displaying a plurality of image data at a time as in an album, it is possible to use composite image data of an image collection. it can.

  A desired template is used as appropriate. For example, in the case of a slide show, a desired template may be used for each image. In the case of an album, a template may be used for each page on which a plurality of image data reproduced at a time is combined.

  In addition, music that is played back together with a slide show, an album, or the like is arbitrary music. For example, music that is previously associated with a template is used, or desired music that is selected according to user preferences is used. Or Note that such music data may be stored in a storage unit together with various data of related templates, or may be stored in a separate storage unit.

  The present invention is not limited to the above-described embodiments and modifications thereof, and various modifications such as design changes can be added based on the knowledge of those skilled in the art. The described embodiments can also be included in the scope of the present invention.

  For example, since the composite photo generation apparatus according to the first and second embodiments described above can detect the face shown in the group photo, the number of the faces is calculated and used for order data of the generated composite photo. You may make it do. For example, if nine faces can be detected from the image data of the group photo, the number of composite photos to be printed may be set to nine and automatically printed.

  In addition, since the number of groups can be detected by the composite photo generation apparatus according to the first and second embodiments described above, order data for photo prints may be set in units of groups. For example, for example, when three groups can be detected from the image data of the group photograph, the number of composite photographs to be printed may be set to three and automatically printed. Usually, if it is a family, it is sufficient if there is one print in the family, so unnecessary prints can be prevented.

  In each of the above-described embodiments, an example using a dedicated composite photo generation apparatus has been described. However, the composite photo generation apparatus may be configured by a general computer (for example, a personal computer). That is, the composite photo generation program according to the present invention may be installed in a general computer so that the computer can realize the functions of the composite photo generation apparatus of the above-described embodiment.

  In each of the above embodiments, the example in which the template data and the template information are stored in separate storage units and managed separately has been described, but the present invention is not limited to this. For example, the template information may be associated with the template data in advance as “accompanying information of the template data”, and the template data and the template information may be stored in a single storage unit. In this case, management of template data and template information is facilitated, and related template data and template information can be called by a simple process.

  Further, the face image to be synthesized may be adjusted according to the size of the template synthesis position. For example, when the composition position adjustment unit 22 determines the arrangement of the face image to be synthesized with the template, the size of the face image to be synthesized is normalized according to the size of the composition position to be arranged. And the face image can be synthesized in a natural state. For example, (I) the size of the face image to be adjusted (hereinafter also referred to as “face image reference size”) is set for each template, and the face image to be synthesized is set as the face image adjustment method. A method of matching with a reference size, (II) a method of setting a face image reference size for each combination position, and adjusting a face image to be combined with the face image reference size, etc. are conceivable. Thus, natural composite image data can be obtained even when the size of a plurality of face images to be combined is not uniform, for example, when composite image data is created based on a plurality of image data. Can do. When the face image reference size is different for each combination position, the compatibility between the face image reference size at the combination position and the size of the face image before adjustment may be determined. In this case, the arrangement of the face image on the template can be determined according to the compatibility between the two, or the compatibility between the two can be used as one of the weighting elements that influence the arrangement of the face image on the template. .

  Further, the hue of the face image to be synthesized may be adjusted for each template or each synthesis position using image processing such as so-called setup processing. For example, when the image synthesis unit 24 synthesizes a face image and a template, it is possible to adjust the color of the face image according to a theme defined for each template or a theme defined for each synthesis position. . Specifically, for example, when a summer template is used, every template or each combination position is adjusted so that the hue of the face image arranged at all the combination positions or a specific combination position is adjusted to a tanned face color. A target hue is set in advance, and the hue of the face image arranged at the synthesis position is adjusted to the target hue. As a result, it is possible to obtain composite image data that matches the theme of the template or the theme of the composite position, and it is possible to create composite image data that is more interesting. The theme can be appropriately selected according to the season, time, place, etc., and can be stored in the storage unit together with the template data as supplementary information of the template. Further, the adjustment of the hue of the face image can be partially performed, and for example, only the face color or the hair color can be changed.

  Further, the inclination of the face image to be combined may be adjusted for each template or for each combining position. For example, when the composition position adjustment unit 22 determines the arrangement of the face image to be combined with the template, the composition and the face image are appropriately combined by adjusting the inclination of the face image to be combined according to the combination position. Is possible. Specifically, for example, a target value of the “inclination of the face image” to be synthesized can be determined for each template or each synthesis position, and the inclination of the face image to be synthesized can be adjusted according to the target value. is there. Thereby, the face image can be appropriately arranged on the template.

  In each of the above embodiments, an example in which the measured distance is weighted according to the face direction and the face size of the person to be measured has been described. Relationships may be weighted. For example, it seems that usually good friends often face each other. Therefore, weighting can be performed so that face images of a plurality of persons whose faces are facing each other are grouped. Specifically, for example, the composition position adjustment unit 22 determines that the persons facing each other are a good group and weights the measured distance so that the corresponding face images are arranged nearby, And the composite position can be determined. Note that the determination of “persons facing each other” is made based on the direction of the face image, for example.

  Further, based on the facial expression of the person to be measured, the measured distance may be weighted or the synthesis position may be determined. FIG. 13 is a block diagram showing a configuration of a composite photo generation apparatus according to this modification. In this modification, a face information detection unit 34 for detecting face information is provided instead of the inter-face distance detection unit 20 shown in FIG. The face information detection unit 34 detects facial expressions of a person's facial image to be measured by a known facial expression recognition method, and groups the facial images so that the detected facial expressions are arranged nearby, or weights the measured distances. To do. The composition position adjustment unit 22 determines the composition position of the face image on the template based on the detection result of the face information detection unit 34. This makes it possible to place a plurality of smiling face images and a plurality of surprised face images close together on the template. It is also possible to combine groupings based on facial expressions and groupings based on other elements. For example, a plurality of face images may be grouped first based on the detected facial image expressions, and then the face image in the grouped group may be weighted by distance to determine the face image synthesis position. Is possible. As a facial expression recognition method, any method can be used. For example, known methods disclosed in Japanese Patent Application Laid-Open Nos. 2005-149302 and 2005-044330 can be used. For example, Japanese Patent Laying-Open No. 2005-149302 discloses a face image recognition device and a face image recognition program. These techniques register feature point arrangement information and person IDs of registered face images registered in advance as face templates in association with the registered face images, and use them as feature point arrangement information of a plurality of face images with different facial expressions. Based on this, face deformation information is extracted. Based on the face template and the face deformation information, a person in the face image included in the image can be specified. In Japanese Patent Application Laid-Open No. 2005-044330, a facial image detected as a detection target is filtered by a Gabor filter to extract a facial feature, and a facial expression that recognizes the facial expression of a given facial image based on the facial feature. A recognition method is disclosed. Recognize facial expressions of a given facial image using a facial expression learning data set consisting of a plurality of facial images showing a specific facial expression to be recognized and a plurality of facial images showing facial expressions different from the specific facial expression. A facial expression recognition method is also disclosed. In this facial expression recognition method, a facial image showing a specific facial expression is identified from a given facial image based on facial features extracted by a Gabor filter from the facial expression learning data set. In addition, an apparatus for performing this facial expression recognition method includes a facial expression identifier that identifies a facial image showing a specific facial expression to be recognized from a given facial image. This facial expression classifier is a support vector machine (SVM) that identifies whether the facial feature extracted by the facial feature extraction means is the specific facial expression using a previously learned support vector. May be. The facial expression classifier includes an estimation result output unit including a plurality of weak hypotheses, and a determination unit that determines whether or not a face image provided based on the estimation result output from the estimation result output unit is a specific facial expression. You may have. The estimation result output means estimates whether or not the given face image is a specific facial expression based on the previously learned feature amount and outputs it for each of a plurality of weak hypotheses, and the discrimination means outputs the weak hypothesis estimation result And the reliability learned based on the estimation performance of the weak hypothesis each time one weak hypothesis outputs an estimation result, and whether or not the face image is a specific facial expression based on the addition result It may be determined. In addition, an apparatus for performing the above facial expression recognition method includes a facial expression identifier for identifying a plurality of facial images showing a specific facial expression, and a facial expression of a given facial image based on the identification results of the plurality of facial expression identifiers. You may have the facial expression determination means to determine. This facial expression determination means may determine the facial expression of a given face image based on the addition result of each facial expression classifier. The facial expression classifier is a support vector machine (SVM) that identifies whether the facial feature extracted by the facial feature extraction means is a specific facial expression using a previously learned support vector. May be. The support vector uses a facial expression learning data set including a plurality of facial images indicating specific facial expressions to be recognized and a plurality of facial images indicating facial expressions different from the specific facial expressions. Based on the filtering result of one selected Gabor filter, all Gabor filters selected as the weak hypothesis when the weak hypothesis for estimating whether a given face image is a specific facial expression are repeatedly generated, The facial features may be extracted from the facial expression learning data set and learned in advance based on the extracted facial features.

  Note that a program that causes a computer to function the above-described methods for generating composite image data can also be included in the scope of the present invention.

  The face image grouping method is not limited to the above-described method. For example, it is possible to simply perform grouping according to the frequency of (included) in a photograph (image data), and it is also possible to group people with high frequency into the same group. In addition, even for face images that have been leaked from a higher-ranking group due to grouping, face images that are frequently included (included) in photographs (image data) are defined in advance on the template. It may be synthesized at a specific location. Such a specific place defined on the template is appropriately determined according to the theme of the template and the like. For example, a conspicuous position can be set as the specific place.

  Further, the criteria for grouping facial images are not limited to the above-described method. For example, people who are close friends often stay together when shooting, so the frequency of being included (included) in a photo (image data) together may be one of the criteria for grouping facial images. . In this case, the face image and the template can be synthesized so that the higher the frequency, the closer the face images are arranged.

It is a figure which shows an example of the whole flow of preparation of synthetic image data. 1 is a block diagram showing a first embodiment of a composite photo generation apparatus according to the present invention. It is a flowchart which shows the production | generation procedure of the composite photograph in the composite photograph production | generation apparatus of 1st Embodiment. It is a flowchart which shows the determination procedure of arrangement | positioning of a face image. It is a figure which shows an example of a template. It is a figure which shows an example of a group photograph. It is a block diagram which shows the structure of the composite photograph production | generation apparatus of 2nd Embodiment. It is a flowchart which shows the production | generation procedure of the composite photograph in the composite image generation apparatus of 2nd Embodiment. It is a flowchart which shows the procedure of grouping of a face image. It is a figure which shows an example of a group photograph. It is a figure which shows the example which applied this invention to the moving image which consists of several image data. It is a figure which shows an example in case the synthetic image data used for a slide show are produced based on the music played in the case of a slide show. It is a block diagram which shows the modification of a structure of a composite photo production | generation apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Composite photograph production | generation apparatus, 12 ... Image input part, 14 ... Face detection part, 16 ... Template storage part, 18 ... Template information storage part, 20 ... Inter-face distance detection part, 22 ... Composite position adjustment part, 24 ... Image Composition unit, 26 ... Image output unit, 30 ... Composite photo generation device, 32 ... Change amount detection unit, 34 ... Face information detection unit

Claims (19)

A composite image data generation method for generating composite image data by extracting a face image of each person included in the image data and combining the extracted face image with the template at a composite position set in advance on the template In
Combining adjacent synthesis positions together to group the synthesis positions set on the template,
Decide the rank of the group according to the number of composite positions to which it belongs,
Detect the face image of the person included in the image data,
Detect the relationship between the detected individual face images,
According to the relationship between the face images, the face images are allocated to the groups in the order,
A composite image creation method, wherein composite image data is generated by combining the allocated face image with the template at a combination position of each group. A composite image data generation method for generating composite image data by extracting a face image of each person included in the image data and combining the extracted face image with the template at a composite position set in advance on the template In
Detect the face image of the person included in the image data,
Detect the relationship between the detected individual face images,
Group face images according to the relationship between face images,
The face image of each person belonging to the same group is synthesized with the template at the composition position set on the template so that the face image of each person is arranged at a close composition position. A method for creating a composite image.   The synthetic image creation method according to claim 1, wherein the relationship between the face images is weighted according to the direction of the face of the person to be measured.   The synthetic image creation method according to claim 1, wherein the relationship between the face images is weighted according to the size of the face of the person to be measured.   The synthetic image creation method according to claim 1, wherein the relationship between face images is weighted according to the facial expression of a person to be measured. A composite image data generation method for generating composite image data by extracting a face image of each person included in the image data and combining the extracted face image with the template at a composite position set in advance on the template In
Shoot the image data in two steps,
A face image is detected from the two captured image data,
Measure the moving direction and moving amount of each detected face image,
Detect a face image whose measured movement amount is greater than or equal to a threshold,
Group detected face images based on the measured movement direction and at least one position before and after movement,
The face image of each person belonging to the same group is synthesized with the template at the composition position set on the template so that the face image of each person is arranged at a close composition position. A method for creating a composite image.   Furthermore, according to the said template, the hue of the face image synthesize | combined with the said template is adjusted, The synthesized image production method in any one of Claim 1 thru | or 6 characterized by the above-mentioned. The composite image creation method according to any one of claims 1 to 7, further comprising: sequentially outputting a plurality of image data including the composite image data together with music.
8. The composite image creation method according to claim 1, wherein the output timing of the composite image data is determined according to the music. In a composite image creation device that generates a composite image data by extracting a face image of each person included in image data and combining the extracted face image with the template at a composite position set in advance on the template ,
A combination position grouping unit that combines adjacent combination positions and groups the combination positions set on the template;
Group rank setting means for setting the rank of the group according to the number of combined positions of each group grouped by the grouping means;
Face image detection means for detecting a face image of a person included in the image data;
Relationship detecting means for detecting a relationship between individual face images detected by the face image detecting means;
Faces that are combined in the order set by the group order setting means to the combination position grouped by the grouping means by combining face images according to the relationship between the face images detected by the relation detecting means. A composite position adjusting means for assigning an image;
A composite image creating apparatus comprising: an image composition unit that composes each face image with a template at a composition position assigned by the composition position adjustment unit to generate composite image data. In a composite image creation device that generates a composite image data by extracting a face image of each person included in image data and combining the extracted face image with the template at a composite position set in advance on the template ,
Face image detection means for detecting a face image of a person included in the image data;
Relationship detecting means for detecting a relationship between individual face images detected by the face image detecting means;
Grouping means for combining and grouping face images according to the relationship between the face images detected by the relationship detecting means;
A combination position adjusting means for assigning each person's face image to a combination position set on the template so that the face images of the persons belonging to the same group are arranged at a close combination position;
A composite image creating apparatus comprising: an image composition unit that composes each face image with a template at a composition position assigned by the composition position adjustment unit to generate composite image data.   The composite image creating apparatus according to claim 9, wherein the relationship detection unit weights a relationship between face images according to a direction of the face image to be measured.   The composite image creating apparatus according to claim 9, wherein the relationship detecting unit weights the relationship between the face images according to the size of the face image to be measured.   The composite image creation apparatus according to claim 9 or 10, wherein the relationship detection unit weights a relationship between face images according to facial expressions of a person to be measured. In a composite image creation device that generates a composite image data by extracting a face image of each person included in image data and combining the extracted face image with the template at a composite position set in advance on the template ,
A face image detecting means for detecting a face image from two image data shot in two steps,
A moving direction measuring means for measuring the moving direction of each face image detected by the face image detecting means;
A moving amount measuring means for measuring a moving amount of each face image detected by the face image detecting means;
A face image grouping unit that detects a face image whose movement amount measured by the movement amount measurement unit is equal to or greater than a threshold, and groups the detected face images based on a movement direction and at least one position before and after the movement;
A combination position adjusting means for assigning each person's face image to a combination position set on the template so that the face images of the persons belonging to the same group are arranged at a close combination position;
A composite image creating apparatus comprising: an image composition unit that composes each face image with a template at a composition position assigned by the composition position adjustment unit to generate composite image data.   The composite image creating apparatus according to claim 9, wherein the image composition unit further adjusts a hue of a face image to be composited with the template according to the template. The composite image creation device according to any one of claims 9 to 15, further comprising image output means for sequentially outputting a plurality of image data including the composite image data together with music.
The composite image creating apparatus according to claim 9, wherein the image output unit determines the output timing of the composite image data according to the music. A computer for extracting a face image of each person included in the image data and combining the extracted face image with the template at a pre-set synthesis position on the template to generate composite image data,
A composite position grouping means for grouping composite positions set on the template by combining adjacent composite positions;
Group rank setting means for setting the rank of the groups according to the number of combined positions of the groups grouped by the grouping means;
Face image detecting means for detecting a face image of a person included in the image data;
Relationship detecting means for detecting a relationship between individual face images detected by the face image detecting means;
Faces that are combined according to the order set by the group order setting unit by combining the face images according to the relationship between the face images detected by the relationship detection unit and grouped by the grouping unit. Composite position adjustment means for assigning images,
A cutout unit that cuts out the face image of each person from the image data, and a face image of each person cut out by the cutout unit is combined with the template at a combination position on the template assigned by the combination position adjustment unit. Image synthesis means for generating synthesized image data;
A composite image data generation program characterized by being made to function as: A computer for extracting a face image of each person included in the image data and combining the extracted face image with the template at a pre-set synthesis position on the template to generate composite image data,
Face image detecting means for detecting a face image of a person included in the image data;
Relationship detecting means for detecting a relationship between individual face images detected by the face image detecting means;
Grouping means for combining and grouping face images according to the relationship between the face images detected by the relationship detecting means;
A combination position adjusting means for assigning each person's face image to a combination position set on the template so that the face images of each person belonging to the same group are arranged at a close combination position;
A cutout unit that cuts out the face image of each person from the image data, and a face image of each person cut out by the cutout unit is combined with the template at a combination position on the template assigned by the combination position adjustment unit. Image synthesis means for generating synthesized image data;
A composite image data generation program characterized by being made to function as: A computer for extracting a face image of each person included in the image data and combining the extracted face image with the template at a pre-set synthesis position on the template to generate composite image data,
A face image detecting means for detecting a face image from two image data shot in two steps,
A moving direction measuring means for measuring a moving direction of each face image detected by the face image detecting means;
A moving amount measuring means for measuring a moving amount of each face image detected by the face image detecting means;
Face image grouping means for detecting a face image having a movement amount measured by the movement amount measurement means equal to or greater than a threshold value, and grouping the detected face images based on a movement direction and at least one position before and after the movement;
A combination position adjusting means for assigning each person's face image to a combination position set on the template so that the face images of each person belonging to the same group are arranged at a close combination position;
A cutting unit that cuts out the face image of each person from one or both of the image data, and the face image of each person cut out by the cutting unit at the combined position on the template assigned by the combining position adjusting unit Image synthesizing means for synthesizing and generating synthesized image data;
A composite image data generation program characterized by being made to function as:

Images