JP2006094035A - Image processor and processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor, processing method, and program in which a face image can be compounded appropriately with a template image like a "back cloth" depending on the mode of a group in a photographed image. <P>SOLUTION: A face detecting section 3 detects the face part of a person Fn from a photographed image and detects a marker mk put on a part of the person Fn other than the face part. A group specifying setion 3a specifies a person belonging to the same group among persons whose face part is detected according to a marker mk put on the detected person Fn. A template input section 2 outputs a template image having composite regions Pn of the same number as the total number N of calculated markers to a compounding section 5. A trimming section 4 extracts a face image fn, i.e. the image at the face part of a person Fn belonging to the same group, from the photographed image. An arranging section 5a arranges the face image fn in the composite region Pn and the compounding section 5 compounds the face image fn arranged in the composite region Pn and the template image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program, and more particularly, to an apparatus, method, and program for extracting a face portion of an image in which a person is recorded and synthesizing it at a predetermined position of a template image.

Conventionally, various techniques for easily synthesizing a face image, which is an image of a face portion of a person image, a background image, and a clothing image have been developed. For example, according to Patent Document 1, while specifying two points as a reference for synthesis in a background image, a clothing image, and the like, the hair region of the face image and the inside of the face outline are used as a region for synthesis, Specify two points. Two points serving as a reference for the synthesis of the face image are specified so that the midpoint of the line connecting the two points is on the tip of the jaw on the horizontal line passing through the tip of the jaw and the length is the width of the face. Then, the portrait image is generated by mapping the region used for the synthesis of the face image so that the two points designated for the face image and the two points designated for the background image overlap.
Japanese Patent Laid-Open No. 10-222649

  By the way, recently, the character character's face part is blanked out, and a so-called “skin” -like template is created by inserting a face image extracted from an image obtained by photographing a plurality of persons into the blank part. Images are being developed. In the captured image of the person from whom this face image is extracted, there are a mixture of people who are not related to the group and groups formed by friends and colleagues. Often people are recorded. If a plurality of face images obtained from such a photographed image are mechanically placed in the blank portion of the template image, the face images of persons who are not very close to each other are adjacent to each other, or there is no relationship between the persons belonging to a certain group. There arises a problem that a face image of a person without a mark is arranged. The present invention has been made in view of such problems, and provides an image processing apparatus and method capable of appropriately compositing a face image with a “crisp” template image according to the mode of a group in a captured image. The purpose is to do.

  In order to solve the above-described problem, an image processing apparatus according to the present invention includes a photographed image input unit that inputs a photographed image in which a predetermined marker attached to a part other than a person and a person's face part is recorded, and a person's face part A template image having a detection unit for detecting a predetermined marker, an extraction unit for extracting a face image that is an image of the detected human face portion from the photographed image, and a composite region that is a blank region for arranging the face image A template image input unit for inputting a face image, a placement unit for placing a face image extracted according to whether or not a predetermined marker is detected, in a composite region of the input template image, a face image and a template image placed in the composite region, And a synthesis unit for synthesizing.

  In this image processing apparatus, a face image of a person recorded in a photographed image with a predetermined marker such as a hat or tie attached to a portion other than the face portion of the person is combined with a blank portion of the template image. For this reason, if a group member wears a specific marker such as a hat or tie, even if a group in the captured image and a person unrelated to the group are mixed, a facial image of only the group member And a template image can be obtained.

  The image processing apparatus includes a group identification unit that identifies persons belonging to the same group from persons whose face portions are detected according to a predetermined marker, and the number of persons belonging to the same group from the input template images. A selection unit that selects a template image having the above-described composite region, and the extraction unit extracts a face image that is an image of a face part of a person belonging to the same group from the photographed image, and the arrangement unit is extracted The face image may be arranged in the synthesis area of the selected template image.

  In this way, a template image having a blank portion corresponding to the number of face images of persons belonging to the same group is selected, and the face images of the group members are combined with the blank portion.

  In order to solve the above-described problem, an image processing apparatus according to the present invention includes a captured image input unit that inputs a captured image in which a predetermined marker attached to a portion other than a person and a person's face portion is recorded, A detection unit that detects a face part and a predetermined marker, a group specification part that specifies a person belonging to the same group from persons whose face parts are detected according to the predetermined marker, and a face that is an image of the face part of the person An extraction unit that extracts an image from a photographed image, a template image input unit that inputs a template image having a group area that is a blank area in which face images of persons belonging to the same group are arranged, and a person belonging to the same group An arrangement unit that arranges the face image in the group area and a synthesis unit that synthesizes the face image arranged in the group area and the template image are provided.

  In this image processing apparatus, it is specified that a person recorded in a photographed image belongs to the same group, that is, a member of a certain group, by a predetermined marker such as a hat or a tie attached to a part other than the face part of the person. The Face images of persons belonging to the same group are arranged in the same group area and synthesized with the template image. In other words, if a small group such as a good friend group included in a group wears a specific marker such as a hat or tie, the small group is synthesized into a particularly unity area in the “feel” of the template image. can do.

  The placement unit may place a face image of a person who does not belong to the same group in a composite area that is a blank area that is not included in the group area.

  In order to solve the above-described problem, an image processing method according to the present invention includes a photographed image input step of inputting a photographed image in which a predetermined marker attached to a part other than a person and a person's face part is recorded, A detection step for detecting a face portion and a predetermined marker; an extraction step for extracting a face image that is an image of the detected face portion of a person from a photographed image; and a synthesis region that is a blank region for arranging the face image. A template image input step for inputting a template image, a placement step for placing a face image extracted according to the presence or absence of detection of a predetermined marker in a composite region of the input template image, and a face image and a template placed in the composite region Synthesizing with the image.

  In order to solve the above-described problem, an image processing method according to the present invention includes a photographed image input step of inputting a photographed image in which a predetermined marker attached to a part other than a person and a person's face part is recorded, A detecting step for detecting a face part and a predetermined marker, a group specifying step for specifying a person belonging to the same group from persons whose face parts are detected according to the predetermined marker, and a face which is an image of the face part of the person An extraction step for extracting an image from a photographed image, a template image input step for inputting a template image having a group area that is a blank area for placing face images of persons belonging to the same group, An arrangement step of arranging a face image in a group area; a face image and a template image arranged in the group area; Comprising a synthesizing step of synthesizing a.

  In order to solve the above-described problem, an image processing program according to the present invention includes a captured image input step of inputting a captured image in which a predetermined marker attached to a portion other than a person and a person's face portion is recorded, A detection step for detecting a face portion and a predetermined marker; an extraction step for extracting a face image that is an image of the detected face portion of a person from a photographed image; and a synthesis region that is a blank region for arranging the face image. A template image input step for inputting a template image, a placement step for placing a face image extracted according to the presence or absence of detection of a predetermined marker in a composite region of the input template image, and a face image and a template placed in the composite region And causing the computer to execute a combining step of combining the image.

  In order to solve the above-described problem, an image processing program according to the present invention includes a captured image input step of inputting a captured image in which a predetermined marker attached to a portion other than a person and a person's face portion is recorded, A detecting step for detecting a face part and a predetermined marker, a group specifying step for specifying a person belonging to the same group from persons whose face parts are detected according to the predetermined marker, and a face which is an image of the face part of the person An extraction step for extracting an image from a photographed image, a template image input step for inputting a template image having a group area that is a blank area for placing face images of persons belonging to the same group, Arrangement step for arranging a face image in a group area, and a face image and a template arranged in the group area To perform a synthesizing step of synthesizing an image, to the computer.

  This image processing program has the same effects as the above-described image processing apparatus. The image processing program may be provided by being recorded on a CD-ROM, DVD, MO, or other computer-readable recording medium.

  According to this invention, if a certain group wears a specific marker such as a hat or tie, even if a group in the captured image and a person unrelated to the group are mixed, a face image of only the group members And a template image can be obtained.

  Also, according to the present invention, if a small group such as a good friend group included in a certain group wears a specific marker such as a hat or a tie, the small group is particularly sensed in the “feel” of the template image. Can be synthesized in a certain area.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

<First Embodiment>
[Schematic configuration]
FIG. 1 is a schematic functional configuration diagram of an image processing apparatus 100 according to a preferred first embodiment of the present invention. The image processing apparatus 100 includes a captured image input unit 1, a template input unit 2, a face detection unit 3, a trimming unit 4, a synthesis unit 5, and a template selection unit 6. The captured image input unit 1 inputs captured images acquired by a digital still camera, a film scanner, a media driver, various wireless / wired communication devices, and the like. The face detection unit 3, the trimming unit 4, the synthesis unit 5, and the template selection unit 6 are included in a processing unit 9 configured by a one-chip microcomputer or the like. The face detection unit 3 detects a face image that is a face portion of a person from a captured image by a known face recognition technique. When a plurality of persons are recorded in the captured image, a plurality of face images are individually detected. The trimming unit 4 extracts each detected face image as an independent image from the captured image. The synthesizing unit 5 creates a synthesized image by synthesizing the face image and the template image. The face detection unit 3 includes a group specifying unit 3a, and the combining unit includes an arrangement unit 5a, which will be described in detail later. The image processing apparatus 100 may be connected to a printer 200 that prints a composite image. Although not shown, the image processing apparatus 100 may include an operation unit configured by a keyboard, a touch panel, and the like that accepts user operation input, a media writer that stores a composite image in a predetermined storage medium, and the like.

[Process flow]
Next, the flow of the composition process executed by the image processing apparatus 100 will be described based on the flowchart of FIG.

  In S1, the captured image input unit 1 inputs a captured image. FIG. 3 is an example of a photographed image input to the photographed image input unit 1. A plurality of persons Fn (n = 1, 2,...) Are recorded in the photographed image. In this figure, n = 8 as an example. A person belonging to a specific group such as a family member, a friend, or a colleague is photographed while wearing a marker mk indicating that the person belongs to the same group. Do not let anyone who does not belong to any group wear anything. The marker mk needs to be worn so as not to cover the face part for face detection. For example, a shallow hat or neck tie may be used so as not to cover the face part. In this figure, F1 to F5 are taken with the hat marker mk being worn. Hereinafter, a state in which a person in a captured image wears the marker mk is also referred to as a marker.

  In S2, the face detection unit 3 detects the face part of the person Fn from the photographed image and also detects a marker mk attached to a part other than the face part of the person Fn. The group specifying unit 3a specifies persons belonging to the same group among the persons whose face portions are detected according to the marker mk attached to the detected person Fn. For example, in FIG. 3, if the face portions of the persons F1 to F8 are detected, the persons belonging to the same group are specified as the persons F1 to F5 accompanied by the marker mk.

  In S3, the group specifying unit 3a calculates the number of persons belonging to the same group, that is, the total number N of detected markers mk. Further, a numerical value n for counting the number of times of repeatedly executing step S8 described later is set to n = 0.

  In S <b> 4, the template input unit 2 inputs a template image having the same number of synthesis regions Pn (n = 1 to N) as the total number N of calculated markers, and outputs this template image to the synthesis unit 5. As shown in FIG. 4, the template image is a blank area for synthesizing a face part image fn (hereinafter referred to as a face image) extracted from a person Fn (n = 1 to N) belonging to the same group. Pn (n = 1 to N). When a plurality of template images having different numbers of synthesis regions are input to the template input unit 2, the template selection unit 6 selects a template image having a total number N of synthesis regions from the template images and sends the template image to the synthesis unit 5. Output.

  In S5, the trimming unit 4 extracts a face image fn that is an image of the face portion of the person Fn belonging to the same group from the captured image. In S6, the arrangement unit 5a arranges the face image fn in the synthesis area Pn, and the synthesis unit 5 synthesizes the face image fn arranged in the synthesis area Pn and the template image. The combining unit 5 may combine the face image fn after appropriately performing image processing such as enlargement / reduction, aspect ratio change, centering, and color change so that the face image fn is appropriately combined with the combining region Pn.

  In S7, n is incremented, and in S8, it is determined whether or not n = N. If n = N, the process returns to S5, and if n = N, the process ends. FIG. 5 is a conceptual explanatory diagram of a composite image obtained when this processing is completed. This figure shows a state in which the face images f1 to f5 of the persons F1 to F5 to which the same marker mk is given in FIG. 3 are synthesized in the synthesis areas P1 to P5, respectively.

  Note that an image processing program for causing the processing unit 9 to execute steps S1 to S8 is stored in the storage unit 30 configured by a semiconductor memory. The image processing program may be recorded on a CD-ROM, DVD, MO, or other computer-readable recording medium and provided to the processing unit 9.

  As described above, the face image of a person to whom a specific marker is assigned is synthesized with a template having the same number of synthesis regions as the number of markers. That is, if a group wears a specific marker such as a hat or tie, even if it is recorded on a single photographed image together with a person who is completely unrelated to the group, only the face images of the members of the group are used as templates. It is possible to obtain an image synthesized with the “crisp” portion of the image.

Second Embodiment
FIG. 6 is a block diagram of an image processing apparatus 100 according to the second embodiment of the present invention. Unlike the first embodiment, the image processing apparatus 100 does not include the template selection unit 6 but includes the arrangement unit 5a. As will be described later, the arrangement unit 5a arranges face images belonging to a small group to which the same marker is assigned in a composite area in a predetermined small group area in the template image. A small group means a group included in a large group, such as a close group within a large group such as a school class or a workplace department.

  As shown in FIG. 7, in the captured image of this embodiment, a person who is a member of a large group is recorded, and in the same captured image together with other people who are completely irrelevant as in the first embodiment. It is not supposed to be recorded. FIG. 8 is an example of a template image having a small group region G. The small group area G has synthesis areas g1 to g3 in which face images for three persons belonging to the same small group are arranged. In order to give a sense of unity of the small group, a sentence may be put in the small group region G as shown in FIG. 8, or the distance between the combined regions gn may be made closer to the other combined regions Pn. . Of course, the number of synthesis regions gn in the small group region is not limited to that shown in this figure.

  FIG. 9 is a flowchart showing the flow of the composition process executed by the image processing apparatus 100 according to this embodiment. In S101, the captured image input unit 1 inputs a captured image. However, unlike the first embodiment, the photographed image input here records members belonging to one large group, such as a school class and a workplace department, and the person who becomes a member of the small group is a marker. mk is attached (see FIG. 7).

  In S102, the face detection unit 3 detects a person's face part from the photographed image, and also detects a marker mk attached to a part other than the person's face part. The group specifying unit 3a specifies a person Fn belonging to the same small group and a person Fn 'not belonging to the small group among the persons whose face portions are detected, according to the marker mk attached to the detected person. For example, in FIG. 7, persons belonging to the same small group are identified as F1 to F3, and persons not belonging to the small group are identified as F1 'to F2'.

  In S103, the group specifying unit 3a calculates the number of persons belonging to the same small group, that is, the total number N of detected markers mk and the total number N 'of persons not belonging to the small group. Further, a numerical value n for counting the number of times of repeatedly executing step S108 described later is set to n = 0.

  In S104, the template input unit 2 determines that the small group region G includes the same number of combined regions gn (n = 1 to N) as the total number N of people belonging to the same small group, and the total number N ′ of people not belonging to the small group. A template image having only a synthesis region Pn (n = 1 to N ′) is input, and this template image is output to the synthesis unit 5.

  In S105, the trimming unit 4 extracts a face image fn, which is an image of the face portion of the person Fn belonging to the same small group, from the captured image. In S106, the arrangement unit 5a arranges the face image fn in the synthesis region gn of the small group region G, and the synthesis unit 5 synthesizes the face image fn arranged in the synthesis region gn and the template image. The synthesizing unit 5 may synthesize the face image fn after appropriately performing image processing such as enlargement / reduction, aspect ratio change, centering, and color change so that the face image fn is appropriately synthesized in the synthesis region gn.

  In S107, n is incremented, and in S108, it is determined whether or not n = N. If n = N, the process returns to S105, and if n = N, the process proceeds to S109.

  In S109, n = 0, and in S110, the trimming unit 4 extracts a face image fn ′, which is an image of the face portion of the person Fn ′ not belonging to the small group, from the captured image. In S111, the arrangement unit 5a arranges the face image fn ′ in the synthesis area Pn not included in the small group area G, and the synthesis unit 5 synthesizes the face image fn ′ arranged in the synthesis area Pn and the template image. The synthesizing unit 5 may synthesize the face image fn ′ after appropriately performing image processing such as enlargement / reduction, aspect ratio change, centering, and color change so that the face image fn ′ is appropriately synthesized in the synthesis region Pn. .

  In S112, it is determined whether or not n = N ′. If not n = N ′, the process returns to S110, and if n = N, the process is terminated.

  Note that an image processing program that causes the processing unit 9 to execute steps S101 to S112 is stored in the storage unit 30 configured by a semiconductor memory. The image processing program may be recorded on a CD-ROM, DVD, MO, or other computer-readable recording medium and provided to the processing unit 9.

  FIG. 10 is a conceptual explanatory diagram of a composite image obtained when this processing is completed. In this figure, the face images f1 to f3 of the persons F1 to F3 belonging to the small group to which the same marker mk is assigned in FIG. 7 are respectively combined with the combined regions g1 to g3, and the person F1 ′ not belonging to the small group and F2 ′ shows a state in which face images f1 ′ and f2 ′ of F2 ′ are synthesized in the synthesis areas P1 and P2, respectively.

  As described above, face images of persons belonging to a small group to which a specific marker is attached are combined with the combined region of the small group region. In other words, if a certain small group wears a specific marker such as a hat or a tie, the small group can be synthesized into a particularly united area in the “feel” of the template image.

1 is a schematic functional configuration diagram of an image processing apparatus according to a first embodiment. The flowchart which shows the flow of the synthetic | combination process which concerns on 1st Embodiment. Figure showing the appearance of the captured image Diagram showing the appearance of a template image The figure which shows the state by which the face image was synthesize | combined with the template image Schematic functional configuration diagram of an image processing apparatus according to the second embodiment Figure showing the appearance of the captured image Diagram showing the appearance of a template image Flowchart showing the flow of composition processing according to the second embodiment The figure which shows the state by which the face image was synthesize | combined with the template image

Explanation of symbols

1: shot image input unit, 2: template input unit, 3: face detection unit, 3a: group specifying unit, 4: trimming unit, 5: composition unit, 5a: placement unit, 6: template selection unit, 9: processing unit

Claims (8)

A photographed image input unit for inputting a photographed image in which a predetermined marker attached to a person and a part other than the face part of the person is recorded;
A detection unit for detecting the face portion of the person and the predetermined marker;
An extraction unit that extracts a face image, which is an image of the detected face portion of the person, from the captured image;
A template image input unit for inputting a template image having a composite area which is a blank area in which the face image is arranged;
An arrangement unit that arranges the extracted face image in a composite region of the input template image according to the presence or absence of detection of the predetermined marker;
A synthesis unit that synthesizes the face image and the template image arranged in the synthesis region;
An image processing apparatus comprising: A group specifying unit for specifying persons belonging to the same group from the persons detected in the face portion according to the predetermined marker;
A selection unit that selects, from the input template images, a template image having the composite region equal to or more than the number of persons belonging to the same group;
Further comprising
The extraction unit extracts a face image that is an image of a face portion of a person belonging to the same group from the photographed image,
The image processing apparatus according to claim 1, wherein the arrangement unit arranges the extracted face image in a synthesis region of the selected template image. A photographed image input unit for inputting a photographed image in which a predetermined marker attached to a person and a part other than the face part of the person is recorded;
A detection unit for detecting the face portion of the person and the predetermined marker;
A group specifying unit for specifying persons belonging to the same group from the persons detected in the face portion according to the predetermined marker;
An extraction unit that extracts a face image that is an image of the face portion of the person from the photographed image;
A template image input unit for inputting a template image having a group area which is a blank area in which face images of persons belonging to the same group are arranged;
An arrangement unit that arranges face images of persons belonging to the same group in the group region;
A synthesizing unit that synthesizes the face image arranged in the group region and the template image;
An image processing apparatus comprising:   The image processing apparatus according to claim 3, wherein the arrangement unit arranges a face image of a person who does not belong to the same group in a composite area which is a blank area not included in the group area. A photographed image input step of inputting a photographed image in which a predetermined marker attached to a person and a part other than the face part of the person is recorded;
A detection step of detecting the face portion of the person and the predetermined marker;
An extraction step of extracting a face image, which is an image of the detected human face portion, from the captured image;
A template image input step for inputting a template image having a composite area which is a blank area in which the face image is arranged;
An arrangement step of arranging the extracted face image in a composite region of the input template image according to presence or absence of detection of the predetermined marker;
A synthesis step of synthesizing the face image arranged in the synthesis area with the template image;
An image processing method including: A photographed image input step of inputting a photographed image in which a predetermined marker attached to a person and a part other than the face part of the person is recorded;
A detection step of detecting the face portion of the person and the predetermined marker;
A group specifying step of specifying persons belonging to the same group from persons detected in the face portion according to the predetermined marker;
An extraction step of extracting a face image that is an image of the face portion of the person from the photographed image;
A template image input step for inputting a template image having a group area that is a blank area in which face images of persons belonging to the same group are arranged;
An arrangement step of arranging face images of persons belonging to the same group in the group area;
A synthesizing step of synthesizing the face image arranged in the group region and the template image;
An image processing method including: A photographed image input step of inputting a photographed image in which a predetermined marker attached to a person and a part other than the face part of the person is recorded;
A detection step of detecting the face portion of the person and the predetermined marker;
An extraction step of extracting a face image, which is an image of the detected human face portion, from the captured image;
A template image input step for inputting a template image having a composite area which is a blank area in which the face image is arranged;
An arrangement step of arranging the extracted face image in a composite region of the input template image according to presence or absence of detection of the predetermined marker;
A synthesis step of synthesizing the face image arranged in the synthesis area with the template image;
An image processing program for causing a computer to execute. A photographed image input step of inputting a photographed image in which a predetermined marker attached to a person and a part other than the face part of the person is recorded;
A detection step of detecting the face portion of the person and the predetermined marker;
A group specifying step of specifying persons belonging to the same group from persons detected in the face portion according to the predetermined marker;
An extraction step of extracting a face image that is an image of the face portion of the person from the photographed image;
A template image input step for inputting a template image having a group area that is a blank area in which face images of persons belonging to the same group are arranged;
An arrangement step of arranging face images of persons belonging to the same group in the group area;
A synthesizing step of synthesizing the face image arranged in the group region and the template image;
An image processing program for causing a computer to execute.

Images