JP2005182512A - Image synthesis processing method, device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To synthesize an image showing a boundary part between a face image and a background image more naturally in synthesizing the face image cut out from an original image to the background image having no image only in a face area. <P>SOLUTION: Face parts such as eyes and a mouth are extracted from the face image of the original image, and on the basis of the positional information of each face part, the direction and size of the face of the face image are calculated. In the background image storage part 17, a plurality of background images matching the direction and size of the face are previously stored. A background image selection part 16 selects a background image most closely matching the face image to be synthesized from the background image storage part 17 according to the calculated direction and size of the face. The background image selected in this way is merged with the face image extracted by the face image extraction part 13. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image composition processing method, apparatus, and program, and more particularly to a technique for extracting a face image from an original image including a face image and compositing the face image into a background image without an image of only the face area.

  In general, when a human face is combined with another background image, a process for extracting the outline of the face is required. This contour extraction largely depends on the image quality, background color, and background complexity of the original image, and processing corresponding to all images is very difficult.

Therefore, a technique is known in which a human face is photographed together with a blue single-color background (blue back) and the facial contour is easily extracted using the color difference (Patent Document 1). .
JP-A-9-190522

  However, in the case of an original image taken with a simple background such as a blue background, it is easy to extract a face image, but there is a problem that the background is limited.

  On the other hand, without extracting the outline of the face, the face image is displayed from the back side in the background image without the face area image, such as a face hame commemorative photo panel installed in sightseeing spots or amusement parks. There is a method of creating a composite image in which a face image is combined by pasting, but this method has a problem that the boundary portion between the face image and the background image tends to be unnatural.

  The present invention has been made in view of such circumstances, and when a face image cut out from an original image is combined with a background image without an image of only a face area, a boundary portion between the face image and the background image is more naturally generated. An object of the present invention is to provide an image composition processing method and apparatus and program that can be shown.

  In order to achieve the above object, the invention according to claim 1 is directed to an image composition processing method for extracting a face image from an original image including a face image and combining the face image with a background image without an image of only a face area. Preparing a plurality of background images corresponding to at least one of the orientation and the face size, and calculating at least one of the face orientation and the face size of the face image included in the original image Selecting a background image that best matches the synthesis of the face image based on at least one of the face orientation and the face size of the calculated face image from the plurality of background images; and the selection Synthesizing the face image of the original image with the background image.

  A plurality of background images having no image only in the face area are prepared in advance. The plurality of background images are different from each other in correspondence with at least one of the face orientation and the face size. On the other hand, at least one of the face orientation and the face size of the face image included in the original image is calculated. The face direction and size of the face image can be calculated from the position information of the face parts extracted from the face image such as face eyes and mouth.

  Then, based on at least one of the calculated face orientation and face size, a background image that best matches the synthesis of the face image is selected from a plurality of background images. That is, a background image that faces the same direction as the face of the face image and / or a background image that has a face area with a size corresponding to the face size of the face image is selected. The face image of the original image is synthesized with the face area of the selected background image. Thereby, it is possible to synthesize the boundary portion between the face image and the background image so that they appear more natural.

  The invention according to claim 2 is an image composition processing device that extracts a face image from an original image including a face image, and synthesizes the face image with a background image without an image of the face area. Storage means for storing a plurality of background images corresponding to at least one of them, calculation means for calculating at least one of a face orientation and a face size of a face image included in the original image, and the calculated Selection means for selecting a background image that most closely matches the synthesis of the face image based on at least one of the face orientation and face size of the face image from a plurality of background images stored in the storage means; Image synthesis means for synthesizing the face image of the original image with the selected background image.

  According to a third aspect of the present invention, in the image composition processing device according to the second aspect, the calculation unit includes an extraction unit that extracts eyes and a mouth from a face image included in the original image. At least one of the face orientation and the face size of the face image included in the original image is calculated based on the mouth position information.

  According to a fourth aspect of the present invention, in the image composition processing device according to the second aspect, the image composition means pastes the face image of the original image on the face area of the selected background image from the back side and composes it. It is a feature. That is, a face image larger than the face area of the background image is cut out from the original image without extracting the outline of the face from the original image, and this is pasted from the back side of the background image and synthesized.

  According to a fifth aspect of the present invention, in an image composition processing program for extracting a face image from an original image including a face image and combining the face image with a background image without an image of the face region, the face image included in the original image is synthesized. One background from a storage unit that stores a plurality of background images corresponding to at least one of the face orientation and the face size, and a function for calculating at least one of the face orientation and the face size A function for selecting an image, wherein a background image that best matches the synthesis of the face image based on at least one of the face orientation and the face size of the calculated face image is stored in the storage means; And a function of selecting a plurality of background images and a function of synthesizing the face image of the original image with the selected background image.

  According to the present invention configured as described above, an optimal background image corresponding to the face orientation and size of the face image of the original image to be combined is selected, and the face image is combined with the selected background image. The boundary part between the face image and the background image can be seen more naturally.

  Hereinafter, preferred embodiments of an image composition processing method and apparatus and program according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a functional block diagram showing a main part of an image composition processing apparatus according to the present invention.

  As shown in the figure, the image composition processing apparatus 10 can be constituted by a personal computer, for example, and includes an image data input unit 11, a face part extraction unit 12, a face image extraction unit 13, and a facial image. The orientation calculation unit 14, the face size calculation unit 15, the background image selection unit 16, the background image storage unit 17, the synthesis processing unit 18, and the image data output unit 19 are configured.

  Original image data captured by a digital still camera (hereinafter referred to as DSC), a mobile phone with a camera, or the like is input to the image composition processing apparatus 10 via the image data input unit 11. The original image data is person image data obtained by photographing a person under an arbitrary background.

  In addition to the media interface of DSC media, a USB interface, an infrared communication (IrDA) interface, an Ethernet interface, and a wireless communication interface may be used for the image data input unit 11, and which interface is applied depends on the original image data Is appropriately selected by the user depending on the medium on which the image is recorded and the recording format of the original image data.

  The original image data input to the image composition processing apparatus 10 is sent to the face part extraction unit 12 and the face image extraction unit 13, where face parts and face images are extracted. The face part extraction unit 12 analyzes the input image data and extracts face parts such as eyes, mouths, and jaws. The extraction result is output to the face direction calculation unit 14 and the face size calculation unit 15 as position information of each face part.

  The face image extraction unit 13 performs a process of extracting a face image from the input original image. For example, the face image extraction unit 13 extracts a skin color in the original image, and is a connected region that is considered to belong to the same region from a certain point of the skin color region. The region is sequentially expanded, and a face region is extracted depending on whether or not the shape of the region extracted in this way is a face shape (a shape close to a circle or an ellipse). Is cut out from the original image as a face image.

  The face direction calculation unit 14 and the face size calculation unit 15 calculate the face direction and the face size based on the position information of the face parts extracted by the face part extraction unit 12, respectively. For example, as shown in FIG. 2, the orientation of the face (left and right) depends on the ratio of the horizontal distance d1 between the pupil position of the left eye and the left end position of the mouth and the horizontal distance d2 between the pupil position of the right eye and the right end position of the mouth. Direction angle) can be calculated. Further, the size of the face can be calculated from the vertical distance between the eyes and mouth or between the eyes and chin. The size of the face is, for example, when the image size of the original image is a fixed value, the number of pixels between the distances corresponds to the size of the face.

  The calculation results of the face direction and the face size calculated by the face direction calculation unit 14 and the face size calculation unit 15 are sent to the background image selection unit 16.

  The background image storage unit 17 stores a plurality of types of background images, and stores a plurality of background images corresponding to the face orientation and the face size even for the same type of background images. The user can select a desired type of background image via a predetermined user interface.

  The background image is an image having no image only in the face area (an image in which only the face area is extracted), and as shown in FIGS. 3 and 4, the face orientation and the face size with respect to the same type of background image. A plurality of background images corresponding to the above are prepared.

  3A to 3C show the same kind of background images having the same face size and different face directions, and FIGS. 4A and 4B have the same face direction ( The same kind of background images with different face sizes are shown in FIG. Note that the background images shown in FIGS. 3 and 4 are examples, and background images are stored in the background image storage unit 17 by the number of combinations of face orientations and face sizes.

  The background image selection unit 16 includes a face image face to be synthesized from the background image storage unit 17 based on the face orientation and face size calculation results input from the face orientation calculation unit 14 and the face size calculation unit 15. The background image most suitable for the orientation and the face size is read out, and the background image is output to the composition processing unit 18.

  The synthesis processing unit 18 synthesizes the face image input from the face image extraction unit 13 and the background image input from the background image selection unit 16. That is, the composition processing unit 18 performs a composition process in which the face image input from the face image extraction unit 13 is positioned in the face area of the background image (the area from which the face portion is removed) and pasted from the back side of the background image.

  The synthesized image synthesized by the synthesis processing unit 18 is output from the image data output unit 19. The output form includes a form for outputting an image to a display device and a printer, a form for recording in a recording medium such as a memory card and a CD-R, and a storage device such as a built-in hard disk, and communication means (both wired and wireless). ) And the like, and so on.

  The image composition processing apparatus 10 can be realized by a personal computer in which the image composition processing program for performing the above-described processing is installed, but is not limited thereto, and may be realized by a service server for image processing on a network.

  FIG. 5 is a block diagram of a network system to which the image composition processing method according to the present invention is applied.

  In FIG. 5, reference numeral 130 is a camera-equipped mobile phone that can be connected to a network 140 such as the Internet, and 150 is a user computer (PC) that can be connected to the network 140. The PC 150 is connected to a DSC 152 via an interface such as a USB, and can capture an image from the DSC 152.

  Also connected to the network 140 are a service server 160 that performs image processing similar to the image processing in the image composition processing apparatus 10 described above, a print server 170 that prints a composite image processed by the service server 160, and the like. ing.

  When using the background image composition processing service provided by the service server 160, use the camera-equipped mobile phone 130 or the PC 150 to access the home page of the service server 160 and provide a portrait image for requesting the background image composition processing. Upload to server 160. In addition, the service server 160 can present a list of background images to the user and allow the user to select a background image.

  The service server 160 stores a server computer 162 having a function similar to that of the image composition processing apparatus 10 shown in FIG. 1 and a communication function, an image uploaded from a user, and a user ID, e-mail address, etc. And a large capacity recording device (storage) 164 for managing information. When the service server 160 receives a request for a background image composition process for an original image uploaded from a user, the service server 160 extracts a face image from the original image and, as described above, a background that most closely matches the face orientation and size. An image is selected, and the selected background image and face image are combined. The composite image created in this way is provided to the user's camera-equipped mobile phone 130 and PC 150 via the network 140.

  As an image providing method, there is a mode in which image data is sent back to the requesting user terminal (in this case, the camera-equipped mobile phone 130 or the PC 150), or a mode in which the image data is attached and distributed. There is also an aspect in which the composite image is stored in a predetermined storage location (for example, a dedicated browsing site set for each user), and the address information (for example, URL) is notified by e-mail or the like. In this case, the user can access the designated site and browse or download the facial expression change image from there.

  In this way, the user can view the composite image generated by the service server 160 on the display 131 of the camera-equipped mobile phone 130 or the display 151 of the PC 150.

  When the service server 160 receives a print order of a composite image from the user, the service server 160 transfers the composite image to the print server 170. The print server 170 includes a server computer 172 and a print device 174, and prints out a composite image in which the background is combined by the print device 174 based on the composite image received from the service server 160. The printed photo print is delivered to a receiver such as a convenience store or a photo store designated by the user, or directly to the user's home.

  Next, the flow of processing from taking a portrait image with a camera-equipped mobile phone until obtaining a background image will be described.

  As shown in FIG. 6, a portrait image is taken by the camera-equipped mobile phone 130, and the portrait image is uploaded to the service server 160 (step S10). The service server 160 acquires the uploaded portrait image and presents a list of background images to be combined with the portrait image to the user (step S12).

  When an instruction to select or combine a background image is received from the camera-equipped mobile phone 130 (step S14), the service server 160 detects the position of facial parts such as eyes and mouth from the captured portrait image (step S16). Based on the position information of each face part, the face orientation and face size of the portrait image are calculated (steps S18 and S20).

  Subsequently, based on the calculated face orientation and face size of the portrait image, a background image that most closely matches the face orientation and face size is selected from a plurality of background images stored in the background image storage unit. Select (step S22). Then, the selected background image and the face image of the portrait image are combined (step S24), and this combined image is attached to an e-mail and transmitted to the camera-equipped mobile phone 130.

  The user can display the composite image on the display 131 of the camera-equipped mobile phone 130 by opening the attached file (image file) of the e-mail transmitted from the service server 160 (step S26).

FIG. 1 is a functional block diagram showing an embodiment of an image composition processing apparatus according to the present invention. FIG. 2 is a diagram used for explaining a method of measuring the face orientation and the face size. 3A to 3C are diagrams showing examples of background images having the same face size and different face orientations. 4A and 4B are diagrams illustrating examples of background images having the same face direction (front) and different face sizes. FIG. 5 is a block diagram of a network system to which the image composition processing method according to the present invention is applied. FIG. 6 is a flowchart showing the flow of processing from taking a portrait image with a camera-equipped mobile phone until obtaining a background image.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Image composition processing apparatus, 12 ... Face parts extraction part, 13 ... Face image extraction part, 14 ... Face direction calculation part, 15 ... Face size calculation part, 16 ... Background image selection part, 17 ... Background image memory | storage , 18 ... composite processing unit, 130 ... mobile phone with camera, 140 ... network, 150 ... PC, 160 ... service server

Claims (5)

In an image composition processing method that cuts out a face image from an original image including a face image, and synthesizes the face image with a background image without an image of only the face region,
Preparing a plurality of background images corresponding to at least one of a face orientation and a face size in advance;
Calculating at least one of a face orientation and a face size of a face image included in the original image;
Selecting from the plurality of background images a background image that most closely matches the synthesis of the face image based on at least one of the face orientation and the face size of the calculated face image;
Synthesizing the face image of the original image with the selected background image;
An image composition processing method. In an image composition processing apparatus that cuts out a face image from an original image including a face image, and synthesizes the face image with a background image without an image of only the face region,
Storage means for storing a plurality of background images corresponding to at least one of the face orientation and the face size;
Calculating means for calculating at least one of a face orientation and a face size of a face image included in the original image;
Selection for selecting a background image that best matches the synthesis of the face image based on at least one of the face orientation and the face size of the calculated face image from a plurality of background images stored in the storage means Means,
Image synthesizing means for synthesizing the face image of the original image with the selected background image;
An image composition processing apparatus comprising:   The calculation means includes extraction means for extracting eyes and mouth from a face image included in the original image, and the orientation of the face of the face image included in the original image based on the extracted position information of the eyes and mouth The image composition processing apparatus according to claim 2, wherein at least one of the size and the face size is calculated.   The image composition processing apparatus according to claim 2, wherein the image composition unit pastes the face image of the original image on the face area of the selected background image from the back side to compose the image. In an image composition processing program that cuts out a face image from an original image including a face image and synthesizes the face image with a background image without an image of only the face region,
A function of calculating at least one of a face orientation and a face size of a face image included in the original image;
A function of selecting one background image from a storage unit that stores a plurality of background images corresponding to at least one of the face orientation and the face size, the face orientation of the calculated face image and A function of selecting a background image that most closely matches the synthesis of the face image based on at least one of the face sizes from a plurality of background images stored in the storage means;
A function of synthesizing the face image of the original image with the selected background image;
An image composition processing program for causing a computer to realize the above.

Images