JP2000036032A - Foreground picture extracting method, picture processor, automatic trimming device, recording medium and portrait picture device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide method/device for extracting a foreground picture by means of deleting a background from one picture, a device for automatically trimming the extracted picture, a recording medium stored with a program code for realizing them by a computer and a portrait picture device as a device synthesizing them. SOLUTION: A difference between a background picture where only a background is previously photographed and an objective picture is obtained. The change quantity (difference) of brightness by the difference of both pictures at the time of photographing is corrected and therefore a mask picture is generated. An expansion/reduction processing is executed, a boundary between the area of the object and the area of the background is discriminated and the areas are divided. Thus, the mask picture of a man is generated and it is overapped with an objective picture so as to synthesize them. Thus, a man picture obtained by converting the background of the objective picture into a single color is generated. The vertex and the jaw of the man are retrieved from the person picture and a face picture is generated. Then, the center line and an inclination are detected and the position of the face is decided so that the face of the man becomes a prescribed position in the range of an outline in a prescribed size.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method, and more particularly to a method for extracting a foreground image by erasing a background, and automatically extracting an extracted image of a person's upper body so as to meet predetermined conditions. More particularly, the present invention relates to an image processing apparatus for executing such an image processing method, a recording medium storing a computer program to be executed by a computer, and a portrait photograph apparatus as a practical example.

[0002]

2. Description of the Related Art Conventionally, when taking a portrait photograph for a license, a passport, various certificates, a funeral photograph, or a portrait photograph for a criminal investigation, the background is white or gray so as to be originally used. They photographed a larger area than the required size, trimmed the face of the person to meet the required conditions, or cut the printed photo to the required size.

[0003] Such a conventional portrait photography method requires a dedicated screen and appropriate lighting to obtain a white or gray background, and furthermore, the printed photograph must meet the required conditions. It had to be trimmed or cut, which was troublesome.

On the other hand, various image processing softwares have been marketed with the recent improvement and spread of personal computers. With such commercial software,
Any area in one image, for example, the whole image of a person,
In many cases, a function is provided that can extract a face or the like by designating the face and synthesize it with another image, for example, a landscape. Therefore, it is considered easy to create such a portrait photograph for a certificate as described above by using such a function, but in reality, an arbitrary region is automatically extracted from one image. It is difficult for the user to trace the boundaries of the image with a pointing device such as a mouse, or even when fully automatic, it is necessary to give some outline instructions by the user's manual operation. There are many cases.

[0005]

As described above, conventionally, dedicated equipment (screen, lighting, etc.) is required to take a portrait photograph, and trimming is performed so as to meet the required conditions. Or the print needs to be cut, and this requires the photographing of a wider area than the required size.

In addition, when using image processing software, it is a fact that the operation of extracting a necessary area from one image has to be manually performed, and for a general personal computer user. Quite advanced technology will be required.

The present invention has been made in view of such circumstances, and a method and an apparatus for extracting a foreground image by erasing a background from one image, and an apparatus for automatically trimming an extracted image, It is another object of the present invention to provide a recording medium on which a program code for realizing them by a computer is recorded, and a portrait photographing apparatus as an apparatus integrating them.

[0008]

SUMMARY OF THE INVENTION A foreground image extraction method according to the present invention is a method for extracting an object photographed in the foreground in a single image, wherein a background image having no object is photographed. Taking a target image in which the target object is positioned in the foreground in the same frame as the background image;
Calculating a difference in brightness in a predetermined common area between the target image and the background image; obtaining a difference between the background image and the target image; and correcting the difference in brightness to generate a first mask image Performing a brightness expansion / reduction process on the first mask image, and performing a color expansion / reduction process on the first mask image that has been subjected to the brightness expansion / reduction process. Applying a second mask by determining a boundary between a target object region and a background region in the first mask image subjected to the color expansion / reduction processing, and dividing the region at the boundary. It is characterized by including a step of generating an image and a step of generating an image in which the background of the target image is converted into a single color by superimposing the second mask image on the target image and synthesizing the second mask image.

Further, an image processing apparatus according to the present invention is an apparatus for extracting an object photographed in the foreground in one image, wherein a background image photographing a state in which no object is present and a background image Bias calculating means for calculating a difference in brightness in a predetermined common region from a target image photographed with the target positioned in the foreground in the same frame, and a difference between the background image and the target image is calculated. First mask image generation means for generating a first mask image by correcting the difference;
First dilation / reduction processing means for performing dilation / reduction processing of brightness on the first mask image generated by the first mask image generation means, and a first dilation / reduction processing means processed by the first dilation / reduction processing means. Second dilation / reduction processing means for performing color dilation / reduction processing on the mask image, and a target object area and a background area in the first mask image processed by the second dilation / reduction processing means. A second mask image generating means for generating a second mask image by dividing an area at the boundary line, and a second mask image generated by the second mask image generating means for the target image. A synthesizing means for generating an image in which the background is deleted from the target image by superimposing and synthesizing the mask images.

According to the foreground image extraction method and image processing apparatus of the present invention, when an object photographed in the foreground in one image is extracted, the background image having no object is the same as the background image. By photographing the target image in which the object is positioned in the foreground in the frame, the difference in brightness in a predetermined common area between the two images is first calculated, and then the difference between the background image and the target image is calculated. Then, the difference in brightness is further corrected to generate a first mask image. Then, brightness expansion / reduction processing and color expansion / reduction processing are performed on the first mask image, and thereafter, a boundary between the object area and the background area is determined, and the boundary line is determined. The second mask image is generated by dividing the region in.
Finally, an image in which the background of the target image is converted to a single color is generated by superimposing and synthesizing the second mask image on the target image.

Further, a recording medium according to the present invention is a computer-readable recording medium recording a computer program for causing a computer to execute a process of extracting an object photographed in the foreground in one image, A computer calculates a difference in brightness in a predetermined common area between a background image photographing a state where no target object is present and a target image photographing a state where the target object is positioned in the foreground in the same frame as the background image. Computer readable program code means, computer readable program code means for causing a computer to calculate a difference in brightness in a predetermined common area between the target image and the background image,
Calculating a difference between the background image and the target image, and correcting the difference in brightness to generate a first mask image by a computer; Computer-readable program code means for causing the computer to perform brightness expansion / reduction processing, and to perform color expansion / reduction processing on the first mask image subjected to brightness expansion / reduction processing to the computer. Computer-readable program code means for performing, and a boundary line between an area of the object and an area of the background in the first mask image subjected to the color expansion / reduction processing,
Computer-readable program code means for causing a computer to generate a second mask image by dividing a region at the boundary line; and superimposing and synthesizing the second mask image on the target image, thereby setting the background of the target image. And computer readable program code means for causing a computer to generate an image converted to a single color.

Further, in the recording medium of the present invention as described above,
By causing the computer to read the recorded contents, the above-described image processing apparatus is realized.

Further, the automatic trimming device according to the present invention comprises:
A device for automatically trimming an image in which a person's upper body is photographed on a single color background so that a person's face is at a predetermined position within a range of a contour line of a predetermined size, and Means for searching for the top of the person, means for searching for the skin color in the vicinity below the searched top, means for calculating the average value of the searched skin color, and Means for generating a face image using the region as the face of a person; means for detecting the center line of the face image and its inclination; means for detecting the position of the chin in the face image; Means for deciding the position of the face so that the face of the person is at a predetermined position.

In such an automatic trimming apparatus of the present invention, an image obtained by photographing the upper body of a person on a background of a single color is set so that the face of the person is at a predetermined position within a range of a contour line of a predetermined size. When the image is automatically trimmed, the top of the person is searched for in the image, the skin color is searched for in the vicinity below the image, the average value of the searched skin color is calculated, and a predetermined range around the average value of the skin color is calculated. A face image is generated using the skin color area as the face of the person. Then, the center line of the face image and the inclination thereof are detected, and the chin position is further detected. According to the detection results, the face of the person is positioned at a predetermined position within a range of a contour line of a predetermined size. Is determined.

Further, the portrait photographing apparatus according to the present invention combines the above-described image processing apparatus and the automatic trimming apparatus to limit the object to the upper body of the person, thereby converting the background of the object image into a single color. Is generated, and a portrait photograph is automatically generated by determining the position of the face so that the face of the person is in a predetermined position within the range of the contour line of a predetermined size.

In such a portrait photographing apparatus, a portrait photograph is automatically generated from a background image photographed without a person and an image photographed in the same frame with a person in the foreground.

In the above-described foreground image extraction method, image processing apparatus, recording medium, and portrait photographing apparatus, the background image may be photographed and stored in advance. As a result, only one background image needs to be photographed first.

In the above-described foreground image extraction method, image processing apparatus, recording medium, and portrait photographing apparatus, the first mask image is generated after performing the same color system conversion processing on the background image and the target image. May be. This allows more effective processing by processing in a color system other than the color system at the time when the image was first captured.

In the above-described foreground image extraction method, image processing apparatus, recording medium, and portrait photographing apparatus, the first and second expansion / reduction processes may be repeated a plurality of times as necessary. As a result, it is possible to sufficiently obtain an image state necessary for the next processing.

In the above-described automatic trimming device and portrait photographing device, the second mask image may be subjected to a blurring process. This prevents the contour of the target object from becoming unnatural when the images are finally combined.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the drawings showing the embodiments.

First, an outline of a procedure of extracting a foreground image and a procedure of automatic trimming according to the present invention will be described with reference to a schematic diagram of an image showing the procedure shown in FIG. The outline procedure of the foreground image extraction method of the present invention is as follows. First, as shown in FIG. 1A, a background image is captured in advance. Then, as shown in FIG. 1B, the photographing is performed with the person in the foreground in the same frame as the previously photographed background image. In other words, the camera itself performs so-called fixed-point imaging in which the position and the field of view are fixed. Then, by extracting the difference from both,
As shown in FIG. 1 (c), only the image of the foreground person is automatically extracted. The image of the person, especially the face, extracted by the foreground image extraction method as described above is automatically trimmed so as to finally meet the previously required condition as shown in FIG. 1 (d). I do.

By automatically executing the procedure shown in each schematic diagram of FIG. 1, an apparatus for automatically creating a portrait photograph such as an ID photograph is realized. FIG.
(a) through automatically executing the procedure shown in FIG. 1 (c), an image in which not only a person but also a region (specific subject) desired by the user is automatically extracted from an arbitrary image. Processing software is implemented.

Hereinafter, a foreground for obtaining an image in the state shown in the schematic diagram of FIG. 1C from two images shown in the schematic diagrams of FIGS. 1A and 1B will be described. The image extraction method will be described in detail. In the following description, an image of only the background that has been photographed in advance, which is shown in the schematic diagram of FIG. 1A, is referred to as a background image, and a person is photographed in the foreground in the same frame as this background image. The image as shown in FIG. 1B is called a target image. The background image and the target image only need to be prepared in a state that can be processed as digital data.

First, the background bias is measured. This is because the brightness of the background is not exactly the same at the time when the background image is taken and the time when the target image is taken, and the background image and the target image are taken at intervals of several seconds. This is also because it is inevitable that the brightness of the background changes slightly when a person enters the foreground. Therefore, when the difference between the background image and the target image is simply obtained, the entire target image is extracted. In order to avoid such a situation, a reference area (hereinafter, referred to as a reference area) in a part of the target image where the background is always photographed
(Referred to as a reference area). If the purpose of taking the target image is a portrait photograph of a person, as shown in the schematic diagram of FIG. 2B, both sides of the upper end of the image frame, specifically, the head of the person, The background is always photographed on both sides. For this reason, a reference area is set in advance as indicated by reference numeral R in FIGS. 2 (a) and 2 (b).

Here, the reference region of the background image is RB, the reference region of the target image is RO, and the respective brightnesses are B (RB) and B (R
O), B (RB) = B (RO) + Bi where Bi is a bias value. The bias value Bi
Is the amount of change in the brightness of both images at the time of shooting, but is detected as a difference in brightness on both of the shot images.

Next, as shown in each schematic diagram of FIG. 3, a mask image (first mask image) is obtained by taking a difference between the background image and the target image. The color system is converted accordingly. When an image is processed by a computer, a film or print is captured by a scanner, directly input from a digital camera (still or movie), or recorded on a recording medium (various disk media, RAM, etc.).
An operation such as reading from a card or the like or storing in a nonvolatile storage means (a hard disk or the like) in advance is required. In any case, when an image is processed by a computer, the RGB color system is often used.However, when the image is converted into a YIQ color system, an HSI color system, or an XYZ color system, for example, Sensitivity may be improved for specific vectors such as hue and luminance. Therefore, when the difference between the background image and the target image is not so large in the RGB color system, for example, when the background image is close to the skin color, or when the clothing of the person is close to the background color, the above-mentioned non-RGB system is used. After converting to any of the color systems, take the difference between them, or convert to some color system, take the difference in each, and perform a process to comprehensively judge the results It is desirable.

The background image shown in FIG. 3A and the target image shown in FIG. 3B after being converted to the RGB color system as it is or to another color system as described above. Take the difference
Further, by correcting the above-mentioned bias value Bi, a mask image (first mask image) as shown in the schematic diagram of FIG. 3C in which only the foreground (in this case, a person) is extracted is obtained. can get. However, as shown on both sides of the part corresponding to the head of the person in FIG. 3 (c), when there is a change in the background, that part may be extracted on the mask image, In addition, there is a possibility that noise is mixed.

Next, as shown in each schematic diagram of FIG. 4, the brightness expansion processing and the drawing are performed on the mask image of FIG. 4 (a) as shown in FIG. 4 (b). As shown in Fig. 4 (c), brightness reduction processing (maximum / minimum brightness)
To smooth the portion corresponding to the person in the image. By adding such processing, it is possible to determine, for example, the hair and the like as a part of the person, and to identify even when the clothes have the same system color as the background.

Further, in the mask image obtained by repeating the expansion and contraction as described above, the entire area surrounding the part corresponding to the person is filled up, so that the noise and the change of the background which may exist in the background part are obtained. In addition to erasing the portion, it is possible to fill in a missing portion that may have occurred inside the portion corresponding to the person.

More specifically, as shown in FIG. 5 (a), since the mask image may contain noise and a change in the background, the color expansion / reduction process is repeated. By doing so, a certain degree of guarantee is given, though not completely, and the boundary between the part corresponding to the person and the background can be determined as shown in FIGS. 5 (b) and 5 (c). . Finally, by separating the area at the boundary between the background and the part corresponding to the person, a mask image (second mask image) of the person as shown in FIG. 5D is obtained.

Next, FIG. 6A obtained as described above is used.
The mask image of the person is smoothed as shown in FIG. 6B by performing processing such as Gaussian blurring.

Finally, the target image of FIG. 7A (same as the target image of FIG. 1B) and the mask image of the person of FIG. A person image with no background, more specifically, a single color background as in c) is obtained. Specifically, the mask value “0” is set as the background color and the mask value “
The two images are combined by processing 1 "as a person. If the above-described blurring processing shown in FIG. 6 is not performed, the target image shown in FIG. 7A and the target image shown in FIG. The boundary line when combining with a mask image of a person is too sharp, and an unnatural image is obtained.

As described above, the background image shown in FIG.
From the target image in (b), a person image without a background in FIG. 7 (c) is automatically obtained. However, in this state, the conditions defined in various identification photographs, specifically, the outline size of the photograph, the size of the face, the position with respect to the outline, and the like do not match, so that the schematic diagram of FIG. Needs to be trimmed to the state shown in the schematic diagram of FIG. Hereinafter, such an automatic trimming procedure will be described.

First, the top of a person is detected, and a portion of a color (skin color) that can be regarded as a face is detected around the top. Specifically, the top of the head is detected by retrieving the contour line as shown in FIG. 9 (b) of the person image with the single color background of FIG. By searching in a relatively wide range from red to yellow, a skin color is searched as shown in FIG. 9 (c).

Since the portion corresponding to the skin color can be regarded as a face portion, an average value thereof is obtained, and a portion in a certain color range around the obtained average value is regarded as a face to create a mask. . Specifically, the average of the skin color is obtained from the image in which the portion corresponding to the skin color as shown in FIG. 10 (a) is acquired, and the dark portion is determined in consideration of searching the position of the chin. Narrows the search range (decreases the threshold) and widens the search range (increases the threshold) for bright parts. As a result, an image of only the face as shown in FIG. 10B (hereinafter referred to as a face mask image) is obtained.

Next, the center of gravity of the face is determined from the face mask image, and an image of only a closed region including the center of gravity is created. Specifically, an image of only the closed area as shown in FIG. 11 (b) is created from the face mask image of FIG. 11 (a), and as shown in FIG. 11 (c), X 11 (d), the center of gravity of the face portion in the X-axis direction at each Y coordinate is obtained, and the face is determined from the array of the obtained centers of gravity, as shown in FIG. The X coordinate value and the inclination of the center of are calculated. Note that, as shown in FIG. 11 (c), the lowermost part of the closed area is detected as a position of the chin.

As described above, the size of the face in the vertical direction (the top is set as the upper limit position and the chin is set as the lower limit position), the central axis and the inclination are determined as shown in FIG. 12 (b). The prescribed outline dimensions of the photograph as shown in (a), for example, 3.5 cm x 4.
Trimming is performed by determining the enlargement / reduction of the person image and the arrangement with respect to the outline in accordance with the position of 5 cm, the top and the chin. As a result, an identification photograph in which the face is arranged at a predetermined position within a predetermined outline size as shown in FIG. 12 (c) is generated.

In this manner, by photographing the background image as shown in FIG. 1A in advance, a fixed point photograph as shown in FIG. 1B with the same background is taken. A portrait image of a person with no background as shown in FIG. 7 (c) is automatically obtained from the image of the person, and a proof that satisfies predetermined conditions as shown in FIG. 12 (c). A photo is obtained.

Next, an image processing apparatus for carrying out the method of the present invention as described above will be described. FIG. 13 is a functional block diagram showing a configuration example of an image processing device for implementing the foreground image extraction method.

The camera 11 is used for photographing a background image and a target image. The photographed background image as shown in FIG. 1A is stored in the background image storage unit 12 and in FIG. 1B. The target images as shown are stored in the target image storage unit 13, respectively. Note that the images (background image and target image) captured by the camera 11 are usually of the RGB color system.

It is needless to say that one storage device may be used as the hardware for both storage units 12 and 13. Further, the background image needs to be photographed in advance and stored in the background image storage unit 12, but the target image may be directly captured from the camera 11 each time.

The data of both images is input to the bias calculator 14, and as shown in FIGS. 2A and 2B, the brightness B of the reference region RB of the background image and the reference region RO of the target image. (RB),
A bias value Bi is calculated from B (RO). The bias value Bi calculated by the bias calculator 14 is provided to a mask image generator 16 described later.

On the other hand, the data of both images is also input to the color system conversion unit 15. The color system conversion unit 15 performs the same color system conversion processing on both the background image and the target image. For example,
As described above, the image captured by the camera 11 is generally of the RGB color system, but the color system conversion unit 15 converts the background image and the target image into other images such as the YIQ color system and the HSI color system. System, XYZ color system, etc. As a result, the hue at the time of the difference extraction processing performed by the mask image generation unit 16 next,
Improve sensitivity to specific vectors such as luminance.

The mask image generation unit 16 extracts the difference between the background image converted from the RGB color system to any other color system by the above-described color system conversion unit 15 and the target image. By correcting the bias value Bi calculated by the bias calculator 14, the mask image as shown in FIG. 3C is generated.

In the RGB color system, when the difference between the background image and the target image is not so large, for example, when the background image is close to the skin color, or when the clothes of the person are close to the background color, the RGB color system is used. After converting both the background image and the target image into one of the above-described color systems other than the color system by the color system conversion unit 15, take the difference between them, or convert them to some color systems and It is desirable to perform a process of taking a difference between them and judging the results comprehensively. For this reason, the color system conversion unit 15 described above is provided. However, the color system (in general, RGB) at the time when the camera 11 first shoots the image without performing the color system conversion process is provided. Needless to say, the processing by the mask image generation unit 16 may be performed as it is in the color system).

FIG. 3 generated by the mask image generation unit 16
The mask image as shown in (c) is then provided to the dilation / reduction processing unit 17 to perform the dilation processing of the brightness as shown in FIGS. 4 (a), (b) and (c). By performing a reduction process (maximum / minimum brightness), a portion corresponding to a person in the image is smoothed. As a result, a mask image as shown in FIG. 5A is generated.

Further, the mask image subjected to the dilation / reduction processing by the dilation / reduction processing unit 17 is given to a person mask image generation unit 18 and shown in FIGS. 5 (a), 5 (b) and 5 (c). As shown in FIG. 5 (d), a boundary line between a person portion and a background portion in an image is detected and subjected to region division by performing the color expansion and reduction processes as described above. A unique person mask image is generated.

The person mask image generated by the person mask image generation unit 18 is then given to a blur processing unit 19, for example, subjected to processing such as Gaussian blurring, and smoothed as shown in FIG. 6 (b). A person mask image is obtained as if it were.

The human mask image obtained by the blur processing unit 19 is then provided to the synthesizing unit 20. FIG.
The data of the target image shown in (b) is also stored in the target image storage unit.
The target image and the mask image of the person generated by the blur processing unit 19 shown in FIG. 6B are combined, for example, α combined. As a result, a non-background (single-color background) human image as shown in FIG.

Specifically, the processing in the synthesizing unit 20 synthesizes both images by processing the mask value “0” as the background color and the mask value “1” as the person. If the blur processing by the above-described blur processing unit 19 is not performed, the boundary line at the time of combining the target image and the mask image of the person becomes too sharp, resulting in a slightly unnatural image. However, since this is not an essential problem, it goes without saying that the blur processing by the blur processing unit 19 may be omitted.

As described above, the background image as shown in FIG. 1 (a) and the background image as shown in FIG. From the target image, a person image without a background as shown in FIG. 7C is automatically obtained.

Next, the automatic trimming device of the present invention will be described with reference to a functional block diagram of FIG.

The person image storage unit 31 stores the backgroundless person image generated by the combining unit 20 described above, and the photograph condition storage unit.
32 is the condition of the photo to be generated (for example, in the case of a passport ID photo, the outline size is 3.5 cm × 4.5 cm,
7 ± 2mm from top to top, 11 from bottom to chin
± 2 mm) is stored in advance.

The data of the person image stored in the person image storage unit 31 is given to the top search unit 33, and the top is searched. More specifically, the processing in the crown search unit 33 scans in the X direction downward from the upper end of the person image, and calculates the Y coordinate value in which a color different from the color of the single color background synthesized by the synthesis unit 20 exists. It is done by asking.

When the top of the head is found in this way, the skin color search unit 34 next sets the center of the top of the head, for example, in HIS.
In the color system, a color that can be regarded as a face is searched by performing a search in a relatively wide range from flesh color, more specifically, from red to yellow.

In the next facial image generation unit 35, a skin color search unit 34
Since the skin color part searched by the above can be regarded as a face part, an average value is obtained, and a part of a certain color range centered on the obtained average value is regarded as a face to create a mask.

More specifically, the face image generation unit 35 finds the average of the skin colors searched by the skin color search unit 34, and narrows the search range for dark parts in consideration of searching for the position of the chin ( The threshold is set small), and the search range is widened (the threshold is set large) for a bright part. As a result, an image of only the face as shown in FIG. 10B (hereinafter referred to as a face mask image) is obtained.

Next, the face center / inclination detecting section 36 obtains the center of gravity of the face from the face mask image generated by the face image generating section 35, and creates an image of only a closed region including the center of gravity. Specifically, the face center / inclination detection unit 36 converts the face mask image as shown in FIG. 11A generated by the face image generation unit 35 as shown in FIG. 11B. An image of only the closed area is created, and as shown in FIG.
11 (d), the center of gravity of the face portion in the X-axis direction at each Y coordinate is obtained, and the face is determined from the array of the obtained centers of gravity, as shown in FIG. The X coordinate value and the inclination of the center of are calculated.

Next, as shown in FIG. 11 (c), the jaw position detector 37 detects the lowermost portion of the closed area as the position of the jaw.

As described above, the size of the face in the vertical direction (the top of the head is set as the upper limit position and the chin is set as the lower limit position), the center axis and the inclination are determined, and these data are given to the portrait photograph generation unit 38. In addition, the portrait photograph generation unit 38 is provided with data of a person image from the person image storage unit 31 and the photograph condition from the photograph condition storage unit 32. The portrait photograph generation unit 38 enlarges or reduces the portrait image data provided from the portrait image storage unit 31 so that the top and the chin are at predetermined positions with respect to the outline of the photograph. The position of the outline is determined so that the face is located at the center of the photograph, and the inclination is corrected.

As described above, a certification photograph that meets predetermined conditions as shown in FIG. 12C is generated from a portrait image of a person with no background as shown in FIG. 8A. .

The image processing apparatus having the configuration shown in FIG. 14 includes a person image storage section 31 for storing data of a person image to be processed, but is shown in FIG. The data of the person image output from the synthesizing unit 20 may be directly input to the crown retrieval unit 33. In this case, a portrait photographing apparatus is realized in which an image processing apparatus for performing the foreground image extracting method shown in FIG. 13 and the automatic trimming apparatus shown in FIG. 14 are integrated.

The image processing apparatuses shown in FIGS. 13 and 14 each include storage units MU1 and MU2 and arithmetic units PU1 and PU2. These units are configured as dedicated hardware. Of course, it is possible to use a general-purpose arithmetic device, that is, a personal computer, as the arithmetic devices PU1 and PU2, and to realize each function by software. In that case, the two devices as described above are read by one personal computer, and more specifically, one computer program recorded on a recording medium (flexible disk, CD-ROM, etc.) is read by the personal computer ( Installation).

FIG. 15 shows such a personal computer.
FIG. 50 is a schematic view showing an appearance of 50. The personal computer 50 has a flexible disk drive for reading the recorded content (program code) from a flexible disk FD or a CD-ROM CD on which a computer program PG for performing image processing as described above is recorded.
A (FDD) 61 and a CD-ROM drive (CDD) 62 are provided. Reference numeral 63 denotes a hard disk as a non-volatile storage means, and the above-mentioned flexible disk
FD or CD-ROM Stores each program code of the computer program PG read from the CD.

Reference numeral 55 denotes a CRT as a display means.
Reference numerals 52 and 53 indicate a display and a keyboard and a mouse, respectively, as input means and a pointing device. However, flexible disk FD, CD-R
It goes without saying that various recording media other than the OM CD may be used.

FIG. 16 is a block diagram showing an example of the internal configuration of the personal computer 50. The above-mentioned FDD 61, CDD 62, HDD 63, CRT display 55, etc. are connected by a bus 60 with a CPU 51 at the center. In addition, camera 11, keyboard
52 and a mouse 53 are connected to a bus 60 via an interface (I / O) 54.
It is connected to the. Reference numerals 56 and 57 indicate RAM and ROM as models, respectively.

The program code of the computer program PG read from the flexible disk FD by the flexible disk drive 61 or from the CD-ROM CD by the CD-ROM drive 62 is temporarily stored in, for example, the hard disk 63. When it is executed, it is transferred to the RAM 56 and stored.

Here, the recording medium is a flexible disk FD and / or a CD-ROM CD. However, the recording medium is not limited to these, and a magnetic tape, an optical disk, Of course, use of a magnetic disk or the like is also possible.

FIG. 17 shows a computer program recorded on a flexible disk FD as an example of a recording medium.
It is a schematic diagram which shows the content of PG.

The flexible disk shown in FIG.
In the FD, a program code (PC11) for calculating the bias value Bi between the background image and the target image, a program code (PC12) for performing the same color system conversion processing on the background image and the target image, and after the color system conversion The difference between the background image and the target image is obtained, the bias value Bi is corrected to generate a mask image, a program code (PC13), a program code for performing a brightness expansion / reduction process on the mask image (PC14), A program code (PC15) for generating a person mask image by dividing an area by a boundary line between a person region and a background region, a program code (PC16) for blurring a person mask image, and a person mask image and a target image. A program code (PC17) for generating a person image by combining, a program code (PC21) for searching for the top of a person from a person image, a program code (PC22) for searching for a flesh color near the lower part of the searched head, Was A program code for calculating an average value of color (PC23), a program code for generating a face image using a predetermined range of skin color area around the average value of skin color as a person's face (PC24), a center line of the face image, Program code (PC2
5), Program code for detecting chin position from facial image
(PC26), a program code (PC27) for generating a portrait photograph according to the photograph condition is recorded.

When a recording medium on which such a program code is recorded, for example, a CD-ROM CD, is inserted into the CDD 62, each program code recorded on the CDD 62 is once stored in an HD.
Stored in D63. Then, each program code is read out from the HDD 63 as necessary and stored in the RAM 56, and each is sequentially executed by the CPU 51. The operation of the personal computer 50 in this case, in other words, the operation of the portrait photographing apparatus of the present invention is as shown in the flowchart of FIG.

However, it is assumed that the background image is photographed in advance by the camera 11 and stored in the HDD 63, and the target image is photographed by the camera 11 and stored in the HDD 63 in advance, or has not been photographed yet. . Further, it is assumed that various ID photos, for example, photo conditions corresponding to passports, licenses, and the like are stored in the HDD 63 in advance.

When the target image is already stored in the HDD 63, the operator operates the keyboard 52 or the mouse 53 to specify which of the several photos stored in the HDD 63 is to be processed. On the other hand, if the photograph of the target image has not been taken yet, the operator operates the camera 11 to photograph the target image with the person positioned at an appropriate position. In this case, the photograph data may be temporarily transferred from the camera 11 to the HDD 63 of the personal computer 50 and stored, or may be read directly from the camera 11 and processed.

Next, when the operator gives an instruction to start processing by operating the keyboard 52 or the mouse 53, the CPU 51
First, load the background image from HDD 63,
Read directly from D63 or camera 11 (Step S1
0). Then, the CPU 51 calculates the bias value Bi of both images.
(Step S11). Next, the CPU 51 performs the same color system conversion processing on the background image and the target image (step S12), obtains the difference between the background image after the color system conversion and the target image, and further corrects the bias value Bi. To generate a mask image
(Step S13). Next, the CPU 51 performs brightness expansion / reduction processing on the mask image (step S14), and further performs color expansion / reduction processing (step S15). These expansions
The reduction process is repeated a plurality of times as necessary.

Then, the CPU 51 determines a boundary line between the person region and the background region from the mask image obtained by repeating the expansion and contraction of the brightness a suitable number of times, and divides the region at the boundary line to divide the person mask image. (Step S1
6). This person mask image is then processed by Gaussian
(Step S17) is performed and is appropriately blurred, but may be omitted in some cases. Next, the CPU 51 reads out the target image from the HDD 63 again, and superimposes the target image on the target image after the above-described blurring process is performed, thereby combining the target image with no background (white or gray background). A person image is generated (step S18).

Next, the CPU 51 searches the top of the person from the person image generated as described above (step S19), and searches for a flesh color near the lower part of the searched top (step S2).
0), calculate the average value of the searched skin color (step S2
1). Then, the CPU 51 generates a face image using a predetermined range of the skin color area centered on the average value of the skin color as the face of the person.
(Step S22), the center line of the face image and its inclination are detected (Step S23), and the chin position is further detected from the face image (Step S24). Finally, the CPU 51 reads out the photograph conditions preset by the operator from the HDD 63 (step S25), and generates a portrait photograph according to the photograph conditions.
(Step S26).

As described above, the photograph designated by the operator,
For example, a photo that meets the conditions for a passport ID photo is automatically generated.

[0079]

As described above in detail, according to the foreground image extraction method and the image processing apparatus of the present invention, a person or an object photographed in the foreground from a background image can be extracted from a background (single color background). It can be easily extracted as an image.

According to the automatic trimming device of the present invention, the face of an image of a person having no background (single color background) is automatically trimmed so that the face is located at a predetermined position within a predetermined outline size. Becomes possible.

According to the recording medium of the present invention, the above-described apparatus can be easily realized by causing a personal computer to read the program code as the recorded content.

Further, according to the portrait photographing apparatus of the present invention, by taking a background image in advance, a person with no background (single color background) can be prepared without preparing a white or gray screen and a special certificate. An image of the upper body is generated, and a photograph in which the face is arranged at a predetermined position within a predetermined outline dimension, that is, an identification photograph can be easily created.

[Brief description of the drawings]

FIG. 1 is a schematic diagram for explaining the outline of the procedure of foreground image extraction and automatic trimming according to the present invention.

FIG. 2 is a schematic diagram illustrating a procedure of a foreground image extraction method according to the present invention.

FIG. 3 is a schematic diagram for explaining a procedure of a foreground image extraction method according to the present invention.

FIG. 4 is a schematic diagram for explaining a procedure of a foreground image extraction method according to the present invention.

FIG. 5 is a schematic diagram illustrating a procedure of a foreground image extraction method according to the present invention.

FIG. 6 is a schematic diagram for explaining a procedure of a foreground image extraction method according to the present invention.

FIG. 7 is a schematic diagram for explaining a procedure of a foreground image extraction method according to the present invention.

FIG. 8 is a schematic diagram for explaining a procedure of automatic trimming according to the present invention.

FIG. 9 is a schematic diagram for explaining an automatic trimming procedure according to the present invention.

FIG. 10 is a schematic diagram for explaining the procedure of automatic trimming according to the present invention.

FIG. 11 is a schematic diagram for explaining the procedure of automatic trimming according to the present invention.

FIG. 12 is a schematic diagram for explaining a procedure of automatic trimming according to the present invention.

FIG. 13 is a functional block diagram illustrating a configuration example of an image processing apparatus according to the present invention.

FIG. 14 is a functional block diagram illustrating a configuration example of an automatic trimming device according to the present invention.

FIG. 15 is a schematic diagram showing the appearance of a personal computer on which the recording medium of the present invention is installed.

FIG. 16 is a block diagram showing an example of the internal configuration of a personal computer on which the recording medium of the present invention is installed.

FIG. 17 is a schematic diagram showing the contents of a computer program recorded on a recording medium of the present invention.

FIG. 18 is a flowchart showing an operation procedure of a personal computer as a portrait photographing device of the present invention.

[Explanation of symbols]

 FD Flexible disk CD CD-ROM PG program PC11 to PC17, PC21 to PC27 Program code 11 Camera 12 Background image storage unit 13 Target image storage unit 14 Bias calculation unit 15 Color system conversion unit 16 Mask image generation unit 17 Expansion / reduction processing Unit 18 Person mask image generation unit 19 Blurring processing unit 20 Compositing unit 31 Person image storage unit 32 Photo condition storage unit 33 Head search unit 34 Skin color search unit 35 Face image generation unit 36 Face center / tilt detection unit 37 Chin position detection unit 38 Portrait photo generator 50 Personal computer 51 CPU 56 RAM 61 Flexible disk drive 62 CD-ROM drive 63 Hard disk drive

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06F 15/66 450 F-term (Reference) 5B050 AA09 BA12 DA02 DA04 EA06 EA12 EA15 EA19 FA02 FA05 5B057 AA20 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CC03 CE08 CE09 DA08 DB02 DB06 DB09 DC16 DC25 5C076 AA13 AA21 AA22 AA36 5C082 AA27 AA32 BA20 BB15 CA54 CA55 CA59 CB01 CB06 DA53 DA87 MM05

Claims (8)

[Claims] 1. A foreground image extraction method for extracting an object photographed in the foreground in one image, comprising: photographing a background image in which no object is present; Photographing a target image in which a target object is located; calculating a brightness difference in a predetermined common area between the target image and the background image; and calculating a difference between the background image and the target image. Calculating a first mask image by correcting the difference in brightness; performing a brightness expansion / reduction process on the first mask image; Performing a color expansion / reduction process on the first mask image that has been subjected to the reduction process; and a region of the object and a background region in the first mask image that has been subjected to the color expansion / reduction process. To determine the boundary line,
Generating a second mask image by dividing an area at the boundary line; and superimposing and synthesizing the second mask image on the target image to convert the background of the target image into a single color. Generating an image. 2. An image processing apparatus for extracting an object photographed in the foreground in one image, comprising: a background image photographing a state in which no object is present; and an object in the foreground in the same frame as the background image. Bias calculating means for calculating a difference in brightness in a predetermined common area from a target image photographed by positioning the image, calculating a difference between the background image and the target image, and correcting the difference in brightness. A first mask image generating means for generating a first mask image by using the first mask image generating means, and a first dilation / reduction processing for performing brightness dilation / reduction processing on the first mask image generated by the first mask image generation means.
Reduction processing means; second expansion / reduction processing means for performing color expansion / reduction processing on the first mask image processed by the first expansion / reduction processing means; A second mask for generating a second mask image by determining a boundary between the object area and the background area in the first mask image processed by the reduction processing means, and dividing the area at the boundary; An image generating unit, by superimposing and synthesizing the second mask image generated by the second mask image generating unit on the target image,
An image processing apparatus comprising: a synthesis unit configured to generate an image in which a background is deleted from the target image. 3. A color system conversion unit for performing the same color system conversion processing on the background image and the target image, wherein the first mask image generation unit is provided with a color system based on the color system conversion unit. 3. The image processing apparatus according to claim 2, wherein a first mask image is generated by using the background image and the target image that have been converted as the processing targets. 4. 4. The image processing apparatus according to claim 2, wherein said first and second expansion / reduction processing means repeat the expansion / reduction processing a plurality of times as necessary. 5. The image processing apparatus according to claim 2, further comprising: a blur processing means for performing a blur processing on the second mask image generated by the second mask image generating means. 6. An automatic trimming device for automatically trimming an image obtained by photographing the upper body of a person on a background of a single color so that the face of the person is at a predetermined position within a range of a contour line having a predetermined size. Means for searching for the top of a person from an image, means for searching for a flesh color in the vicinity below the searched top, means for calculating an average value for the searched skin color, and centering on the average value for the skin color Means for generating a face image using a predetermined range of skin color area as the face of a person; means for detecting a center line of the face image and its inclination; means for detecting a chin position in the face image; Means for determining the position of the face so that the face of the person is at a predetermined position within the range of the outline. 7. A computer-readable recording medium on which a computer program for causing a computer to execute a process of extracting an object photographed in the foreground in one image is recorded, wherein a state in which the object does not exist is recorded. Computer-readable program code for causing a computer to calculate a difference in brightness in a predetermined common region between a captured background image and a target image obtained by capturing a state where the target is positioned in the foreground in the same frame as the background image. Means, a computer-readable program code means for causing a computer to calculate a difference in brightness in a predetermined common area between the target image and the background image, and obtaining a difference between the background image and the target image, Computer that causes a computer to generate a first mask image by correcting a difference in brightness Computer readable program code means readable; computer readable program code means for causing a computer to perform brightness expansion / reduction processing on the first mask image; and brightness expansion / reduction processing. Computer-readable program code means for causing a computer to perform a color expansion / reduction process on the first mask image on which the color expansion / reduction process has been performed. Determine the boundary between the object area and the background area,
Computer readable program code means for causing a computer to generate a second mask image by dividing an area at the boundary line; and superimposing the second mask image on the target image to synthesize the target image. And a computer-readable program code means for causing a computer to generate an image having the background converted to a single color. 8. A portrait photographing apparatus for generating a portrait photograph in which the upper body of a person is arranged on a single color background such that the face of the person is located at a predetermined position within a contour line of a predetermined size, A camera that captures a background image in which no person is present and a target image in which the upper body of the person is positioned in the foreground in the same frame as the background image; andan image storage unit that stores the background image captured in advance by the camera. Bias calculating means for calculating a brightness difference in a predetermined common area between the background image stored in the image storing means and the target image captured by the camera; and a difference between the background image and the target image. And a first mask image generating means for generating a first mask image by correcting the difference in brightness, and a first mask image generated by the first mask image generating means The first expansion and subjected to expansion and contraction processing of brightness for
Reduction processing means; second expansion / reduction processing means for performing color expansion / reduction processing on the first mask image processed by the first expansion / reduction processing means; A second mask for generating a second mask image by determining a boundary between the object area and the background area in the first mask image processed by the reduction processing means, and dividing the area at the boundary; An image generating unit, by superimposing and synthesizing the second mask image generated by the second mask image generating unit on the target image,
Synthesizing means for generating a person image in which the background of the target image is converted to a single color; means for searching for the top of a person from the person image; means for searching for a flesh color near the lower part of the searched top; Means for calculating the average value of the extracted skin color, means for generating a face image using a predetermined range of the skin color area centered on the average value of the skin color as the face of the person, and detecting the center line of the face image and its inclination Means for detecting the position of the chin in the face image; and means for determining the position of the face so that the face of the person is at a predetermined position within the range of the contour line of the predetermined size. A portrait photographing device characterized by the above-mentioned.