JPH07274067A - Human body picture background synthesis device - Google Patents

Abstract

PURPOSE:To provide a human body picture background synthesis device capable of performing background synthesis using the human body picture of an optional background, eliminating a patchy unnatural feeling and performing the natural background synthesis. CONSTITUTION:A scanner 1 inputs the human body pictures and background pictures to be synthesized, a contour extraction part 12 extracts the contour information of a human body from the inputted human body pictures and an area division part 13 divides the inputted human body pictures into three areas of the human body, the background and a boundary based on the extracted contour information of the human body. A background synthesis part 14, by successively scanning pictures obtained as the result of dividing the areas from an upper left end, outputs the inputted human body pictures when the pertinent picture element is in a human body area, outputs the inputted background pictures in the case of a background area, synthesizes the inputted human body pictures and background pictures based on the density values of the respective picture elements inside the boundary area in the case of the boundary area and outputs the synthesized result. A controller 6 outputs the synthesized result for which the output results of the three areas of the human body, the background and the boundary obtained by the background synthesis are added in a printer 4.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates, for example, to a person image background composition in which a person image and a background photographed at different places are combined by image processing and the person image is displayed as if it was photographed in front of the background. Regarding the device.

[0002]

2. Description of the Related Art A background keying technique for synthesizing an object and a background photographed at different places by image processing and displaying the object as if it was photographed in front of the background is generally called a chroma key technique. , Is used as a special technique in television and movies.

When using the chroma key technology, one TV camera shoots a subject in front of a uniform blue background called a chroma key back, and another TV camera shoots a background scene. Then, the background and the subject are synthesized by outputting the background to be synthesized in the blue portion of the image in which the subject is captured and outputting the image of the subject to be fitted into the background in the other portions.

[0004]

In the case of general chroma key technology, in order to remove the background and identify the human area, the background color to be photographed is uniform over the entire image, and the illumination is lit. It is necessary for the background to have uniform color. Therefore, under the present circumstances, it is difficult to deal with images taken under general conditions, because it is required to take images under a specific background and favorable conditions.

Further, in order to remove the background from the image photographed in front of the chroma keyback, an area having the same component as the background color component is extracted, the image is divided into a background and a subject, and a new background area is created. Synthesis is performed by embedding the background. This cut-and-paste process is effective for objects that have many straight line contours or whose contours are clear, but include many areas where the contours are not clear (such as the boundary between the hair and the background) like humans. In the case of an object, there is a drawback in that those parts result in an unnatural composite result as if they were cut out with scissors.

Therefore, an object of the present invention is to provide a person image background synthesizing apparatus which can perform background synthesizing using a person image of an arbitrary background, and can eliminate natural feeling of cut and paste and perform natural background synthesizing. And

[0007]

A person image background synthesizing apparatus according to the present invention includes image input means for inputting a person image and a background image to be synthesized, and person outline information from the person image input by the image input means. Based on the contour information of the person extracted by the contour extracting means and the contour information of the person extracted by the contour extracting means.
When the corresponding pixel is a human region, the input human image is output by sequentially scanning the image obtained as a result of the region division by the region dividing unit and the region dividing unit, and the background region In the case of, the input background image is output, and in the case of a boundary area, the input person image and the background image are combined based on the gray value of each pixel in the boundary area, and the combined result is output. Synthesizing means,
The output means outputs the combined result obtained by adding the output results of the three areas of the person, the background, and the boundary obtained by the combining means.

The person image background synthesizing apparatus of the present invention is
Image input means for inputting a person image and a background image to be combined, contour extraction means for extracting contour information of a person from the person image input by the image input means, and contour of the person extracted by the contour extraction means Based on the information, by dividing the input person image into three areas of a person, a background and a boundary, and an image obtained as a result of the area division by the area dividing means, sequentially scanning from one end,
If the corresponding pixel is a person area, the input person image is output, if it is a background area, the input background image is output, and in the case of a boundary area, the distance of each pixel in the boundary area from the background Based on the above, the input person image and the background image are combined, and the combining result for outputting the combined result and the combined result obtained by adding the output results of the three areas of the person, the background and the boundary obtained by this combining means are obtained. And output means for outputting.

Further, the person image background synthesizing apparatus of the present invention includes image input means for inputting a person image and a background image to be synthesized, and a contour for extracting contour information of a person from the person image input by the image input means. Based on the extraction means and the contour information of the person extracted by the contour extraction means,
A region dividing unit that divides the input person image into three regions of a person, a background, and a boundary, and an image obtained as a result of the region division by the region dividing unit are sequentially scanned from one end, so that the corresponding pixel is a person region. In the case of a background area, the input background image is output, and in the case of a boundary area, the input person image is output based on the gray value of each pixel in the boundary area. A first combining process of combining the image and the background image and outputting the combined result is performed. Further, in the case of a boundary area, the input is performed based on the distance from the background of each pixel in the boundary area. A synthesizing unit for synthesizing the person image and the processing result of the first synthesizing process and performing a second synthesizing process for outputting the synthesizing result, and a person, background, and boundary 3 obtained by this synthesizing unit.
And an output unit that outputs a combined result obtained by adding the output results of the regions.

[0010]

According to the present invention, it is possible to process a color or monochrome still image of a person photographed in a general situation, and it is possible to perform background composition without causing a sense of discomfort at the boundary portion due to cut-and-paste threshold processing. . Moreover,
In image processing, a person image is divided into three areas of a person, a background, and a boundary, and output values are determined by different processing methods in each area. Especially, in the boundary part, since the process of keeping the relationship of the density value between the person and the background in the input image as much as possible is performed by using the grayscale information and the distance information, there is no sense of cut and paste and the natural background composition result is output. can do.

[0011]

[Embodiment] In this embodiment, as an example, background composition of an ID photograph will be described. Currently, many cards such as cash cards and credit cards are issued, and these cards have registration numbers, names,
Although character information such as a personal identification number is recorded,
Many cards with facial photographs are also used to make it easier to identify individuals.

As means for photographing these ID photographs, such as photographing in front of a specific background, using an automatic photographer for ID photographs, or photographing in front of a white wall at home, Various cases are possible. And it is expected that portraits taken in such different situations will have various backgrounds.

As described above, if various backgrounds exist for personal identification cards for the same purpose, not only there is a possibility of forgery, but even forgery cards are difficult to distinguish. Therefore, it is considered that forgery can be prevented if there is a device that can perform the process of replacing the background with the same one that is predetermined.

In this embodiment, a case will be described in which a scanner reads a background to be combined with a black-and-white ID photo and a composite image is printed out by a printer. An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 schematically shows the configuration of a person image background synthesizing apparatus according to this embodiment. In the figure, 1
Is a color scanner as an image input means, photoelectrically converts a photograph of a person by optical scanning, and converts it into R (red),
Converted to G (green), B (blue) digital signals, R,
It is output as G and B image data. The scanner 1
It is also possible to input a monochrome image by switching a switch (not shown). The R, G, B image data (or monochrome image data) captured by the scanner 1 is temporarily stored in the frame memory 2.

The D / A converter 3 converts the R, G, B image data from the frame memory 2 or the image data from the image memory 11 described later into an analog signal, and the color printer 4 and the color monitor display. Send to 5. The printer 4 prints out the combined image, and the monitor display 5 displays the image being processed and the output image.

These scanner 1, frame memory 2 and D
/ A converter 3, printer 4, and display 5
Are connected to a controller 6 that controls the entire apparatus.

The controller 6 includes a mouse 7 used by the operator for operation instruction and image area selection, a tablet 8 used by the operator for image area instruction and selection, a keyboard 9 used by the operator for operation instruction and numerical input, and calculation. Time data and mouse 7, tablet 8, keyboard 9
A memory 10 for temporarily storing input information from the input device, an image memory 11 for temporarily storing image data such as an input image, an image being processed, and an output image, a contour extracting unit 12 for extracting contour information from the input image, A region dividing unit 13 that divides an image into regions by using the output of the contour extracting unit 12, an input image, a region division result image, a background synthesizing unit 14 that synthesizes a background using a synthetic background image, and an input color image. Is connected to a color / monochrome conversion unit 15 for converting the image into a monochrome grayscale image.

Next, the operation of the above configuration will be described with reference to the flow charts shown in FIGS. It should be noted that this operation example is a case where a monochrome photograph is used as the input image, and therefore the scanner 1 inputs a monochrome image.

The flow chart of FIG. 2 shows the flow of the entire processing, and the flow chart of FIG. 3 shows the flow of the combining processing.
The flowchart of FIG. 4 shows the flow of the background synthesizing process based on the gray value during the synthesizing process, and the flowchart of FIG. 5 shows the flow of the background synthesizing process based on the distance from the background during the synthesizing process. In each figure, A is an input image, B is a background image to be combined, C is an image after contour extraction, D is an image after region division, E is an image after combining processing based on gray values, and F is from the background. Image after composite processing (output image)
Indicates.

First, in step S1, an input process for capturing an input image and a background image is performed. That is, the scanner 1 is used to scan and capture a photograph of an input person image and a photograph of a background image to be combined, and the respective image data are captured as a person image A (x, y) and a background image B (x, y) to be combined. y) and stores it in the image memory 11.

Next, in step S2, a person contour information extraction process is performed. That is, in order to specify the person portion of the input image, the operator inputs an instruction for specifying the contour information of the person using the mouse 7 or the tablet 8. When specified in this way, the contour extraction unit 1
The image memory 1 stores the contour information of the designated person.
It is extracted from the input image in 1 and stored in the image memory 11 as C (x, y). In this case, the contour information is the peripheral part of the boundary between the person and the background, and it is not necessary to specify the fine part, and both the person part and the background part are included in a somewhat thick and smooth band. Just do it. The extracted contour information C (x, y) is as shown in FIG. 6, for example.

Next, in step S3, area division processing is performed. That is, the region dividing unit 13 roughly divides the image into three regions of a person, a background, and a boundary based on the contour information of the person extracted by the contour extracting unit 12, and then divides the image D (x, y) into three regions. Are stored in the image memory 11. Here, the person region is a portion inside the contour of the person that does not include any background, the background region is a portion outside the contour that does not include a person region, and the boundary region is a region around the contour that also includes the background. Refers to the part that includes both persons (see FIG. 7). The operator designates each area with the mouse 7 or the like, as when the contour is designated.

Next, in step S4, a combining process is performed. This combining process is performed according to the flowchart shown in FIG. That is, the background synthesis unit 14
The result image D (x, y) divided into three areas, that is, the person area, the background area, and the boundary area by the area dividing unit 13, is located at the upper left corner (x
= 0, y = 0), and if the corresponding pixel is a human region, the value of the human image A (x, y) is output without conversion, and if it is a background region, it is combined. Background image B
A process of outputting the value of (x, y) is performed, and in the case of a boundary region in which the background and the person are mixed, the background synthesizing process based on the gray value and the background synthesizing process based on the distance from the background are sequentially performed. .

Then, the same processing as described above is performed for all the pixels in the image. Here, an outline of the background synthesizing process based on the gray value and the background synthesizing process based on the distance from the background, which is performed in the boundary region, will be described.

First, the background synthesizing process based on the gray value of the pixels in the boundary area is performed according to the flowchart shown in FIG. That is, the image D (x,
Starting from the top row (y = 0) of y), the following processing is performed in each boundary area of each row. As shown in FIG.
For example, in the column of y = y1, first, scanning is performed from the left end (x = 0, y = y1) of the image D (x, y) after the region division until it reaches a boundary pixel, and a boundary region continuous with the boundary pixel. X-coordinates of the left and right edge pixels of
It is stored in the memory 10 as 1, x2. If neither the left or right end point is in contact with the background, the boundary area outputs the value of the input image without conversion, and the search for the next boundary area is started.

For example, when the left or right end point is in contact with the background area as in the column L1 in FIG. 8, the value of the pixel in the input image is stored in the memory 10 as the input background value bi. When both end points are in contact with the background as in the column L2 of FIG. 8, the value in the input image of either end point is selected and set as the input background value bi.

The pixel value in the composite background image of each pixel in the boundary is set as the output background value bo. Here, assuming that the background image to be synthesized is a plain background of uniform gray value, the output background value is constant for every pixel in the image.

Based on the input background value bi and the output background value bo, a gray value conversion table as shown in FIG. 10 is created. The point A in FIG. 10 is set so that the output background value bo is output if the input value is equal to the input background value bi, and the origins O and A, A and the maximum point B (input value = output value = maximum The light and shade values = 255) are connected by straight lines.

Using the conversion table created in this way, the value A (x, y) in each input image is obtained at each pixel in the boundary area from x = x1 to x = x2.
When the input is, the output value is obtained and is set as the output value E (x, y) of the background synthesizing process based on the gray value.

After the conversion to the end x = x2 of the boundary is performed in this way, the next boundary area is scanned and the same processing as described above is repeated. This operation is performed for all the front rows in the image.

In the above description, the background image to be combined is an image of uniform gray value, but if it is not uniform, it is necessary to obtain the output background value bo for each pixel and perform data conversion.

By the background synthesizing process based on the gray value, the pixel which is close to the gray value of the background in the input image at the boundary portion becomes a value close to the background value in the output, and conversely the value of the person in the input image. Pixels that are close to are not converted so much and have an output value close to the human value. For example, a portion having an intermediate value between the background and the person, in which the background can be seen through the person's hair, becomes an intermediate color between the background and the person portion to be combined in the output.

Specifically, data conversion as shown in FIG. 10 is performed based on the input background value and the output background value. The image has 256 shades of gray and the input background value is "10".
If the input value of a pixel in the boundary is "150" when the output background value is "0" and the output background value is "200", the output is "2".
18 ".

In the input image, when the target is a person who is dressed in a color similar to the background, for example, a person wearing a white shirt in front of a white background, the conversion method using only the grayscale information as described above, Almost all of the outline becomes the background value to be combined. In order to prevent this, background synthesis processing is performed based on the distance from the background. However, if it is known that such a situation will not occur, this processing is unnecessary.

The background composition processing based on the distance from the background is performed according to the flowchart shown in FIG. That is, similar to the background synthesizing process based on the gray value described above, for example, in the column of y = y1, the boundary from the left end (x = 0, y = y1) of the image D (x, y) after the region division. Scanning is performed until the pixel reaches the pixel, and the x-coordinates of the pixels at the left end and the right end of the boundary area continuous with the boundary pixel are obtained and stored in the memory 10 as x1 and x2, respectively.

When neither the left or right end point is in contact with the background, the boundary area outputs the value of the input image without conversion, and the search for the next boundary area is started. When one of the left and right end points is in contact with the background, the number of pixels L of the width of the boundary is obtained from the x coordinates x1 and x2 of the end points.

Next, for each pixel (x, y) in the boundary area, the distance w between the x coordinate of the end point pixel on the person side and the corresponding pixel is obtained and compared with the width L of the boundary to obtain the distance. Calculate the coefficient (= w / L). The image value E (x, y) which is the result of the background synthesizing process based on the distance coefficient and the gray value described above.
And the person image A (x, y), the product of the differences is the converted value of the corresponding pixel (x, y). Therefore, a value obtained by adding the converted value ([E (x, y) -A (x, y)] * w / L) and the value A (x, y) of the input image is the combined result F.
Output as (x, y).

The above process is performed for each pixel in the boundary region from x = x1 to x = x2, and the end of the boundary x = x2.
After conversion to, the next boundary area is scanned and the same processing is repeated. This operation is performed for all columns in the image.

This is because the distance from the inner contour (person side) of each pixel in the boundary area and the width of the boundary are used to obtain a conversion result based on the gray value if the pixel is located near the background side. This is a process of outputting a value close to the input value with almost no conversion if a value close to the person is located near the person. As a result, even if pixels that output the same value in the conversion process based only on the gray value, the background value gradually changes from the person side to the background side, and the background value naturally blends into the person part in the output image. Becomes

For example, it is assumed that there is a boundary portion as shown in FIG. 9, and the input background value and the pixel value within the boundary are all "100". When the output background value is "200", all the pixels within the boundary are "20" by the conversion based on the gray value described above.
Will be converted to "0" (see FIG. 10). The difference "100" between the output value "200" and the input value "100" is the converted value.

Next, in step S5, output processing is performed. That is, the output image F (x, y) obtained by adding the output results of the three regions of the person, the background, and the boundary obtained by the background synthesizing unit 14 is converted into an analog signal by the D / A converter 3 and displayed on the display 5. By displaying
Ask the operator to evaluate the processing results. Here, when the operator inputs information indicating that the processing result has succeeded by the keyboard 9, the color printer 4 operates and prints out the above-mentioned composition result.

Next, another embodiment of the present invention will be described. In this embodiment, the contour extraction unit 12 in the above embodiment is used.
Further, the processing by the manual designation of the area dividing unit 13 is automated by using a general image processing technique.

For example, in the case of a simple background such as an identification photograph, the contour of a person can be obtained by detecting a portion in the image where the change in pixel shade in the scanning direction is large. This is a process in which the difference between the relevant pixel and the previous pixel in the scanning direction is calculated, and if the difference is larger than a certain value, it is regarded as a contour.

For each of the connected regions in the image after the contour extraction, a person region that does not include any background inside the contour line of the person, a background region that does not include the person region outside the contour line, or the contour line The image can be automatically divided by determining whether it is the boundary area including both the background and the person around the area based on a preset condition. This judgment method is, for example, in the case of an ID photo, the connected area at the top of the image is generally the background, so if you scan the image from the upper left corner and find pixels other than the first outline, , The background area is obtained by searching for pixels other than the contour connected to the pixel.

Area division is performed by defining the contour as the boundary area and the portion that is neither the background nor the contour as the human area. As another determination method, an area at the upper left corner or the upper right corner of the image is assumed to be the background area, and the average value of the area in the original image is stored as the background average value of the input image, and each area in the contour image is stored. If the average value of the input images in the connected area is close to the background average value, it can be regarded as the background area.

In the above embodiment, the background synthesizing process using the black-and-white photograph as the input image has been described, but a color image can also be carried out. In that case, the input image is captured by the scanner 1 in color, stored in the image memory 11,
The monochrome conversion unit 15 converts the image into a monochrome gray-scale image, the contour extraction process is performed using the monochrome image in the same manner as in the above-described embodiment, and the R, G, B color images stored in the image memory 11 are combined. Then, each is converted into a color image of an output image and is output by the printer 4.

Further, the image input means is means for taking in the person image to be processed and the background image to be combined,
In the above-described embodiment, the scanner was used for the description, but it is also possible to use an image from a video camera or image information recorded on a CD-ROM or the like.

Similarly, as the output means of the composite image, by storing the composite image in the image memory 11,
Although various output formats can be adopted according to the wishes of the user, for example, in addition to the output to the printer 4 as in the above-described embodiment, it is stored in a recording medium capable of recording image information such as a floppy disk or an optical disk. It is also possible to do so.

As described above, according to the above embodiment,
The person image is divided into three areas, that is, a person area, a background area, and a boundary area, and the output value is determined by a different processing method in each area. In particular, in the boundary portion, since the processing for maintaining the relationship between the density value of the person and the background in the input image as much as possible is performed by using the grayscale information and the distance information, the cut-and-paste discomfort is eliminated and the natural background synthesis result is output. To do.

Further, since the background synthesis is performed based on the gray value, the input value of each pixel at the boundary portion between the background and the person, the background value to be synthesized, and the background value in the input image are compared to obtain the respective values in the input image. Outputs a value that maintains the relationship between the pixel and the background. For example, in the hair part of a person,
For the part where the background can be seen through between the hairs of the input image, conversion is performed so that it appears as an intermediate color between the background and the person part to be combined in the output.

Further, when the background and the clothing color are similar to each other in the input, if the background synthesizing process based on the gray value is performed, the boundary portion is converted into the same value as the background to be almost synthesized. In order to prevent this, when the input values of the background and the boundary are close to each other, background synthesis processing is performed based on the distance from the background in addition to the gray value. This uses the distance from the inner contour (person side) of each pixel in the boundary area, and if the pixel is located near the background side, a value close to the processing result of the background synthesis processing based on the gray value is used. Conversely, if it is located near the person, a value close to the input is output. That is,
In the background synthesis processing based on the gray value, even if the output of all the pixels within the boundary becomes the background value to be synthesized, by adding this conversion processing, the value gradually changes from the person side to the background side. The result is that the background naturally blends into the person portion in the output image.

Therefore, it is possible to perform background composition using a human image with an arbitrary background. In other words, unlike background synthesis with chroma key, it does not require shooting in front of a specific background,
A person image in front of an arbitrary background can be combined.

Further, the background synthesis can be performed without any discomfort.
That is, in the background synthesis of an object having a contour that is not separated from the background by a clear boundary, such as around a person's hair, a natural synthesis result that is not a cut-and-paste processing result can be output.

[0055]

As described above in detail, according to the present invention, it is possible to perform background composition using a human image of an arbitrary background, and to eliminate the feeling of strangeness caused by cut and paste, and to perform natural background composition. A synthesizer can be provided.

[Brief description of drawings]

FIG. 1 is a block diagram schematically showing the configuration of a person image background composition device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing the overall flow of processing.

FIG. 3 is a flowchart showing a flow of synthesizing processing.

FIG. 4 is a flowchart showing a flow of background synthesizing processing based on a gray value during the synthesizing processing.

FIG. 5 is a flowchart showing the flow of background synthesizing processing based on the distance from the background during the synthesizing processing.

FIG. 6 is a diagram showing an image after contour information extraction processing.

FIG. 7 is a diagram showing an image after region division processing.

FIG. 8 is a diagram for explaining parameters of background synthesis processing.

FIG. 9 is a diagram illustrating parameters in a boundary area.

FIG. 10 is a diagram illustrating a data conversion table for background synthesis processing based on grayscale values.

[Explanation of symbols]

1 ... Scanner, 2 ... Frame memory, 3 ... D / A
Converter, 4 ... Printer, 5 ... Display, 6 ...
Controller, 7 ... Mouse, 8 ... Tablet, 9 ...
... keyboard, 10 ... memory, 11 ... image memory,
12 ... contour extraction unit, 13 ... region division unit, 14 ... background synthesis unit, 15 ... color / monochrome conversion unit.

Claims (3)

[Claims] 1. An image input means for inputting a person image and a background image to be combined, a contour extracting means for extracting contour information of a person from the person image input by the image input means, and the contour extracting means. Area dividing means for dividing the input person image into three areas of a person, a background, and a boundary based on the contour information of the person, and an image obtained as a result of the area division by the area dividing means is sequentially scanned from one end. Thus, when the corresponding pixel is the human area, the input human image is output, when the pixel is the background area, the input background image is output, and when the pixel is the boundary area, the grayscale of each pixel in the boundary area is output. A synthesizing unit that synthesizes the input person image and the background image based on the value and outputs the synthesis result, and the output results of the three areas of the person, the background, and the boundary obtained by the synthesizing unit. People image background synthesis apparatus characterized by comprising output means for outputting the added synthetic result. 2. An image input means for inputting a person image and a background image to be combined, a contour extracting means for extracting contour information of a person from the person image input by the image input means, and the contour extracting means for extracting the contour information. Area dividing means for dividing the input person image into three areas of a person, a background, and a boundary based on the contour information of the person, and an image obtained as a result of the area division by the area dividing means is sequentially scanned from one end. Thus, when the corresponding pixel is a person area, the input person image is output, when the pixel is a background area, the input background image is output, and when the pixel is a boundary area, the background of each pixel in the boundary area is output. A combination means for combining the input person image and the background image based on the distance from, and outputting the combined result, and a person, background, and boundary three areas obtained by the combining means. People image background synthesizer to output means for outputting a synthesized result of adding the force results, characterized by comprising a. 3. An image input unit for inputting a person image and a background image to be combined, a contour extracting unit for extracting contour information of a person from the person image input by the image input unit, and an extraction by the contour extracting unit. Area dividing means for dividing the input person image into three areas of a person, a background, and a boundary based on the contour information of the person, and an image obtained as a result of the area division by the area dividing means is sequentially scanned from one end. Thus, when the corresponding pixel is the person area, the input person image is output, in the case of the background area, the input background image is output, and in the case of the boundary area, each pixel in the boundary area is output. A first combining process of combining the input human image and the background image based on the gray value and outputting the combined result is performed, and in the case of a boundary area, each pixel in the boundary area The input person image and the processing result of the first combining processing based on the distance from the background, and a second combining processing for outputting the combined result, and the combining means A person image background synthesizing apparatus comprising: an output unit that outputs a synthesizing result obtained by adding output results of the obtained three regions of a person, a background, and a boundary.