JP2008250366A - Person extraction method, its device, its program, and recording medium with same program recorded - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a person extraction system for efficiently extracting a person from a mixed image. <P>SOLUTION: A person extraction device 100A divides a background image and a mixed image into a plurality of blocks configured of a plurality of block configuring elements, and calculates the representative value of the color specification values of those block configuring pixels, and generates a representative value background image and a representative value mixed image in which the representative value is set in the block central pixel, and compares the representative value background image and the representative value mixed image for every block, and when the difference of the representative values is a predetermined value or more, sets the block central pixel to "1" as a person image block, and when the difference of the representative values is less than the predetermined value, sets the block central pixel to "0" as a background image block to generate a background difference binary image, and sets the central pixel of the block set to "1" in the background difference binary image to the color of the mixed image, and sets the block central pixel set to "0" to a predetermined color to generate a block filter processing color image by extracting a person from the mixed image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention is used for an interphone with a camera, a video conference system, a video phone, and the like, and in particular, a person extraction method for performing person extraction and person face extraction by a background difference method based on a shooting screen, its apparatus, its program, and The present invention relates to a recording medium on which the program is recorded.

  Conventionally, in camera intercoms, video conferencing systems, and videophones, methods to transmit or display unnecessary background images by extracting people or people's faces from the captured images are considered. Has been. When a person is extracted using a background difference or an interframe difference, the background may be output as noise, or the extracted person image may be worm-eaten. Thus, a method for extracting only a person cleanly has been studied (see, for example, Patent Document 1).

  According to the method described in Patent Document 1, a person image is extracted from a mixed image in which a person and a background are captured by a background difference method in units of pixels, and then a majority filter process and a blocking process are performed to remove background noise. The structure which removes the worm-eaten of a person is taken.

Japanese Patent Laid-Open No. 2001-14474

  However, in the case of an interphone camera, the background may be an outdoor road or another house across the road. In addition, in a camera installed outdoors, the camera itself may be shaken by the wind. Therefore, even if a background image is taken immediately before a person image is taken, the background image may change to some extent. In such a case, the method of performing various signal processing as post-processing after extracting a person image based on the pixel unit difference between the mixed image and the background image, as in the configuration of Patent Document 1, is efficient. It may be difficult to perform person extraction.

  An object of the present invention is to provide a person extraction method, an apparatus thereof, a program thereof, and a recording medium on which the program is recorded capable of efficiently extracting a person from a mixed image.

  The invention according to claim 1 divides a background image in which a predetermined background is photographed into blocks composed of a plurality of pixels, calculates a representative value of values representing the color of each block in a predetermined color system, A representative value background image composed of representative values of each block is generated, and a mixed image in which the background of the background image and a person are photographed is divided into blocks composed of the plurality of pixels, and the representative value of each block The block representative value image generation step for generating a representative value mixed image composed of the representative values of each block, the representative value background image and the representative value mixed image are compared for each block, and the representative value By setting the block having a difference equal to or greater than a predetermined value to a first value as a person image block, and setting the block having a difference less than the predetermined value to a second value as a background image block, A background difference binarized image generating step for generating a binarized image, a pixel set as the first value of the background difference binarized image is set as a color of the mixed image, and the second A block extraction process image generation step of generating a block filter process image by setting a pixel set to a value to a predetermined color is a person extraction method characterized by the following.

  The invention according to claim 5 divides a background image in which a predetermined background is photographed into blocks composed of a plurality of pixels, calculates a representative value of values representing the color of each block in a predetermined color system, A representative value background image composed of representative values of each block is generated, and a mixed image in which the background of the background image and a person are photographed is divided into blocks composed of the plurality of pixels, and the representative value of each block Block representative value image generating means for generating a representative value mixed image composed of representative values of each block, the representative value background image and the representative value mixed image are compared for each block, and the representative value By setting the block having a difference equal to or greater than a predetermined value to a first value as a person image block, and setting the block having a difference less than the predetermined value to a second value as a background image block, a background difference Background difference binarized image generating means for generating a binarized image, and the pixel set to the first value of the background difference binarized image is set to the color of the mixed image, and the second value is set. A person extraction device comprising block filter processed image generation means for generating a block filter processed image by setting the set pixels to a predetermined color.

  The invention described in claim 7 is a person extraction program characterized by causing a calculation means to execute the person extraction method according to any one of claims 1 to 4.

  The invention described in claim 8 is a recording medium on which the person extraction program according to claim 7 is recorded so as to be readable by the calculation means.

[First Embodiment]
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a first embodiment according to the invention will be described with reference to the drawings.
In the first embodiment and the second to fifth embodiments to be described later, a person extraction system provided with the person extraction device of the present invention is a background image in which a background is photographed and a mixture in which the background and a person are photographed. A configuration for extracting a person of a mixed image based on the image will be described as an example.
Note that the background image and the mixed image may be either a still image or a moving image. Further, either color or black and white may be targeted.

[Configuration of person extraction system]
First, the configuration of the person extraction system will be described with reference to the drawings.
FIG. 1 is a block diagram showing a schematic configuration of a person extraction system. FIG. 2 is a schematic diagram showing effective areas and blocks in the background image and the mixed image. FIG. 3 is a schematic diagram showing a background difference binarized image. FIG. 4 is a diagram for explaining the isolated point removal processing, where (A), (B), (C), and (D) represent cases where isolated points are removed, and (E) and (F) represent This represents the case where isolated points are not removed. FIG. 5 is a schematic diagram showing a block filter processed color image. FIGS. 6A and 6B are diagrams for explaining edge detection processing in the X direction. FIG. 6A illustrates a case where it is determined to be an edge, and FIG. 6B illustrates a case where it is determined that the edge is not an edge. FIGS. 7A and 7B are diagrams for explaining edge detection processing in the Y direction. FIG. 7A illustrates a case where it is determined that the edge is detected, and FIG. 7B illustrates a case where it is determined that the edge is not an edge. FIG. 8 is a schematic diagram showing an output image.

In FIG. 1, 1A is a person extraction system that extracts a person in a mixed image based on a background image and a mixed image.
The person extraction system 1A includes a photographing unit 10, a display unit 20A, a person extraction device 100A, and the like.

The photographing means 10 is a so-called intercom camera, for example, and is connected to the person extracting device 100A and installed outdoors.
The photographing unit 10 appropriately captures a background image or a mixed image under the control of the person extracting device 100A, and transmits it to the person extracting device 100A.

The display means 20A is an interphone monitor, for example, and is connected to the person extraction device 100A and installed indoors, for example.
The display means 20A appropriately displays a person or the like extracted from the mixed image under the control of the person extracting device 100A.

  The person extraction device 100A includes an image memory 110, a control unit 120A as a calculation unit, and the like.

  The image memory 110 is connected to the control unit 120A. The image memory 110 stores various images so as to be appropriately readable under the control of the control unit 120A.

  The control unit 120A includes various programs, and includes a photographing control unit 121, a block representative value image generation unit 122A, a background difference binarized image generation unit 123A, an isolated point removal unit 124, and a block filter processed image generation unit. A block filter processing color image generation unit 125A, an edge detection processing binarized image generation unit 126, an image output control unit 127A that also functions as an output image generation unit, and the like.

  The imaging control unit 121 controls the imaging unit 10 to capture a background image or a mixed image. Specifically, as shown in FIG. 2, from 400 pixels G (X, Y) (X = 1 to 20, Y = 1 to 20) arranged in parallel in the X direction and the Y direction, respectively. The background image and the mixed image that are configured are photographed. Then, the photographed background image or mixed image is acquired and stored in the image memory 110.

The block representative value image generation unit 122A generates a representative value background image and a representative value mixed image as a block representative value image based on the background image and the mixed image.
Specifically, the block representative value image generation unit 122A performs pixel G (3,3), pixel G (3) in the background image and the mixed image composed of the pixels G (X, Y) as shown in FIG. , 18), pixel G (18, 18), and 256 pixels G (I, J) (I = 3-18, J = 3-18) having the pixel G (18,3) as four corners The effective area U to be recognized is recognized. Further, the effective area U is divided into a pixel G (I, J) (hereinafter referred to as a block center pixel G as appropriate) included in the effective area U, a pixel G (X, Y) around the block center pixel G, Are divided into a plurality of blocks C (I, J) (hereinafter referred to as block C as appropriate). Specifically, the effective area U is divided into a pixel G (I-2, J-2), a pixel G (I-2, J + 2), a pixel G (I + 2, J + 2), and a pixel G. The block is divided into 256 blocks C composed of 25 pixels G (X, Y) (hereinafter referred to as block constituent pixels G) having (I + 2, J-2) as four corners. That is, the block size is composed of five block constituent pixels G in the X direction and five in the Y direction, and the block size is divided into blocks “5 × 5”.
The block representative value image generating unit 122A calculates a representative value of values representing the colors of the 25 block constituent pixels G in each block C of the background image in a predetermined color system (hereinafter referred to as color values). To do. Then, a representative value background image in which the calculated representative value is set in the block center pixel G is generated and stored in the image memory 110 as appropriate.
Here, examples of the predetermined color system include CIE (International Lighting Commission) color system such as RGB color system and XYZ color system, Munsell color system, and the like. Moreover, as a representative value, an average value, an intermediate value, etc. can be illustrated.
Further, the block representative value image generation unit 122A calculates the representative value of the block constituent pixel G in each block C for the mixed image, and displays the representative value mixed image in which the calculated representative value is set in the block center pixel G as an image. The data is appropriately stored in the memory 110.

The background difference binarized image generating means 123A generates a background difference binarized image Ph as shown in FIG.
Specifically, the background difference binarized image generation unit 123A acquires the representative value background image and the representative value mixed image, and determines that the difference between the representative values of the comparison target block C is greater than or equal to a predetermined value. That is, when the difference between the representative values in the block C of the representative value background image and the representative value mixed image is large, the block C of the representative value mixed image is recognized as a person image block representing a person instead of the background. Then, the block center pixel G is set to “1” as the first value. When it is determined that the difference between the representative values is less than the predetermined value, that is, when the difference between the representative values in the block C of the representative value background image and the representative value mixed image is small, the block C of the representative value mixed image represents the background. Recognizing it as a background image block, the block center pixel G is set to “0” as the second value.
Then, the background difference binarized image generation unit 123A performs the above-described processing on the block C for one frame, and the background difference binarized image in which the block center pixel G is set to “1” or “0”. Ph is generated and stored in the image memory 110 as appropriate.

The isolated point removing unit 124 removes isolated points generated in the background difference binarized image Ph due to noise or the like in the background image or the mixed image.
Specifically, the isolated point removing unit 124 acquires the background difference binarized image Ph, the block center pixel G to be processed is “0”, and the block center pixel G of “0” is X When it is determined that the number consecutive in the direction or the Y direction is M or less (M is a natural number), the block center pixel G to be processed is recognized as an isolated point. Then, the setting of the block center pixel G is changed from “0” to “1”. When it is determined that the block center pixel G to be processed is “1” and the number of consecutive block center pixels G of “1” in the X direction or the Y direction is N or less (N is a natural number). Recognizing that this processing target block center pixel G is an isolated point, the setting is changed from “1” to “0”.
For example, in the configuration in which M is set to “2”, when the block center pixel G is set as shown in FIGS. 4A, 4B, 4C, and 4D, “0” is continuous. Since the number to be performed is “2” or “1”, a process of changing the setting of the block center pixel G from “0” to “1” is performed as a process of removing the isolated point. Also, in the case of setting the block center pixel G as shown in FIGS. 4E and 4F, since the number of consecutive “0” s is “3”, it is determined that the point is not an isolated point and the isolated point is not removed. .
Then, the isolated point removing unit 124 performs the above-described processing on the block center pixel G for one frame, and appropriately stores the background difference binarized image Ph from which the isolated points are appropriately removed in the image memory 110.
The values of M and N may be values other than “2”. Further, the values of M and N may be the same or different.

The block filter processing color image generation means 125A generates a block filter processing color image Pf1 as a block filter processing image as shown in FIG.
Specifically, the block filter processing color image generation unit 125A acquires the background difference binarized image Ph from which the isolated points are removed by the isolated point removing unit 124, and the block center pixel G to be processed is “1”. Is determined to be the color of the corresponding block center pixel G in the mixed image. Further, when it is determined that the block center pixel G to be processed is “0”, it is set to white as a predetermined color.
Then, the block filter processing color image generation means 125A performs the above-described processing on the block center pixel G for one frame, and the block filter processing color in which the block center pixel G is set to the color of the mixed image or white. The image Pf1 is stored in the image memory 110 as appropriate.

The edge detection processing binarized image generating means 126 detects the person and background edges in the background difference binarized image Ph, generates an X direction binarized image and a Y direction binarized image, and based on these Then, an edge detection processing binarized image in which the edge is set is generated.
Specifically, the edge detection process binarized image generation means 126 acquires the background difference binarized image Ph, the block center pixel G (I, J) to be processed is “1”, and It is determined whether or not the settings of the pixels G (I−1, J) and G (I + 1, J) adjacent to the block center pixel G (I, J) in the X direction are different. If this condition is satisfied, it is determined that the block center pixel G (I, J) is an edge, and is set to “1”. If the condition is not satisfied, it is determined that the block center pixel G (I, J) is not an edge, and is set to “0”. For example, in the case shown in FIG. 6A, it is determined that the block center pixel G (I, J) is an edge, and is set to “1”. In the case shown in FIG. It is determined that the center pixel G (I, J) is not an edge, and is set to “0”.
Then, an X-direction binarized image in which the block center pixel G is set to “1” or “0” is generated.

Further, the edge detection processing binarized image generating means 126 has a processing target block center pixel G (I, J) of “1” and is adjacent to the pixel G (I, J−1), Y in the Y direction. It is determined whether the setting of G (I, J + 1) is different. If this condition is satisfied, it is determined that the block center pixel G (I, J) is an edge, and is set to “1”. If the above condition is not satisfied, it is determined that the block center pixel G (I, J) is not an edge, and is set to “0”. For example, in the case shown in FIG. 7A, it is determined that the block center pixel G (I, J) is an edge and is set to “1”. In the case shown in FIG. It is determined that the pixel G (I, J) is not an edge, and is set to “0”.
Then, a Y-direction binarized image in which the block center pixel G is set to “1” or “0” is generated.

  Then, the edge detection process binarized image generation unit 126 calculates the logical sum of the X direction binarized image and the Y direction binarized image, and generates an edge detection process binarized image. That is, an edge detection processing binarized image in which the block center pixel G corresponding to the edge of the person is set to “1” and all the block center pixels G corresponding to other than the edge are set to “0” is generated, The data is stored in the image memory 110 as appropriate.

The image output control means 127A generates an output image Pr1 as shown in FIG. 8 and displays it on the display means 20A.
Specifically, the image output control unit 127A acquires the mixed image, the block filter processing color image Pf1, and the edge detection processing binarized image. Then, the block center pixel G to be processed corresponding to each other in the block filter processing color image Pf1 and the edge detection processing binarized image is compared, the block center pixel G is non-edge (not an edge), and It is determined whether or not “0” is set in the block filter processed color image Pf1. When this condition is satisfied, the block center pixel G is determined to be the background, and the block center pixel G is set to a desired background color. If the above condition is not satisfied, it is determined whether the image is non-edge and is set to “1” in the block filter processed color image Pf1. When this condition is satisfied, it is determined that the block center pixel G is a person, and the color of the mixed pixel is set. On the other hand, if the above condition is not satisfied, it is determined whether the edge is set and “1” is set in the block filter processed color image Pf1. If this condition is satisfied, the block center pixel G is determined to be an edge, and is set to the color of the block filter processed color image Pf1. Here, it is set to white.
Then, the image output control means 127A is arranged such that the block center pixel G (3, 3) represented by a plurality of right-sloping diagonal lines having a wide interval as shown in FIG. The block center pixel G (5,3), etc., represented by a plurality of diagonal lines is set to the color of the mixed image, and the block center pixel G (4,3), etc., shown without the diagonal lines, is set to white. The output image Pr1 is displayed on the display means 20A.

[Operation of person extraction system]
First, the operation of the person extraction system will be described with reference to the drawings.
FIG. 9 is a flowchart showing the person extraction process. FIG. 10 is a flowchart showing block representative value image generation processing. FIG. 11 is a flowchart showing background difference binarized image generation processing. FIG. 12 is a flowchart showing isolated point removal processing. FIG. 13 is a flowchart showing block filter processing color image generation processing. FIG. 14 is a flowchart showing the edge detection process binarized image generation process. FIG. 15 is a flowchart showing the image output control process.

First, as shown in FIG. 9, the person extraction system 1A performs block representative value image generation processing as a block representative value image generation step (step S1). Then, based on the representative value background image and the representative value mixed image generated in step S1, background difference binarized image generation processing is performed as a background difference binarized image generation step (step S2).
Thereafter, an isolated point removal process is performed on the background difference binarized image Ph generated in step S2 (step S3). Based on the background difference binarized image Ph from which the isolated points are removed, A block filter process color image generation process is performed as a block filter process image generation step (step S4). Furthermore, based on the background difference binarized image Ph generated in step S2, an edge detection process binarized image generation process is performed as an edge detection process binarized image generation step (step S5). Then, based on the edge detection processing binarized image generated in step S5, the mixed image generated in step S1, and the block filter processing color image Pf1 generated in step S4, an output image Pr1 is obtained. Image output control processing is performed as an output image generation step to be generated and displayed (step S6).

  Here, in the block representative value image generation process in step S1, the person extraction device 100A of the person extraction system 1A generates a background image by controlling the photographing means 10 at a predetermined timing, as shown in FIG. S11). Then, the representative value of each block C of the background image is calculated (step S12), and a representative value background image based on the representative value is generated (step S13). Furthermore, the person extracting device 100A generates a mixed image by controlling the photographing unit 10 in a state where the person is positioned in front of the photographing unit 10 (step S14). Then, the representative value of each block C of the mixed image is calculated (step S15), and a representative value mixed image based on the representative value is generated (step S16).

  In the background difference binarized image generation process in step S2, the person extraction device 100A compares the representative values of the comparison target block C in the representative value background image and the representative value mixed image as shown in FIG. (Step S21), it is determined whether or not the difference between the representative values is greater than or equal to a predetermined value (Step S22). If it is determined in this step S22 that it is equal to or greater than the predetermined value, the block center pixel G of the background difference binarized image Ph is set to “1” (step S23), and if it is determined that it is less than the predetermined value, the block The central pixel G is set to “0” (step S24). Then, it is determined whether or not the processing for one frame has been completed (step S25). If it is determined that the processing has ended, the processing ends. If it is determined that the processing has not ended, the processing returns to step S21.

Furthermore, in the isolated point removal process in step S3, the person extraction device 100A determines whether or not the block center pixel G to be processed in the background difference binarized image Ph is “0” as shown in FIG. Step S31). If it is determined in this step S31 that it is “0”, it is determined whether or not the number of consecutive “0” block center pixels G in the X direction or the Y direction is M or less (step S32). If it is determined in step S32 that it is M or less, the block center pixel G is changed from “0” to “1” (step S33). Further, when step S33 is performed, or when it is determined that it is not M or less in step S32, it is determined whether or not the processing for one frame has been completed (step S34). If it is determined that the process has not been completed, the process returns to step S31.
If it is determined in step S31 that it is “1”, it is determined whether the number of consecutive block center pixels G of “1” in the X direction or the Y direction is equal to or less than N (step S35). If it is determined in this step S35 that it is not N or less, the process of step S34 is performed. If it is determined that it is N or less, the block center pixel G is changed from “1” to “0” (step S36). The process of step S34 is performed.

  In the block filter processing color image generation processing in step S4, the person extraction device 100A determines whether or not the block center pixel G to be processed in the background difference binarized image Ph is “1” as shown in FIG. Judgment is made (step S41). In this step S41, if it is determined that it is “1”, the block center pixel G of the block filter processed color image Pf1 is set to the color of the mixed image (step S42), and if it is determined that it is “0”, the block The center pixel G is set to white (step S43). Then, it is determined whether or not the processing for one frame has been completed (step S44). If it is determined that the processing has ended, the processing ends. If it is determined that the processing has not ended, the processing returns to step S41.

Furthermore, in the edge detection process binarized image generation process in step S5, the person extraction device 100A performs processing with the block center pixel G (I, J) to be processed in the background difference binarized image Ph as shown in FIG. The values of the pixels G (I−1, J) and G (I + 1, J) adjacent in the X direction are compared (step S51), and the block center pixel G (I, J) is “1”. And whether or not the values of the pixels G (I−1, J) and G (I + 1, J) are different (step S52). In step S52, when it is determined that the condition is satisfied, the block center pixel G of the X-direction binarized image is set to “1” (step S53), and when it is determined that the condition is not satisfied, it is set to “0”. (Step S54). Thereafter, it is determined whether or not the processing for one frame has been completed (step S55). If it is determined that the processing has not been completed, the process returns to step S51.
On the other hand, if it is determined in step S55 that the processing has ended, the block center pixel G (I, J) and the pixels G (I, J-1) and G (I, J + 1) adjacent in the Y direction are The values are compared (step S56), the block center pixel G (I, J) is “1”, and the values of the pixels G (I, J−1) and G (I, J + 1) are different. Whether or not (step S57). If it is determined in step S57 that the condition is satisfied, the block center pixel G of the Y-direction binarized image is set to “1” (step S58). If it is determined that the condition is not satisfied, it is set to “0”. (Step S59). Thereafter, it is determined whether or not the processing for one frame has been completed (step S60). If it is determined that the processing has not been completed, the process returns to step S56. Based on the Y-direction binarized image, an edge detection process binarized image is generated (step S61).

In the image output control process in step S6, the person extraction device 100A compares the block center pixel G to be processed in the edge detection process binarized image and the block filter process color image Pf1, as shown in FIG. (Step S71), it is determined whether or not the block center pixel G is a non-edge and is set to “0” in the block filter processed color image Pf1 (Step S72). If it is determined in step S72 that the condition is satisfied, the block center pixel G of the output image Pr1 is set to a desired background color (step S73). If the condition is not satisfied in step S72, it is determined whether the edge is non-edge and is set to “1” in the block filter processed color image Pf1 (step S74).
If the condition is satisfied in step S74, the block center pixel G is set to the color of the mixed pixel (step S75). On the other hand, if the condition is not satisfied in step S74, it is determined whether the edge is set and “1” is set in the block filter processed color image Pf1 (step S76). If the condition is satisfied in step S76, the block center pixel G is set to the color of the block filter processed color image Pf1, that is, white (step S77).
Then, after the process of step S73 or step S75, or when it is determined in step S76 that the condition is not satisfied, it is determined whether or not the process for one frame has been completed (step S78), and has not been completed. If it is determined, the process returns to step S71, and if it is determined that the process has been completed, the output image Pr1 is output by the display means 20A (step S79).

[Effects of First Embodiment]
As described above, in the first embodiment, the background image and the mixed image are divided into a plurality of blocks C each composed of a plurality of block constituent pixels G in the person extraction device 100A. Then, the representative value of the color value of each block constituent pixel G is calculated, and a representative value background image and a representative value mixed image in which this representative value is set in the block center pixel G are generated. Thereafter, the representative value background image and the representative value mixed image are compared for each block C, and when the difference between the representative values is equal to or larger than a predetermined value, the block center pixel G is set to “1” as a person image block, and is less than the predetermined value. In this case, the background difference binarized image Ph set to “0” as the background image block is generated. Then, by setting the block center pixel G of “1” in the background difference binarized image Ph to the color of the mixed image and setting the block center pixel G of “0” to a predetermined color, a person is extracted from the mixed image. The extracted block filter processed color image Pf1 is generated.
For this reason, the representative values of the colorimetric values in the background image and the mixed image are compared in units of block C composed of a plurality of block constituent pixels G, so that the photographing means 10 and the background tree are shaken by the wind. Even if the background image has changed in units of pixel G (X, Y) due to the above, it is possible to efficiently remove a person from the mixed image in a state where the influence of the change in units of pixel G (X, Y) is removed. Can be extracted.

Furthermore, a person and background edges in the background difference binarized image Ph are detected, and an edge detection processing binarized image indicating that the predetermined block center pixel G is an edge or a non-edge is generated. Then, the block center pixel G determined to be the edge of the edge detection processing binarized image is set to white as the color of the block filter processing color image Pf1, and the block center pixel G determined to be a non-edge is set to the mixed image. The color is set, and the block center pixel G determined to be a non-edge is set to a predetermined background color, thereby generating and displaying the output image Pr1.
For this reason, a person who is efficiently extracted in white so-called bordered state can be displayed by the output image Pr1.

And the isolated point which generate | occur | produced in the background difference binarized image Ph by the noise etc. in a background image or a mixed image is removed.
For this reason, a person can be extracted from a mixed image efficiently and appropriately.

[Second Embodiment]
Next, a second embodiment according to the present invention will be described with reference to the drawings.
In addition, about the structure same as 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted suitably. Also, description of the operation of the person extraction system is omitted.

[Configuration of person extraction system]
First, the configuration of the person extraction system will be described with reference to the drawings.
FIG. 16 is a block diagram showing a schematic configuration of the person extraction system. FIG. 17 is a schematic diagram showing a composite image.

  In FIG. 16, 1B is a person extraction system, and this person extraction system 1B includes a photographing means 10, another image output means 30, a display means 20B, a person extraction device 100B, and the like.

The other image output means 30 is, for example, a so-called TV tuner, and is connected to the person extracting device 100B.
The other image output means 30 receives an image (hereinafter referred to as another image) distributed via a wireless medium such as a broadcast wave or a wired medium such as a cable, and outputs the received image to the person extracting device 100B.

The display unit 20B is, for example, a monitor of a television device, and is connected to the person extracting device 100B and installed indoors.
As shown in FIG. 17, the display unit 20B includes an output image Pr2 representing a person extracted from the mixed image and another image Pt received by the other image output unit 30 as shown in FIG. The synthesized composite image Pg1 and the like are displayed as appropriate.

  The person extraction device 100B has a configuration in which only the control unit 120B is different from the person extraction device 100A of the first embodiment. The control unit 120B has the same configuration as the control unit 120A of the first embodiment, and includes an image output control unit 127B instead of the image output control unit 127A.

The image output control means 127B generates a composite image Pg1 and displays it on the display means 20B.
Specifically, when the image output control unit 127B determines that the block center pixel G corresponding to the block filter processing color image Pf1 and the edge detection processing binarized image is the background, the block center pixel of the output image Pr2 G is set to the color of the other image Pt. Further, an output image Pr2 is generated in which the block center pixel G determined to be a person is set to the color of the mixed image, and the block center pixel G determined to be an edge is set to white which is the color of the block filter processed color image Pf1. . Then, the display unit 20B displays a composite image Pg1 obtained by combining the output image Pr2 with a part of another image Pt that is one scene of the television broadcast. That is, the composite image Pg1 in which the person H framed by the so-called white edge region W is combined with the other image Pt is displayed.

[Effects of Second Embodiment]
As described above, in the second embodiment, in addition to the same functions and effects as those in the first embodiment, the following functions and effects can be achieved.

In other words, the person extraction device 100B has the block center pixel G determined to be the background based on the block filter processing color image Pf1 and the edge detection processing binarized image as the color of the other image Pt. An output image Pr2 is generated by setting the determined image to the color of the mixed image and the determined image to white. Then, a synthesized image Pg1 obtained by synthesizing the output image Pr2 is displayed on a part of another image Pt that is one scene of the television broadcast.
For this reason, for example, even during viewing of a television broadcast, the face of the visitor can be recognized without leaving the user away from the site, and convenience can be improved.

[Third Embodiment]
Next, 3rd Embodiment which concerns on this invention is described based on drawing.
In addition, about the structure same as 1st, 2nd embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted suitably. Also, description of the operation of the person extraction system is omitted.

[Configuration of person extraction system]
First, the configuration of the person extraction system will be described with reference to the drawings.
FIG. 18 is a block diagram showing a schematic configuration of the person extraction system. FIG. 19 is a schematic diagram showing a composite image.

  In FIG. 18, 1C is a person extraction system, and this person extraction system 1C includes a photographing means 10, an other image output means 30, a display means 20B, a person extraction device 100C, and the like.

  The person extracting device 100C has a configuration in which only the control unit 120C is different from the person extracting device 100B of the second embodiment. The control unit 120C has the same configuration as the control unit 120B of the second embodiment, and instead of the block filter processing color image generation unit 125A and the image output control unit 127B, the block filter processing color image generation unit 125C and The image output control means 127C is provided, and the edge detection processing binarized image generation means 126 is not provided.

The block filter processing color image generation means 125C generates a block filter processing color image Pf2 as shown in FIG.
Specifically, the block filter processing color image generation unit 125C determines whether or not the block center pixel G to be processed in the background difference binarized image Ph from which the isolated point is removed by the isolated point removing unit 124 is “0”. to decide. If it is “0” and it is determined to be the background, the block center pixel G of the block filter processed color image Pf2 is set to the color of the other image Pt. The color of the image is set to generate a block filter processed color image Pf2.

The image output control means 127C generates a composite image Pg2 as shown in FIG. 19 and displays it on the display means 20B.
Specifically, the image output control unit 127C acquires the block filter processed color image Pf2, and causes the display unit 20B to display the composite image Pg2 synthesized with a part of the other image Pt. That is, the composite image Pg2 in which the person H that is not bordered in the border region W is combined with the other image Pt is displayed.

[Effects of Third Embodiment]
As described above, in the third embodiment, in addition to the same functions and effects as in the first and second embodiments, the following functions and effects can be achieved.

That is, in the background difference binarized image Ph by the person extraction device 100C, the block center pixel G determined to be the background is the color of the other image Pt, and the image determined to be the person is the color of the mixed image. Then, the set block filter processed color image Pf2 is generated. Then, a synthesized image Pg2 obtained by synthesizing the block filter processed color image Pf2 is displayed on a part of the other image Pt.
For this reason, without providing the edge detection processing binarized image generation means 126, for example, the user's face can be recognized without leaving the user who is watching the television broadcast away from the spot. Accordingly, convenience can be improved while simplifying the configuration.

[Fourth Embodiment]
Next, 4th Embodiment which concerns on this invention is described based on drawing.
In addition, about the structure same as 1st-3rd embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted suitably. Also, description of the operation of the person extraction system is omitted.

[Configuration of person extraction system]
First, the configuration of the person extraction system will be described with reference to FIG.

  In FIG. 1, a person extraction system 1D has a configuration in which only a person extraction device 100D is different from the person extraction system 1A of the first embodiment. Further, the person extracting device 100D has a configuration in which only the control unit 120D is different from the person extracting device 100A of the first embodiment. Furthermore, the control unit 120D has the same configuration as the control unit 120A of the first embodiment, and includes a block representative value image generation unit 122A, a background difference binarized image generation unit 123A, and a block filter processing color image generation unit. Instead of 125A, block representative value image generation means 122D, background difference binarized image generation means 123D, and block filter processing color image generation means 125D are provided.

  The block representative value image generating unit 122D divides the background image and the mixed image into blocks C having a block size of “5 × 5” by the same process as the block representative value image generating unit 122A of the first embodiment. Based on the representative value of block C, a representative value background image and a representative value mixed image are generated. Further, for example, the pixel G (I−2, J−2), the pixel G (I−2, J + 1), the pixel G (I + 1, J + 1), and the pixel G (I + 1, J−) 2) A block C having a block size of “4 × 4” composed of 16 block constituent pixels G having four corners, a pixel G (I−1, J−1), a pixel G (I−1, J +) 1), the block size composed of the pixel G (I + 1, J + 1) and the nine block constituent pixels G having the pixel G (I + 1, J-1) as the four corners is “3 × 3 And the representative value background image and the representative value mixed image corresponding to each of the blocks are generated.

  The background difference binarized image generating means 123D uses the background difference binarized images corresponding to the blocks C having the block sizes of “3 × 3”, “4 × 4”, and “5 × 5”, respectively. Generate Ph. Here, since the block sizes of the blocks C are different, each background difference binarized image Ph is appropriately different in the number, range, or position of the block C set to “1”, that is, the person image blocks.

The block filter processing color image generation means 125D selects a background difference binarized image Ph corresponding to each block size that is the same as or approximately equal to the preset number, range, or position of human image blocks. Then, based on the background difference binarized image Ph corresponding to the selected block size, a block filter processed color image Pf1 is generated and stored in the image memory 110 as appropriate.
Based on the block filter processed color image Pf1, the output image Pr1 is displayed on the display means 20A.

[Effects of Fourth Embodiment]
As described above, in the fourth embodiment, the following operational effects can be obtained in addition to the operational effects similar to those of the first embodiment.

That is, the person extraction device 100D generates a plurality of representative value background images, representative value mixed images, and background difference binarized images Ph having different block sizes. Then, from these background difference binarized images Ph of the respective block sizes, those which are the same as or approximately equal to the preset number, range or position of the person image blocks are selected, and this selected background difference binarized image An output image Pr1 based on Ph is displayed.
For this reason, for example, when the photographing means 10 or the background tree is shaken by the wind, by selecting the background difference binarized image Ph having a large block size, a certain amount of movement is made in units of block C. The background image block can be easily extracted. Therefore, a person can be extracted with high accuracy based on the background difference binarized image Ph having a block size corresponding to the installation environment of the photographing means 10.

[Fifth Embodiment]
Next, a fifth embodiment according to the present invention will be described with reference to the drawings.
In addition, about the structure same as 1st-4th embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted suitably. Also, description of the operation of the person extraction system is omitted.

[Configuration of person extraction system]
First, the configuration of the person extraction system will be described with reference to FIG.

  In FIG. 1, a person extraction system 1E has a configuration in which only a person extraction device 100E is different from the person extraction system 1A of the first embodiment. Further, the person extracting device 100E has a configuration in which only the control unit 120E is different from the person extracting device 100A of the first embodiment. Furthermore, the control unit 120E has the same configuration as the control unit 120A of the first embodiment, and includes a background difference binarized image generation unit 123E instead of the background difference binarized image generation unit 123A.

  The background difference binarized image generation means 123E compares the representative values of the block C to be compared in the representative value background image and the representative value mixed image. When it is determined that the difference between the representative values is less than the first predetermined value, the block C of the representative value mixed image is recognized as a background image block, and the block center pixel G is set to “0”. When it is determined that the difference between the representative values is equal to or greater than the second predetermined value that is larger than the first predetermined value, the block C of the representative value mixed image is recognized as a person image block, and the block center pixel G is set to “ Set to “1”. Further, if it is determined that the difference between the representative values is equal to or greater than the first predetermined value and less than the second predetermined value, whether or not the color represented by the representative value of the block C of the representative value mixed image is a flesh color. to decide. When it is determined that the color is skin color, it is recognized as a person image block, and the block center pixel G is set to “1”. When it is determined that the color is not skin color, it is recognized as a background image block, and “ Set to “0”. Then, a background difference binarized image Ph composed of a person image block and a background image block is generated.

[Effects of Fifth Embodiment]
As described above, in the fifth embodiment, in addition to the same functions and effects as those in the first embodiment, the following functions and effects can be achieved.

That is, if the person extraction device 100E determines that the difference between the representative values of the block C in the representative value background image and the representative value mixed image is equal to or greater than the first predetermined value and less than the second predetermined value, Based on whether the color represented by the representative value of the block C of the value-mixed image is a skin color, it is determined that it is a background image block or a person image block.
For this reason, even if it is difficult to determine whether the block is a person image block based on the difference between the representative values, the person can be extracted with higher accuracy by detecting the actual color in the block C.

[Modification of Embodiment]
Note that the present invention is not limited to the above-described embodiment, and includes the following modifications as long as the object of the present invention can be achieved.

  That is, in the first to fifth embodiments, the edge detection processing binarized image generation unit 126 and the image output control units 127A, 127B, and 127C may be omitted. Furthermore, the image capturing control unit 121 and the isolated point removing unit 124 may be omitted.

  The size or position of the output image Pr2 with respect to the other image Pt in the second embodiment and the size or position of the block filter processing color image Pf2 with respect to the other image Pt in the third embodiment can be appropriately changed. Alternatively, a configuration in which a predetermined state is set in advance may be employed.

  Furthermore, in the first to fifth embodiments, the average value or the intermediate value of the color values of the block constituent pixels G constituting the block C is used as the representative value, but the representative pixel G of the block constituent pixels G, for example, the block The color value of the center pixel G may be applied as it is as the representative value of the block C.

In the second and third embodiments, the photographing means 10 is a camera for photographing participants of the video conference system, and the other image output means 30 is a camera for photographing conference materials, and the display means 20B mainly performs the conference. It is good also as a structure which synthesize | combines and displays a participant's face in the part while displaying a document.
Furthermore, the person extracting device of the present invention may be applied to a video phone.

  Each function described above is constructed as a program. However, for example, it may be configured by hardware such as a circuit board or an element such as one integrated circuit (IC), and can be used in any form. Note that, by using a configuration that allows reading from a program or a separate recording medium, as described above, handling is easy, and usage can be easily expanded.

  In addition, the specific structure and procedure for carrying out the present invention can be appropriately changed to other structures and the like within a range in which the object of the present invention can be achieved.

[Effect of the embodiment]
As described above, in the above embodiment, the background image and the mixed image are divided into a plurality of blocks C each composed of a plurality of block constituent pixels G in the person extraction device 100A. Then, the representative value of the color value of each block constituent pixel G is calculated, and a representative value background image and a representative value mixed image in which this representative value is set in the block center pixel G are generated. Thereafter, the representative value background image and the representative value mixed image are compared for each block C, and when the difference between the representative values is equal to or larger than a predetermined value, the block center pixel G is set to “1” as a person image block, and is less than the predetermined value. In this case, the background difference binarized image Ph set to “0” as the background image block is generated. Then, by setting the block center pixel G of “1” in the background difference binarized image Ph to the color of the mixed image and setting the block center pixel G of “0” to a predetermined color, a person is extracted from the mixed image. The extracted block filter processed color image Pf1 is generated.
For this reason, the representative values of the color values in the background image and the mixed image are compared for each block C composed of a plurality of block constituting pixels G, so that the pixel G ( Even if the background image changes in units of X, Y), a person can be efficiently extracted from the mixed image in a state where the influence of the changes in units of pixels G (X, Y) is removed.

It is a block diagram which shows schematic structure of the person extraction system which concerns on 1st, 4th, 5th embodiment of this invention. It is a schematic diagram which shows the effective area | region and block in the background image and mixed image which concern on 1st-5th embodiment of this invention. It is a schematic diagram which shows the background difference binarized image in the said 1st-5th embodiment. It is a figure for demonstrating the isolated point removal process in the said 1st-5th embodiment, (A), (B), (C), (D) represents the case where an isolated point is removed, (E) , (F) represents a case where isolated points are not removed. It is a schematic diagram which shows the block filter process color image in the said 1st, 2nd, 4th, 5th embodiment. It is a figure for demonstrating the edge detection process to the X direction in the said 1st-5th embodiment, (A) represents the case where it is judged that it is an edge, (B) is judged not to be an edge. Represents the case. It is a figure for demonstrating the edge detection process to the Y direction in the said 1st-5th embodiment, (A) represents the case where it is judged that it is an edge, (B) is judged not to be an edge. Represents the case. It is a schematic diagram which shows the output image in the said 1st, 4th, 5th embodiment. It is a flowchart which shows the person extraction process in the said 1st, 2nd, 4th, 5th embodiment. It is a flowchart which shows the block representative value image generation process in the said 1st-3rd, 5th embodiment. It is a flowchart which shows the background difference binarized image generation process in the said 1st, 2nd embodiment. It is a flowchart which shows the isolated point removal process in the said 1st-5th embodiment. It is a flowchart which shows the block filter process color image generation process in the said 1st, 2nd, 4th, 5th embodiment. It is a flowchart which shows the edge detection process binarized image generation process in the said 1st, 2nd, 4th, 5th embodiment. It is a flowchart which shows the image output control process in the said 1st, 5th embodiment. It is a block diagram which shows schematic structure of the person extraction system in the said 2nd Embodiment. It is a schematic diagram which shows the synthesized image in the said 2nd Embodiment. It is a block diagram which shows schematic structure of the person extraction system in the said 3rd Embodiment. It is a schematic diagram which shows the synthesized image in the said 3rd Embodiment.

Explanation of symbols

100A, 100B, 100C, 100D, 100E ... person extraction device 120A, 120B, 120C, 120D, 120E ... control unit 122A, 122D ... block representative value image generating means 123A, 123D, 123E ... background difference binarization Image generation means 125A, 125C, 125D ... Block filter processing color image generation means as block filter processing image generation means 126 ... Edge detection processing binarized image generation means 127A, 127B, 127C ... Images that also function as output image generation means Output control means C ... Block G ... Pixel Ph ... Background difference binarized image Pf1, Pf2 ... Block filter processed color image as block filter processed image Pr1, Pr2 ... Output image

Claims (8)

A background image in which a predetermined background is photographed is divided into blocks composed of a plurality of pixels, a representative value of values representing the color of each block in a predetermined color system is calculated, and the representative value of each block is configured. A representative value background image is generated, a mixed image in which the background of the background image and a person are photographed is divided into blocks composed of the plurality of pixels, the representative value of each block is calculated, and the representative value of each block A block representative value image generating step for generating a representative value mixed image composed of:
The representative value background image and the representative value mixed image are compared for each block, the block in which the difference between the representative values is equal to or greater than a predetermined value is set as a first image block, and the block is less than the predetermined value. A background difference binarized image generating step for generating a background difference binarized image by setting the second value as a background image block;
By setting the pixel set to the first value of the background difference binarized image to the color of the mixed image and setting the pixel set to the second value to a predetermined color, a block filter A block filter processed image generation step for generating a processed image;
The person extraction method characterized by implementing. The person extraction method according to claim 1,
An edge detection processing binarization generation image step for detecting a background and a human edge in the background difference binarization image and generating an edge detection processing binarization image indicating that the predetermined pixel is an edge or a non-edge; and ,
The pixel determined as the edge of the edge detection processing binarized image is set as the color of the block filter processing image, and the pixel consisting of the first value determined as a non-edge is set as the color of the mixed image. An output image generation step of generating an output image by setting a pixel consisting of the second value determined as a non-edge to a predetermined background color;
The person extraction method characterized by implementing. The person extraction method according to claim 1 or 2,
In the background difference binarized image generation step, a color represented by the representative value of the block or a color of a representative pixel in the block is detected, and the block is converted to the first value or the color based on the color. The background extraction binarized image is generated by setting to a second value. A person extraction method according to any one of claims 1 to 3,
In the block representative value image generation step and the background difference binarized image generation step, a plurality of the representative value background images, the representative value mixed images, and the background difference binarized images having different block sizes are generated. And
In the block filter processing image generation step, the background difference of a block size in which the block of the first value is located within a preset range or a block size of the block of the first value closest to the range A person extraction method characterized by selecting a binarized image and performing processing based on the background difference binarized image. A background image in which a predetermined background is photographed is divided into blocks composed of a plurality of pixels, a representative value of values representing the color of each block in a predetermined color system is calculated, and the representative value of each block A representative value background image is generated, a mixed image in which the background of the background image and a person are photographed is divided into blocks composed of the plurality of pixels, the representative value of each block is calculated, and the representative value of each block Block representative value image generating means for generating a representative value mixed image composed of:
The representative value background image and the representative value mixed image are compared for each block, the block in which the difference between the representative values is equal to or greater than a predetermined value is set as a first image block, and the block is less than the predetermined value. A background difference binarized image generating means for generating a background difference binarized image by setting the second value as a background image block;
By setting the pixel set to the first value of the background difference binarized image to the color of the mixed image and setting the pixel set to the second value to a predetermined color, a block filter A block filter processed image generating means for generating a processed image;
A person extracting device characterized by comprising: The person extraction device according to claim 5,
Edge detection processing binarization generation image means for detecting a background and a human edge in the background difference binarized image and generating an edge detection processing binarized image indicating that the predetermined pixel is an edge or a non-edge; ,
The pixel determined as the edge of the edge detection processing binarized image is set as the color of the block filter processing image, and the pixel consisting of the first value determined as a non-edge is set as the color of the mixed image. Output image generation means for generating an output image by setting a pixel consisting of the second value determined as a non-edge to a predetermined background color;
A person extracting device characterized by comprising: A person extraction program that causes a computing means to execute the person extraction method according to claim 1. 8. A recording medium on which the person extraction program according to claim 7 is recorded so as to be readable by an arithmetic means.

Images