JP2000197022A - Image segmenting device and video telephone system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To segment a more clear person by removing a worm-bore state and noise caused in the case of segmenting the person image from a background. SOLUTION: A background reference image without including the person is recorded in a memory 2 prior to communication, the image picked by an image pickup device 1 is compared with the background reference image by a binarization processing circuit 3 during communication, a comparison result image is expressed by binarization, a filter processing is executed by a filter processing circuit 4, in order to remove the noise of the image processed by binarization and the white data part of a binarized image obtained by the filter processing is replaced with a communication image, so that a transmission image which is in the worm-bored state in the figure or from which background noise is removed is created.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image clipping device and a videophone device, and more particularly to a device for performing a person clipping process on an image photographed by using an image pickup device, and a communication device performing such a person clipping process. The present invention relates to a videophone device that performs

[0002]

2. Description of the Related Art A videophone device incorporates an imaging device and a microphone or telephone.
Alternatively, it is used by connecting an imaging device to a microphone or a telephone, and performs mutual communication by video and audio. In this videophone device,
For example, as in JP-A-7-250312, a background reference image and an image during videophone communication (hereinafter, referred to as a communication image)
A method is proposed in which a comparison process is performed to cut out a person image and a background image, and information about only the person portion is transmitted to the other party.

[0003]

However, according to the above-mentioned conventional technology, it is possible to cut out a person part neatly because a person image becomes wormy or noise occurs in a background portion other than a person. There was a problem that it was not possible. SUMMARY OF THE INVENTION The present invention has been made in view of the above-described conventional problems, and has been made in consideration of the above-described problem. The purpose of the present invention is to provide a videophone device that performs the following.

[0004]

According to an aspect of the present invention, there is provided an image cutout apparatus which divides an image photographed by an image pickup device into a background portion and a person portion, and only a person portion. In the image clipping device that creates the information of the above, a background reference image that does not include a person and that is recorded in advance and a person image that includes a person in the reference background are compared for each pixel or for each predetermined block. A binary processing circuit that outputs first data that is either black data or white data for an equal portion, and outputs second data that is the other of the black data or white data for a different portion. A filter processing circuit for performing filter processing for removing noise from the binarization processing output of the binarization processing circuit; and a binarization processing obtained from the filter processing circuit. An image replacement circuit for creating an image of a person portion in which the background portion is either black data or white data by replacing the second data portion with the communication image for the output. It is a feature.

[0005] According to a second aspect of the present invention, there is provided an image clipping apparatus.
2. The image clipping device according to claim 1, wherein the binarization processing circuit compares the background reference image and the human image with respect to both or one of a luminance signal and a color difference signal. is there.

According to a third aspect of the present invention, there is provided an image extracting apparatus comprising:
2. The image clipping device according to claim 1, wherein the filter processing circuit outputs the data of the binarization processing output to be subjected to the filtering, and outputs one of the first data and the second data to the surrounding binarization processing output. The majority is determined by majority processing.

[0007] This makes it possible to create an image of only a person from which the worm-eating state of the person and the background noise have been removed.

According to a fourth aspect of the present invention, there is provided a video telephone apparatus for dividing an image photographed by an imaging device into a background portion and a person portion and transmitting information of only the person portion. A background reference image that does not include a person and a communication image that is an image during videophone communication are compared in advance for each pixel or for each predetermined block. A binarization processing circuit for outputting the first data which is one of the two types of data, and the second data which is different from the second data which is the other of the black data and the white data; A filtering circuit for performing a filtering process for removing the noise from the binarized processing output, and adding a second data portion to the binarized processing output obtained from the filtering processing circuit. By replacing the communication image,
An image replacement circuit for creating a transmission image of a person portion in which a background portion is either black data or white data.

According to a fifth aspect of the present invention, in the video telephone according to the fourth aspect, the binarization processing circuit converts the background reference image and the person image into a luminance signal and a color difference signal. Both or one of them is to be compared.

According to a sixth aspect of the present invention, in the videophone apparatus according to the fourth aspect, the filter processing circuit converts the data of the binarization processing output to be subjected to the filtering to the surrounding binarization data. The decision is made by majority processing to determine which of the first data and the second data is larger in the processing output. As a result, it is possible to create a transmission image from which the worm eating state of the person and the background noise have been removed.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An image cutout apparatus according to the present invention separates an image photographed by an image pickup apparatus into a background portion and a person portion, and records the image in advance in an image cutout device for creating information of only a person portion. The background reference image not including the person and the person image, which is the image including the person in the reference background, are compared for each pixel or for each predetermined block. One of the first data is different from the second data which is the other of the black data or the white data.
, A filter processing circuit for removing the noise from the binarization processing output of the binarization processing circuit, and the binarization obtained from the filter processing circuit. An image replacement circuit that replaces the second data portion with the above-mentioned person image for the processing output, and creates an image of the person portion consisting of only the person portion and the background portion being black data or white data. This has the effect of removing the worm-eating state of the person and the noise of the background, and creating a more beautiful person image.

Further, a videophone device according to the present invention comprises:
The image taken by the imaging device, the background portion and the person portion, the video phone device transmits only the information of the person portion, in the background reference image that does not include the person recorded before communication, A communication image, which is an image at the time of videophone communication, is compared for each pixel or for each predetermined block, and an equal portion is black data or white data of the first data, and a different portion is black data A binarization processing circuit for outputting second data which is the other of the data and the white data, a filter processing circuit for removing noise from the binarization processing output of the binarization processing circuit, 2 obtained from the filter processing circuit
An image replacement circuit that replaces the second data portion with the communication image for the output of the binarization process and creates a transmission image consisting of only the human portion and the background portion being either black data or white data. It is characterized by having the function of removing the worm-eating state of a person and noise of the background, and transmitting a more beautiful person image to the other party.

(Embodiment 1) An image clipping device and a videophone device according to Embodiment 1 of the present invention will be described below with reference to FIG. FIG. 1 is a circuit configuration diagram of a videophone device according to Embodiment 1 of the present invention. This videophone device includes the image clipping device according to Embodiment 1 of the present invention.

In FIG. 1, reference numeral 1 denotes an image pickup device for photographing a background reference image containing no person, or photographing an image during communication. Reference numeral 2 denotes a memory for recording an image previously captured by the imaging device 1 before communication. Reference numeral 3 denotes a binarization processing circuit which converts a background reference image recorded in the memory 2 and an image being communicated from the imaging device 1 during communication, for each pixel or each predetermined block, into a luminance signal and a color difference signal. Comparison is performed for each, and if they are equal, “0” (black data) is output, and if they are different, “1” (white data) is output to generate binary image data.

Reference numeral 4 denotes a filter processing circuit for filtering the binary image created by the binarization processing circuit 3 to remove the worm-like state of a person and background noise. A detailed method of this filter processing will be described later.

FIG. 2 is a diagram showing a processed image in the case where a person is cut out only by comparing the background reference image and the image during communication according to the conventional method. That is, in FIG. 2, 8 is a background reference image stored in the memory 2 before communication, 9 is a communication image captured by the imaging device 1 during communication, and a person who is talking to the image is include. Reference numeral 10 denotes a comparison result image based on the background reference image and the communicating image. In the conventional method, the comparison result image, that is, the transmission image, simply performs comparison between the background reference image and the image during communication, so that noise and the like appear in both the person portion and the background portion. In this case, the person part is in the worm-eating state 10a or the noise part 10b is inserted in the background part, so that it is impossible to cut out a beautiful person.

Referring back to FIG. 1, reference numeral 5 denotes an image replacement circuit.
The white data portion of the filtered binary image is
This is a circuit that is replaced with communication image data from the imaging device 1. Through this image replacement process, a beautiful transmission image of only the person whose background portion has been replaced with black data ("0") can be created. Reference numeral 6 denotes a video codec, which compresses the transmission image created in the process up to the image replacement circuit 5 and transmits the compressed image to the other party. Reference numeral 7 denotes a buffer which holds a communication image to be replaced when the white data portion of the binary image, for which the processing up to the image replacement circuit 5 has been completed, is replaced with an in-communication image until the image replacement processing is completed. It is for.

The image pickup device 1, the memory 2, the binarization processing circuit 3, the filter processing circuit 4, the image replacement circuit 5, and the buffer 7 constitute an image cutout device according to the first embodiment of the present invention. The one to which the codec 6 is added corresponds to the videophone device according to the first embodiment of the present invention.

Next, the details of each of the above processing steps will be described. FIG. 3 is an explanatory diagram illustrating a binarized image according to the first embodiment. In FIG. 3, 11 is
This is a binary image output by the binarization processing circuit 3 in FIG. 1, which compares a background reference image 8 and a communication image 9 with both a luminance signal and a color difference signal for each pixel or for each predetermined block. Then, if they are equal, black data is output, and if they are different, white data is output, and the comparison result image is a binary image. In this binary image, a person is wormy as shown by 12 in the figure, and noise is inserted into the background as shown by 13 in the figure, and the image is never beautiful.
Not a value image.

FIG. 4 is an explanatory diagram showing a filtered image in the first embodiment. In FIG. 4, 14 is
3 is a binary image obtained by processing the binary image 11 of FIG. 3 by the filter processing circuit 4 of FIG. For the filter processing in the filter processing circuit 4, a majority filter is used. The majority filter performs filter processing for each predetermined block, for example, in the first embodiment, using a matrix having a size of 3 × 3.

FIG. 5 is an explanatory diagram showing the filtering process by the majority filter in the first embodiment. In this majority filter processing, the data of the central pixel of the matrix is determined based on the number of surrounding white data and black data. For example, in a 3 × 3 matrix such as pixel data 15,
The center pixel of the matrix (white data in this example)
, There are five pixels of white data and three pixels of black data, so that the center pixel remains white data like pixel data 16 in the filter processing result.

In a 3 × 3 matrix such as the pixel data 17, there are three pixels of white data and five pixels of black data around the center pixel. Converted to black data.

However, it seems that there are many cases as follows. In a 3 × 3 matrix such as the pixel data 19, there are four pixels of white data and four pixels of black data in the periphery, so that the data of the central pixel cannot be determined. In this case, the data of the central pixel is used as it is. Seems to be desirable. This is because the worm-eating state of a person and the background noise often exist in a single pixel, and the number of surrounding data is almost equal at the boundary between the person and the background. This is because it is desirable to keep the pixel value as it is if the image has an appropriate boundary.

It is impossible to determine that there is a 3 × 3 matrix centering on the pixel at the screen edge, and it is therefore impossible to select a matrix at the screen edge. In this way, for pixels having no data, processing is not included in the number of pixels for which majority processing is performed. Then, such processing is sequentially performed from the upper left of the screen. However, the processing order does not have to be from the upper left, and the processing may be started from an arbitrary pixel. Next, the image replacement circuit 5 performs a process of replacing the pixel information of the white data of the binary image output from the filter processing circuit 4 with the image during communication.

FIG. 6 is an explanatory diagram showing an image replacement processing image according to the first embodiment. In FIG.
This is a transmission image output by the image replacement circuit 5. Through the processing by the various processing circuits described above, it is possible to create a transmission image in which only the person 21 is cut out.
This transmission image 21 is compressed by the video codec 6 shown in FIG. 1 and transmitted to the other party. On the receiving side,
By replacing the pixel portion of the black data with a desired image, a videophone can be enjoyed.

As described above, according to the first embodiment, the background reference image of only the background is captured in advance by the imaging device and stored in the memory, and the image including the person is included in the background reference image. Take this and the original background reference image,
Both the luminance signal and the color difference signal are compared, a binary process is performed in which equal parts are set to black data, and different parts are set to white data. By performing a filtering process to remove and replacing the white data portion after the filtering process with an image including a person, a person is cut out, and a cut-out image of a person with a black background is transmitted to the other party. It is possible to transmit a cut-out image of a beautiful person without noise or worms to the other party.

According to the first embodiment, the binarization process is performed by comparing the image including only the background with the image including the person for both the luminance signal and the color difference signal. However, if there is no problem in performing the binarization processing, this may be performed for only one of the luminance signal and the color difference signal. In this case, the configuration of the binarization processing circuit is simplified. it can.

Further, according to the first embodiment, an image including only a background is compared with an image including a person, black data is output for an equal portion, white data is output for a different portion, and white data is output. By replacing the part with an image containing a person, the image data of the person part whose background part was black data was created, but white data was output for equal parts and black data was output for different parts. By replacing the black data portion with an image including a person, it is also possible to create image data of a person portion whose background portion is white data.

Further, according to the first embodiment, a video telephone apparatus in which only a person portion is cut out from a background portion and processed by a video codec and transmitted to the other party has been described as an example. May be used as an image clipping device for clipping the image.In this case, the output is taken into a personal computer or the like to perform image synthesis processing such as replacing the background of the image of the person. It is possible to obtain images at high speed.

[0030]

As described above, according to the image clipping device according to the first aspect of the present invention, an image captured by the imaging device is divided into a background portion and a person portion, and information on only the person portion is obtained. In the image clipping device to be created,
A previously recorded background reference image without a person and a person image with a person in the reference background are compared for each pixel or for each predetermined block. Output the first data which is one of the two data, and output the second data which is the other of the black data and the white data in a different portion.
A binarization processing circuit, a filter processing circuit for performing a filtering process for removing noise from the binarization processing output of the binarization processing circuit, and a binarization circuit obtained from the filtering processing circuit.
An image replacement circuit for replacing the second data portion with the communication image with respect to the value processing output to create an image of a person portion having a background portion of either black data or white data; By replacing the above image, the worm-eating state of the person and the noise of the background are removed, so that a more beautiful human cutout image can be created without the worm-eating state of the person or noise occurring in the background The effect that can be obtained is obtained.

According to the image clipping device of the present invention, in the image clipping device according to the first aspect, the binarization processing circuit converts the background reference image and the person image into two. Since the luminance signal and / or the color difference signal are compared, it is possible to create a clearer cutout image of the person without causing the worm eating state or noise in the background. Since the processing based on the luminance signal and the chrominance signal is used as the binarization processing for determining the portion to be clipped, the effect that the apparatus can be easily realized is obtained.

Further, according to the image clipping device of the present invention, in the image clipping device according to claim 1, the filter processing circuit converts the data of the binarization processing output to be filtered into It is determined by majority processing whether which of the first data and the second data is larger in the binarization processing output around it, so that a person may be wormy or noise may occur in the background. It is possible to create a clearer cut-out image of a person without the need to use it, and to use a process based on a majority decision as a filter process for removing the noise, so that an effect that the device can be easily realized is obtained. .

Further, according to the videophone device of the invention of claim 4 of the present application, the image photographed by the imaging device is
In a videophone device that separates a background portion and a person portion and transmits information of only the person portion, a background reference image that does not include a person and that is recorded in advance before communication and an image during videophone communication are used. When a certain communication image is compared with each pixel or every predetermined block, the equal part is the first data which is either the black data or the white data, and the different part is the other of the black data or the white data. A binarization processing circuit for respectively outputting certain second data, a filter processing circuit for performing filter processing for removing noise from the binarization processing output of the binarization processing circuit, and the filter processing circuit By replacing the second data portion with the communication image in the binarization processing output obtained from the above, the background portion becomes either black data or white data. An image replacement circuit that creates a transmission image of the person portion, and the image is replaced to remove the worm-eating state of the person and the noise of the background. This makes it possible to create a more beautiful cut-out image of a person without causing the occurrence of an image, and to transmit the image to the other party.

Further, according to the videophone device of the present invention, in the videophone device according to the fourth aspect, the binarization processing circuit converts the background reference image and the communication image to Since both or one of the luminance signal and the color difference signal were compared, a clearer cut-out image of the person was created without generating a worm-like state on the person or generating noise on the background,
Since the processing based on the luminance signal and the chrominance signal is used as the binarization processing that can be transmitted to the other party and determines the portion to be cut out, an effect that the apparatus can be easily realized is obtained.

Further, according to the videophone device of the present invention, in the videophone device according to the fourth aspect, the filter processing circuit converts the data of the binarization processing output to be filtered to Since the binarization processing output around that is determined by majority decision processing as to which of the first data and the second data is larger, an worm-eating state occurs in the person, and noise occurs in the background. It is possible to create a clearer cutout image of the person without sending it out, send it to the other party, and use a process based on majority decision as a filter process to remove the noise, so that the device can be easily constructed The effect that can be realized is obtained.

[Brief description of the drawings]

FIG. 1 is a circuit configuration diagram of a videophone device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a cut-out image of a person in the related art.

FIG. 3 is a block diagram of the videophone device according to the first embodiment.
Explanatory diagram showing a binarized image

FIG. 4 is an explanatory diagram showing a filtered image in the videophone device of the first embodiment.

FIG. 5 is an explanatory diagram showing filter processing by a majority filter according to the first embodiment.

FIG. 6 is an explanatory diagram showing an image replacement processing image in the videophone device of the first embodiment.

[Explanation of symbols]

REFERENCE SIGNS LIST 1 imaging device 2 memory 3 binarization processing circuit 4 filter processing circuit 5 image replacement circuit 6 video codec 7 buffer 8 background reference image 9 communicating image 10 comparison result image in prior art 11 binarization processing image 12 human worm 13 Background noise 14 Majority filter processed image 15, 17, 19 Pixel data (3 × 3 matrix before majority filter processing) 16, 18, 20 Pixel data (3 × 3 matrix after majority filter processing) 21 After image replacement processing Transmission image

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Taiichi Iwanaga 8-1, Koshincho, Takamatsu City, Kagawa Prefecture Inside Matsushita Hisashi Denshi Kogyo Co., Ltd. (72) Inventor Junichi Kita 1 of 8 Koshinmachi, Takamatsu City, Kagawa Prefecture Hisashi Matsushita Inside Electronic Industries Co., Ltd. (72) Inventor Mitsumasa Shirada 8-1, Koshinmachi, Takamatsu City, Kagawa Prefecture Matsushita Hisashi Denshi Kogyo Co., Ltd. F-term (reference) 5C064 AA01 AC02 AC08

Claims (6)

[Claims] An image clipping device for dividing an image captured by an image pickup device into a background portion and a person portion to create information of only a person portion, wherein a background reference without a person recorded in advance is stored. An image and a person image in which a person is included in the reference background are compared for each pixel or for each predetermined block, and an equal portion is different from the first data, which is either black data or white data, differently. The output portion outputs the second data, which is the other of the black data and the white data, respectively.
A binarization processing circuit, a filter processing circuit for performing a filtering process for removing noise from the binarization processing output of the binarization processing circuit, and a binarization processing output obtained from the filter processing circuit. On the other hand, an image replacement circuit for creating an image of a person portion in which the background portion has become either black data or white data by replacing the second data portion with the communication image, Image cropping device. 2. The image clipping device according to claim 1, wherein the binarization processing circuit compares the background reference image and the human image with respect to both or one of a luminance signal and a color difference signal. An image clipping device, characterized in that: 3. The image clipping device according to claim 1, wherein the filter processing circuit converts the data of the binarization processing output to be filtered to the first data and the second data to the surrounding binarization processing outputs. An image clipping device, which is determined by majority processing of which of the above data is larger. 4. A videophone device which divides an image photographed by an imaging device into a background portion and a person portion and transmits information of only the person portion, wherein a person who has recorded in advance before communication contains a person. No background reference image and a communication image that is an image during videophone communication are compared for each pixel or for each predetermined block,
A binarization processing circuit that outputs first data that is either one of black data or white data for an equal part, and second data that is the other of the black data or white data for a different part; A filter processing circuit for performing filter processing for removing noise from the binarization processing output of the binarization processing circuit; and a binarization processing output obtained from the filter processing circuit, An image replacement circuit for creating a transmission image of a person portion in which a background portion becomes either black data or white data by replacing a portion with the communication image. 5. The videophone device according to claim 4, wherein the binarization processing circuit compares the background reference image and the communication image for both or one of a luminance signal and a color difference signal. A videophone device. 6. The videophone device according to claim 4, wherein the filter processing circuit converts the data of the binarization processing output to be subjected to the filter processing into first and second binarization processing outputs. The videophone device is determined by a majority decision process as to which of the data is larger.