JPH0591407A - Video communication equipment - Google Patents

Abstract

PURPOSE:To delete a background part with satisfactory precision by detecting a video part having a small moving extent as the background part of a subject out of a video signal and then coding the video signal which backs the video signal of the background part. CONSTITUTION:The common intermediate format CIF signal (c) outputted from a format converter circuit 3 is sent to a background detecting parts 9 and a background deleting part 10 respectively. The part 10 decides a video part having a small moving extent as the background part of a subject with the relative comparison of video signals between the contiguous frames among those video signals photographed by a camera 1. Then this deciding result o12 is sent to the part 10. The part 10 deletes the video signal of a background part detected by the part 10 out of the CIF signal and sends the signals of only the subject to a video coding circuit 4 and also to the part 9 as the signal of the precedent frame. The circuit 4 codes and transmits the signals of only the subject leaking its background.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video communication device suitable for application to a TV telephone device or a TV conference device, and more specifically, a video communication device in which a background portion is removed from a video image captured by a camera for transmission. It relates to the device.

[0002]

2. Description of the Related Art Conventionally, as a video communication apparatus of this type, T
There are V-telephone devices and video conferencing devices.

FIG. 8 is a block diagram showing a general configuration of a conventional TV telephone apparatus, which includes a camera 1, an A / D conversion circuit 2, a format conversion circuit 3, a video coding circuit 4, and a demultiplexing circuit 5. , A format inverse conversion circuit 6, a D / A conversion circuit 7, and a monitor 8.

In this structure, first, an image of a person, a landscape, a calligraphy or the like taken by the camera 1 is input to the A / D conversion circuit 2 as an analog image signal a and converted into a digital image signal b here. To be done. Then, the digital video signal b is transmitted to the T by the format conversion circuit 3.
A common intermediate format used in the video encoding circuit 4 for the V telephone device and the TV conference device.
rmat: hereinafter, abbreviated as CIF signal) converted to a signal c. Then, the CIF signal c is coded by the video coding circuit 4 and sent to the demultiplexing circuit 5 as a video coded signal dl. The demultiplexing circuit 5 multiplexes the coded information j of the audio signal with the video coded signal dl to form the multiplexed signal e, and sends it to the transmission line.

On the other hand, when the multiplexed signal e is received from the other device, the demultiplexing circuit 5 receives the received multiplexed signal e.
Is separated into a received video coded signal d2 and a received voice coded signal j, the received video coded signal d2 is input to the video coding circuit 4, and the received voice coded signal j is input to a voice circuit (not shown). The video encoding circuit D4 converts the received video encoding d2 into CI.
The F signal f is decoded and input to the format inverse conversion circuit 6. The format reverse conversion circuit 6 converts the CIF signal f into a digital video signal g and inputs it to the D / A conversion circuit 7. The D / A conversion circuit 7 converts the digital video signal g into an analog video signal h and reproduces it as a received image on the screen of the monitor 8.

Although the case where the CIF signal is used in the video encoding circuit 4 has been described, the 1/4 CIF signal may be used in some cases.

[0007]

However, in such a conventional configuration, since the video signal picked up by the camera 1 is encoded and transmitted as it is, it is related to the communication contents such as the background of the person image during communication. This means that the video part that is not encoded is also encoded and transmitted. Therefore, when the TV telephone device is used at home or office, the situation inside the home or office is known to the other party, and there is a problem that it is unsatisfactory in terms of protecting personal privacy and confidentiality in the office. It was Further, as the number of coding bits allocated to the video information portion of the person, which is originally necessary, is reduced, there is a problem that improvement in video quality cannot be expected.

In this case, as a method of removing the background portion, the IEICE Spring National Convention (1989) D-
101 "A Study on Video Communication Using Background Separation"
As shown in FIG. 1, there is a method of measuring the distance to the subject using two cameras and removing those having a distance equal to or more than a certain distance as the background. However, this method has a problem that it is difficult to obtain a target point for calculating the distance, many erroneous determinations are made, and the background removal accuracy is poor.

The object of the present invention is to protect the privacy in the home and the confidentiality in the office, and further, to remove the background portion with sufficient accuracy to improve the quality of the transmitted video signal. Is to provide.

[0010]

In order to achieve the above object, the present invention is directed to a video portion having a small amount of movement by subjecting a video signal captured by a camera to a relative comparison of video signals between adjacent frames. Background detection section for detecting the background section of the background, a background removal section for removing the video signal of the background section detected by the background detection section, and an encoding for encoding and transmitting only the video signal of the subject from which the background section is removed. And a circuit.

[0011]

According to the above-mentioned means, a video portion having a small amount of motion is detected as a background portion of a subject by relative comparison of video signals between adjacent frames among video signals captured by a camera, and the background portion of this background portion is detected. Since the video signal of only the subject from which the video signal has been removed is encoded and transmitted, the privacy of the home and the confidentiality of the office are protected, and further, the background portion is removed with sufficient accuracy, and the quality of the video signal to be transmitted. Can be improved.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to illustrated embodiments.

FIG. 1 is a block diagram showing a first embodiment of the present invention, and FIG. 2 is an explanatory diagram showing a concept of a background portion which is not related to communication contents.

First, the concept of the background portion will be described with reference to FIG.

In general, the person who is the object related to the communication content is located immediately in front of the camera, and its movement becomes large during the communication. Therefore, it is possible to supplement a region with motion, define that region as a part related to communication, and define a region with little motion as a background part. For example, as shown in FIG. 2A, a person K having a communication symmetry has a larger amount of movement than a stationary object such as a painting M or a flowerpot L existing in the background, and a stationary object such as the painting M has a movement amount. Is close to zero. Therefore,
The relative comparison of the video signals between the adjacent n-1th frame and the next nth frame is performed, and it is assumed that the part with a large motion vector is the main part in communication and the part with a small motion vector is If the background portion is detected and the background portion is removed, the person K having the communication symmetry as shown in FIG.
Only can be extracted.

Such processing can be realized by a signal processing method called "motion compensation system" generally used in video encoding circuits. What is this motion compensation method?
Specifically, the video signal is divided into blocks composed of a plurality of images, and for one block of the current frame, the correlation between the position of the block in the previous frame and the block in the vicinity position is measured, and the correlation is detected in the previous frame. It is determined that the block of the current frame has moved from the position of the largest block. As shown in FIG. 2B, the amount of movement is large in a part of the person, and the arrows indicating that amount are concentrated. Therefore, by tracking this area, it is possible to detect the person in communication.

Next, referring to FIG. 1 using the definition of the background portion of FIG.
An example will be described.

In FIG. 1, the camera 1, the A / D conversion circuit 2, the format conversion circuit 3, and the video coding circuit 4 have the same configuration as the conventional one, and therefore detailed description thereof will be omitted.
In FIG. 1, the background detection unit 9 is composed of a motion compensation circuit 90 and a background detection circuit 91, and the background removal unit 10 is
It is composed of a delay circuit 100, a background data memory 101, a background removal circuit 102, and a frame memory 103.

In this configuration, first, the CIF signal c output from the format conversion circuit 3 is the motion compensation circuit 9
0 and sent to the delay circuit 100. The motion compensation circuit 90 obtains a correlation value in block units between the input CIF signal c and the CIF signal of the previous frame stored in the frame memory 103, and the motion vector o11 is a vector between the blocks having the highest correlation. To generate. The motion vector o11 is sent to the background detection circuit 91, and whether the background is a background or not is determined in block units depending on whether or not the amount of motion from the previous frame is large, and the determination result o12 is sent to the background removal circuit 102. Be done.

On the other hand, in the delay circuit 100, the delayed CIF signal p11 delayed by the period in which the background detection processing is executed by the motion compensation circuit 90 and the background detection circuit 91 is generated, and then the background data memory 101 is stored. Background data p1
The block-based CIF signal synchronized with 2 is sent to the background removal circuit 102. In the background removal circuit 102, when the input delayed CIF signal p11 and the background data p12 are determined to be the background portion according to the determination result o12 of the background detection circuit 91, the background data p12 is output.
, The delayed CIF signal p
The determination output p13 that outputs 11 is the frame memory 10
Send to 3.

In the frame memory 103, the input from the background removal circuit 102 is accumulated for one frame and sent as a background removal video signal p14 which is coded by the subsequent video coding circuit 4, and at the same time the motion compensation circuit 90 described above. As the signal of the previous frame used, the CIF signal of the previous frame is sent to the motion compensation circuit 90 in accordance with the input of the next frame.

According to the present embodiment, for a block which is determined to have a large image motion and is not the background part, the image signal input from the camera 1 is directly encoded and transmitted, and the motion is small and the background part is not detected. For the determined block, a signal preset as background data is encoded and transmitted. That is, the painting K and the flowerpot L are removed from the image of the subject and the background as shown in FIG.
Only the person K is transmitted as an image as shown in FIG.

Therefore, it is possible to reduce the number of coded bits in the background part and increase the number of coded bits in the video part to be communicated, and to improve the image quality of the video part to be communicated. be able to.

In addition, in the TV telephone device and the like, only the person image to be communicated is transmitted to the partner device, and the background portion thereof is not transmitted. As a result, it is possible to conceal things that the other party does not want to be known, such as the state of the household, and protect privacy.

FIG. 4 is a block diagram showing a second embodiment of the present invention. The difference from FIG. 1 is that the background detecting section 9 is provided with a motion judging circuit 92 and a motion judging memory 93. This is that the delay amount of the delay circuit 100 in the background removal unit P2 is changed according to the background detection unit 9.

In the embodiment shown in FIG. 1, a block having a small motion is determined to be the background on a block-by-block basis as a result of motion compensation. However, in such a configuration, the motion compensation method is applied to the portion that should originally be the background. Due to the erroneous determination of the amount of movement due to
A background block may occur independently in a portion that should be a communication target. The situation is shown in FIG. Figure 5
In (a), the block determined to be the background is indicated by "0", and the block determined not to be the background is indicated by "1".
In order to remove the above-mentioned isolated block, FIG.
The isolation removal process using the isolation removal filter 94 as shown in (b) is effective. This isolation removal process is shown in FIG.
FIG. 5 (c) shows the one applied to. As is clear from FIG. 5C, it can be seen that the isolated blocks are reduced by the isolated removal process of FIG.

In the configuration of FIG. 4, the motion compensator 9
The motion vector o21 is measured at 0, and the motion determination circuit 92
Sent to. In the motion determination circuit 92, the motion amount is compared with a predetermined value, and a block determined to have a small motion amount and a background portion is set to “0”, and a block determined to have a large motion amount and not a background is set to “1”. The motion determination result o22 is obtained,
It is stored in the motion determination memory 93. After that, the background detection circuit 91 sets the value o23 in the motion determination memory 93 to the value o23 in FIG.
The isolation removal processing is performed using the isolation removal filter 94 shown in (4) and is sent to the background removal circuit 102 as the determination result o24 in block units.

According to this embodiment, "0" is shown in FIG.
Since the isolated background block indicated by is removed, the received image quality on the receiving side is improved. In addition, the background portion can be efficiently removed with only one camera.

FIG. 6 is a block diagram showing a third embodiment of the present invention. This embodiment is largely different from the previous embodiment in that two cameras 1A and 1B are provided and a stereo image is inputted. different. As shown in FIG. 7, the cameras 1A and 1B are arranged so that the optical axes intersect at a position where a communication target exists. Then, the analog video signal a1 output from the camera 1A is input to the background detection unit 9 via the A / D conversion circuit 2A and the format conversion circuit 3A, where the same processing as in FIG. 1 is performed, The background portion is determined.

On the other hand, the analog video signal a2 output from the camera 2B is input to the parallax compensation circuit 12 via the A / D conversion circuit 2B and the format conversion circuit 3B, where the parallax similar to the motion compensation method is used. The compensation process is performed. That is, the analog video signal a2 is sent to the parallax compensation circuit 11, where the correlation with the analog video signal a1 is obtained in block units. The correlation value q31 is sent to the parallax vector calculation circuit 12, and the parallax vector q32 is obtained. The background determination result o32 and the parallax vector q32 for the analog video signal a1 from the camera 1 are sent to the background removal circuit 102, and the background determination result o32 indicates a large movement,
When the parallax vector q32 is larger than a predetermined value among the blocks determined not to be the background portion, the determination result of the block is changed to the background portion, and the other processing is performed in the same manner as in FIG.

Therefore, in this embodiment, even in the area where the motion is large, the one having a small parallax vector between the video signals input from the two cameras 1A and 1B is regarded as the background and removed. This is because when a person or the like passes by in the background in the middle of communication, it is determined that the background is not the background in the embodiment of FIGS. 1 and 4, and the image is encoded as it is. In the example, by using the feature that the parallax vector is small for an object that is far from the camera, even if a moving object enters the part that is originally the background, this part is removed as the background. It

Since the other circuit parts operate in the same manner as in the embodiment of FIG. 1, detailed description thereof will be omitted.

In each of the above embodiments, the motion amount based on the block-based motion compensation method is used in the background detection unit for background detection, but it is also possible to use other motion amount detection methods. For example, a pixel-based motion compensation method in which a block is shifted pixel by pixel may be used, and whether or not a pixel is a background portion may be used for background detection. Further, also with respect to the isolated removal processing described with reference to FIG. 5, another method such as a majority decision processing of eight pixels in the peripheral determination result, an AND processing in units of two pixels, an isolated point removal method by an OR processing may be used. ..

[0034]

As described above, according to the present invention, it is possible to remove a background portion unrelated to communication with sufficient accuracy and then encode and transmit the same. Therefore, a TV telephone device or a TV conference device can be used at home or in the office. Even when used internally, it is possible to protect the privacy of individuals and confidentiality in the office.
Furthermore, since the video information of the background portion can be reduced, the number of coding bits assigned to the target video signal portion originally related to the communication content increases, and the video quality can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a definition of a background portion.

FIG. 3 is an explanatory diagram showing an image with a background portion removed.

FIG. 4 is a block diagram showing a second embodiment of the present invention.

FIG. 5 is an explanatory diagram illustrating an isolated block removal process of a background detection result.

FIG. 6 is a block diagram showing a third embodiment of the present invention.

FIG. 7 is an explanatory diagram showing a positional relationship between two cameras in FIG.

FIG. 8 is a configuration diagram of a TV phone device which is an example of a conventional video communication device.

[Explanation of symbols]

1 ... Camera, 2 ... A / D conversion circuit, 3 ... Format conversion circuit, 4 ... Video coding circuit, 5 ... Multiplexing / demultiplexing circuit, 6 ...
Format inverse conversion circuit, 7 ... D / A conversion circuit, 8 ... Monitor, 9 ... Background detection unit, 10 ... Background removal unit, 11 ... Parallax compensation circuit, 12 ... Parallax vector calculation circuit, 90 ... Motion compensation circuit, 91 ... Background detection circuit, 92 ... Motion determination circuit, 1
02 ... Background removal circuit, 103 ... Frame memory.

Claims (1)

[Claims] 1. A video communication apparatus for encoding and transmitting a video signal of an object photographed by a camera, wherein the amount of motion is small by relative comparison of video signals between adjacent frames among the video signals photographed by the camera. A background detection unit that detects the video portion as the background portion of the subject, a background removal unit that removes the video signal of the background portion detected by the background detection unit, and a video signal of only the subject from which the background portion has been removed is encoded. A video communication device, comprising: an encoding circuit for transmitting.