JPH04234284A - Moving image segmenting device - Google Patents

Abstract

PURPOSE:To automatically segment the closed up image of a speaker from image input by using a motion component to dispense with a camera operator required in a conventional video conference system. CONSTITUTION:The device is comprised of means 2, 3 which calculate frame difference for an input video signal, a means 4 which calculates a significant differential picture element in a frame differential result. a means 5 which calculates the histogram of only a block in a vertical direction decided in advance from the significant differential picture element, and a means 6 which detects a position from a histogram result.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera input position selection device for a video conference device.

0002

[Prior Art] In video conferences where multiple people participate at one site, the field of view of the camera is narrow, so a person called an operator moves the camera so that it targets the speaker, or moves the camera in front of each participant. There is a device that automatically selects the output of a camera installed to show each participant by determining the speaker based on the volume from the microphone placed in the conference room. Hereinafter, the former will be referred to as the operator method, and the latter will be referred to as the voice activation method. On the other hand, in the field of broadcasting, a high-definition television system called HDTV has already been developed. In the future, a teleconference system could be considered in which all participants in a video conference are photographed on an HDTV, and only the speaker is selected and displayed on a conventional NTSC television or workstation screen.

0003

[Problem to be solved by the invention] In the operator method,
If the person participating in the conference also serves as the operator,
In many cases, the person in question is distracted by moving the camera and is unable to fully participate in the conference, so a person who is not participating in the conference is designated as a dedicated operator. In this case, extra labor costs are required. On the other hand, the voice activation method requires multiple cameras and has problems such as malfunctions if a person who is not the speaker whispers to the next attendee or coughs. With the voice activation method, it is possible to use a device that uses only one camera to identify and extract the direction of the person from the sound volume, but since it is not possible to identify the speaker's position with sufficient accuracy from the voice, it is necessary to use a device that is determined in advance. It can only be applied to the specified person position. For example, it cannot be used when standing in front of a blackboard and speaking.

0004

SUMMARY OF THE INVENTION The present invention includes a first means for calculating a frame difference of an input video signal, a second means for calculating a significant difference pixel of the frame difference result, and a second means for calculating a significant difference pixel of the frame difference result. A moving image cutting device is provided (first invention), which is constituted by a third means for calculating a histogram only in a predetermined vertical section and a fourth means for detecting a position from the histogram result.

The present invention also provides a method in which the first means in the moving image clipping device is changed to a means for removing noise from the input video signal using correlation between frames and calculating a frame difference. A moving image cutting device characterized by the following features is provided (second invention).

Furthermore, the present invention provides a means for detecting significant pixel differences in a predetermined image area between the second means for detecting significant pixel differences and the third means for calculating a histogram in each of the moving image cutting devices. provides an image cutting device characterized in that a masking means is inserted (third invention).

[0007]

[Operation] The first invention provides a method that focuses on the movement of the person who is speaking and extracts the direction from the image. However, there are movements of people other than the speaker, and moving objects in the background (for example, the pendulum of a clock) are also perceived as movements in the image. Therefore, a means is provided for specifying the direction of a person from movement in a predetermined section in the vertical direction of the screen. Those who wish to speak should first raise their hands. Then, movement appears in, for example, the upper 1/4 section of the screen. The direction of the speaker is determined from the moving part, and a cutout window is set to allow a close-up of the speaker. If you try to identify the speaker's position from the movement of only the upper part, noise from the camera may become a problem. In order to solve this problem, the second invention attempts to apply the method of the first invention after removing noise using correlation between frames. In addition, there may be an object in the background that is in fixed motion, such as a pendulum on a clock. In this case, in order to avoid confusing the direction of the moving object with the direction of the speaker, the third invention is a method of performing a mask operation on a portion of the moving object that is known in advance.

[0008]

Embodiment FIG. 1 is a block diagram showing an embodiment of the first invention. In the figure, 1 is a video input terminal, 2 is a frame memory, 3 is a subtracter, 4 is a threshold circuit, 5 is a selective histogram calculation circuit, and 6 is a position detection circuit. A raster scanned video signal is input to the video input terminal 1. Not only the current NTSC video signal,
An HDTV signal may also be used. In the present invention, only the luminance component of the video signal is used, so the color component is not necessary. This signal is expressed as s(i, j, n). Here, i and j represent the horizontal position and vertical position of the screen, respectively, and n represents the frame. Therefore, the output of the frame memory 2 connected after the video input terminal 1 can be expressed as s(i, j, n-1). The subtracter 3 converts the frame memory 2 output s(i, j, n) from the signal s(i, j, n) input from the video input terminal 1.
j, n-1). Therefore, the output of subtractor 3 is s(
i, j, n)-s(i, j, n-1). Subtractor 3
A threshold circuit 4 connected to the output of the input signal outputs 0 when the input signal is smaller than a predetermined threshold Th, and outputs 1 when the input signal is larger than the predetermined threshold Th. The output is written as t(i, j, n),

[
Equation 1] becomes. A selective histogram calculation circuit 5 connected to the output of the threshold circuit 4 calculates a predetermined selection range w1, w.
The vertical histogram is calculated only when there is j between the two, in other words, only when w1<j<w2. If the histogram is written as h(i, n), then fo
r(J=w1;j<=w2;j++)h(i,n)+=
It can be written as t(i, j, n); However, h(
The initial values of i, n) are set to 0. Based on the output h(i, n) of the selective histogram calculation circuit 5, the position detection circuit 6 outputs X as a horizontal coordinate on the image in the direction of the person, where i, which has the largest histogram, is expressed as X. In the actual image cutting portion, a human portion can be extracted in a predetermined window size based on the horizontal coordinates and a predetermined vertical position. However, since the horizontal coordinate position X determined from the movement of the human part changes with time, it is integrated in the frame (time) direction,
Determined by the integral value over a certain period of time. It also has a function to fix it in a determined position. This fixation can be performed, for example, by the chairperson of the video conference by operating a button.

FIG. 2 is a block diagram showing an embodiment of the second invention. In the figure, 9 is a video input terminal, 2 is a frame memory, 3 and 10 are subtracters, 4 is a threshold circuit, 5 is a selective histogram calculation circuit, 6 is a position detection circuit, 7 is an adder, and 8 is a nonlinear ROM. be. Similar to the embodiment of the first invention, a raster-scanned video signal is input to the video input terminal 9. This is s(i,j,n)
It is written as. The input signal of frame memory 2 is x(i, j
, n), and the output signal is expressed as x (i, j, n-1). In the subtracter 3, the input signal from the video input terminal 1 is s (i,
j, n) to frame memory 2 output x(i, j, n-1
). Therefore, the output of the subtracter 3 is s(i, j, n
)-x(i, j, n-1). This signal is nonlinear R
Input to OM8. The nonlinear ROM 8 has a predetermined input/output function f(.). for example,

[Formula 2] The output is reduced in the range of -Th2<x<Th2. In the adder 7, f(s(i,j,n)
-x(i,j,n-1))+x(i,j,n-1). As a result, the adder 7 output x(i, j, n-
1) results in a signal with noise components between frames suppressed while large motion remains unchanged. Subtractor 1 for motion detection
0 is connected from before and after frame memory 2, and x(
i, j, n)-x(i, j, n-1) is calculated. If a noise removal circuit using frame correlation and a frame difference are connected in series, two frame memories are required.
This configuration is realized with only one frame memory. The output from the subtracter 10 onwards is the same as in the embodiment for the first invention.

FIG. 3 is a diagram showing an embodiment of the third invention. In the figure, 11 is a video input terminal, 3 and 1
0 is a subtracter, 4 is a threshold circuit, 5 is a selective histogram circuit, 6 is a position detection circuit, 7 is an adder, 8 is a nonlinear ROM
, 12 is a mask circuit. The operation from the video input terminal 11 to the threshold circuit 4 is the same as that in the embodiment of the second invention, and will therefore be omitted. The output of the threshold circuit 4 is the mask circuit 1
2 is input. The mask circuit has a memory for lines w1 to w2 selected by the selective histogram calculation circuit 5 at the subsequent stage, and a mask pattern is set in advance in the memory. For example, if there is a clock in the background, the pendulum part of the clock will always be detected as movement.
By setting the area corresponding to the pendulum to 0 and the other areas to 1 in this mask pattern, steady movement is masked in the output of the mask circuit 12. Therefore, only the movement caused by the person raising his hand or the movement caused by the person standing up and giving an explanation can be supplied to the input of the selective histogram calculation circuit 5 at the subsequent stage. The operations of the selective histogram calculation circuit 5 and the position detection circuit 6 are also the same as in the embodiment of the first invention.

[0011]

[Effects of the Invention] By using the first invention, the direction of the person speaking can be extracted from an image based on the movement of the person speaking, and a close-up of the person speaking can be extracted.

By using the second invention, in addition to the first invention, it is possible to extract a stable movement of a person that is resistant to noise from a television camera, and it is possible to cut out a close-up of the speaker. .

[0013] By using the third invention, in addition to the first and second inventions, even if there is a fixed moving object in the background, it is possible to remove it and detect the movement of the speaker. You can take a close-up of the image.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment of the first invention.

FIG. 2 is a block diagram showing an embodiment of the second invention.

FIG. 3 is a block diagram showing an embodiment of the third invention.

[Explanation of symbols]

1, 9, 11 Video input terminal 2 Frame memory 3, 10 Subtractor 4 Threshold circuit 5 Selective histogram calculation circuit 6 Position detection circuit 7, 10 Adder 8 Nonlinear circuit 12 Mask circuit

Claims (3)

[Claims] 1. A first means for calculating a frame difference of an input video signal, a second means for calculating a significant difference pixel of the frame difference result, and a predetermined vertical interval from the significant difference pixel. 3rd to calculate the histogram only
and a fourth means for detecting a position from a histogram result. 2. A moving image cutout characterized in that the first means according to claim 1 is changed to a means for removing noise from an input video signal using correlation between frames and calculating a frame difference. Device. 3. Between the second means for detecting significant pixel differences and the third means for calculating a histogram according to claims 1 and 2, significant pixel differences in a predetermined image area are detected. A moving image cutting device characterized in that a masking means is inserted.