JPH06178295A - Picture signal processing unit for video conference and utterance party pattern mgnification synthesis device - Google Patents

Abstract

PURPOSE:To simplify the setting of a field angle and to switch the display of an utterance party picture smoothly by calling out data in a picture frame set in response to the input from an utterance party discrimination device, magnifying the data and synthesizing the magnified data to an original entire picture. CONSTITUTION:The device is provided with a picture signal processing unit 4 which extracts an optional picture range designated by an operation device 3 in advance as a closeup frame and moves it to other picture position, magnifies it and synthesizes the magnified signal to an original picture and with an utterance party discrimination device 5 which automatically discriminates an utterance party through voice detection to call out the set closeup frame. In this case, a preset frame of a picture frame designating to close up each of conference participants is designated in an optional size and position and stored. For example, when a conference participant makes utterance, the utterance party discrimination device 5 calls one of preset frames and the picture segmented by the preset frame is magnified and inserted and synthesized at a designated position in an original full scene picture and the result is displayed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video conference image signal processing device and a speaker screen enlargement / synthesis device used in a video conference system.

[0002]

2. Description of the Related Art Conventionally, in a video conference in which a plurality of people participate, an electric swivel type TV camera having a camera swivel for moving an image pick-up field and a zoom lens having a variable image pick-up range is used for picking up an image of a speaker. Was. In order to get a close-up of each participant, it was necessary to operate the swivel vertically and horizontally, to make the zoom lens wide-angle or telephoto, and to adjust the focus. Further, for the purpose of omitting those operations, a method is used in which optimum positions of a swivel base and a zoom lens for a close-up screen of each participant's image are stored in advance and preset control is performed. Furthermore, a voice automatic tracking function for automatically determining the speaker by detecting voice and performing preset control is also adopted.

FIG. 4 is a block diagram showing the configuration of a camera control device of a conventional video conference system. In FIG. 4, a microphone 2 is provided to each of the conference participants A, B, and C.
1-1 to 21-3 and voice detectors 22-1 to 22-3, a television camera 23 having a swivel base 24 and a zoom lens 25, an operation device 26, and a camera swivel base control device 27.
And a speaker determination device 28. First,
In order to set the close-up screens of the participants A, B, and C, the operation device 26 is operated to control the swivel base 24 and the zoom lens 25, and the three positions are stored. Here, the respective setting positions are Pa (screen of A), P
These are referred to as b (screen of B) and Pc (screen of C).
The operation unit 26 is provided with a single operation key for calling these positions and performing preset control of the television camera. For example, by pressing the Pa calling key, the swivel base 24 and the zoom lens 25 are operated, and A is closed up. When B speaks, the voice detector 22-2 responds to the voice and outputs a voice detection signal to the speaker determination device 28, and the speaker determination device 28 controls the camera swivel base to call Pb accordingly. The Pb calling signal is output to the device 27. Then, the swivel base 24 and the zoom lens 25 are moved to the Pb position, and B is closed up. Similarly, C is closed up when the Pc call key of the operation remote controller is pressed or when C speaks and voice is detected.

[0004]

However, in the above-mentioned conventional camera control device, the operation of the swivel base and the zoom lens for setting each close-up position is complicated, and in order to obtain a desired image pickup screen. I had to use various operation keys. In addition, it takes some time for the swivel base and the zoom lens to move to the preset position, and there is a problem in that followability to key operations and speech is poor.

The present invention solves the above-mentioned conventional problems by performing image signal processing without using the control of the swivel or the zoom lens, but only by using the image from the television camera that constantly images the entire conference participants. By obtaining each participant's close-up screen, enlarging it, and synthesizing it in the original overall image, the angle of view setting is simplified, the screen call operation and the voice of the speaker are well followed, and the speaker smoothly It is an object of the present invention to provide an excellent speaker conference screen enlarging / synthesizing device for video conference, which enables switching of screen display.

[0006]

In order to achieve this object, the present invention provides a TV camera for photographing the entire conference participants, a monitor TV for displaying images, and a closing of each conference participant from the panoramic screen. An operation device for setting and calling the up screen range and an arbitrary specified screen range are extracted as a close-up frame, and the screen frame is moved to another screen position and enlarged to synthesize the original screen. And an image signal processing device that operates.

[0007]

With this configuration, the camera is fixed so that the entire participants of the conference are photographed, and the close-up screen frame of each participant is set and stored from the display screen, so that the swivel base can be operated as in the conventional case. You can quickly call up the close-up screen without having to operate the or zoom lens. Further, by setting arbitrary plural screen frames within the imaging range of one TV camera, the close-up screen frame of the speaker can be enlarged and combined and displayed on the original panoramic image.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. 1 is a block diagram showing the configuration of an embodiment of the present invention, FIG. 2 is a diagram for explaining the image processing of the embodiment, and (a) is an image of the entire conference participants,
(B) is a diagram showing a close-up screen frame of each conference participant,
(C) is a diagram showing a screen in which a close-up image of a speaker is enlarged and combined with an image of the entire conference participants.
FIG. 3 shows a digital signal processor for video (hereinafter referred to as DS) in an image signal processing apparatus according to an embodiment of the present invention.
FIG. 6 is a diagram for explaining a pixel interpolation operation when enlarging a preset frame screen of a portion (denoted as P).

In FIG. 1, 1 is a television camera for photographing the entire conference scene, 2 is a monitor television for displaying images, and 3 is a panoramic screen for setting the close-up screen range of each conference participant. Or, the operation device 4 for calling up each person's close-up screen extracts an arbitrary screen range specified by the operation device 3 in advance as a close-up frame, moves it to another screen position, and enlarges it. The image signal processing device 5 for synthesizing on the screen is a speaker determination device for automatically determining the speaker by detecting the voice and calling the set close-up frame. The video signal output of the television camera 1 is input to the image signal processing device 4, and after the image signal processing, the monitor television 2
And output to the transmission line. Both the operation signal of the operation device 3 and the preset calling signal of the speaker determination device 5 are input to the image signal processing device 4. The image signal processing device 4 includes a video signal decoder 401, a pulse generator 402, an A / D converter 403, a line memory 404, a video digital signal processor 405, and an external RAM thereof.
406, a character generator ROM 407,
A D / A converter 408, a video signal encoder 409,
Keyboard interface 410 and CPU 411
It consists of and.

The operation of the speaker screen enlarging / synthesizing device configured as described above will be described with reference to FIGS. 1 and 2.

In FIG. 2, (a) is a panoramic screen imaged by the television camera 1, showing an input video signal from the television camera 1 to the image signal processing device 4 in FIG.
(B) is a screen showing a frame specified to close up each participant in the panoramic screen, and the close-up frame specified by the operation device 3 in FIG. 1 is indicated by a line. (C)
Is a screen obtained by enlarging a designated close-up frame screen and combining it with the original screen, which is an output video signal from the image signal processing device 4 to the monitor television 2 in FIG. Figure 2
In (a), the conference participants are A, B, and C, respectively,
In (b), the screen frames designated to close up A, B, and C respectively are designated P1 and P, respectively.
2, P3. These preset frames can be specified in any size and at any position and are stored. For example, when A speaks during the conference, the speaker determination device 5 in FIG. 1 calls P1 from the preset frames P1 to P3, and when B speaks, P2 is called and the preset frame is used. The cropped screen is enlarged, and the original panoramic screen is inset and synthesized at a specified position, as shown in FIG.
The screen display as described above is the operation of one embodiment of the present invention.

Next, the operation of the image signal processing device 4 will be described. The video signal input of the NTSC composite video signal input from the television camera 1 is demodulated by the video signal decoder 401 into an RGB signal, digitized by the A / D converter 403, and then double-speed scan converted by the next line memory 404 (interlace → Non-interlaced scan conversion) is performed, and the data is captured by the video DSP 405 as sampling data. Further, the video signal decoder 401 performs sync separation from the video input signal, and a horizontal sync signal, a vertical sync signal,
A burst signal or the like is generated and output to the pulse generator 402. The pulse generator 402 is a circuit that generates the timing necessary for this device from the separated sync signal,
4fsc (fsc is a color subcarrier frequency) that is the sampling clock of the D converter 403 and the writing clock of the line memory 404, 8fsc that is the reading clock of the line memory 404 and the sampling clock of the video DSP 405, the control signal of the line memory 404, etc. To supply. The RGB signal digitally digitally sampled by the non-interlaced scanning is used for the video DSP 405.
Is stored in the external RAM 406 for each frame and processed by digital image signal processing described later,
After the characters in the character generator ROM 407 are superimposed, the D / A converter 408 is again used.
2 is modulated by the video signal encoder 409 as shown in FIG.
It is output as a screen image as shown in (c). The clock of the D / A converter 408 and the synchronizing signal of the video signal encoder 409 are supplied from the pulse generator 402. The control data from the operation device 3 is processed by the CPU 411 via the keyboard interface 410. CPU411
Is a memory address necessary for constructing a speaker screen or a screen during preset frame setting operation as shown in FIG. 2 by a preset frame calling signal or a preset frame setting processing signal from the speaker determination device 5 or the controller 3. Memory control data such as character information is sent to the video DSP 405. The video DSP 405 performs an image signal processing operation described below according to the control data from the CPU 411.

FIG. 3 is a diagram for explaining the pixel interpolation operation when the preset frame screen is enlarged. First, the operation of the video DSP 405 when calling the set preset frame will be described with reference to FIG. First, the pixels within the selected preset frame are extracted as original data. This is to extract the sampling data stored in the memory addresses corresponding to the line numbers (row numbers) at the upper and lower ends of the preset frame and the sampling data numbers (column numbers) at the left and right ends given from the CPU 411 when setting the preset frame from the area. It is done by Next, the extracted pixel data in the preset frame is expanded in order to enlarge it on the screen. For simplicity, let us consider the case of double expansion. At this time, with the decompression method in which the data of the previous sample and the data of the previous line are simply interpolated as they are, the image quality becomes rough only in the preset frame when displayed on the screen. Therefore, the data to be interpolated is as shown in FIG. For, the average value of the raw data of the sample before and after, the original data of the line before and after is used. The pixel data in the preset frame expanded in this way is stored in the RAM.
The data is written in the combined screen memory area on the sampled data 406 where the sample data of the same frame exists. By reading and reproducing the data of that frame, the speaker screen of the enlarged and extracted preset frame is displayed at a position corresponding to the designated unnecessary portion (lower center in the example of FIG. 2) on the panoramic screen. become. The video signal output subjected to the image signal processing as described above is displayed on the monitor and transmitted to the conference terminal of the other party in communication in the video conference.

Since the output image of the television camera 1 is partially enlarged, a high resolution one is desirable.

[0015]

As described above, according to the present invention, a video signal decoder, a pulse generator, an A / D converter, a line memory, a video digital signal processor, its external RAM, and a character.・ Generator ROM,
D / A converter, video signal encoder, keyboard
A screen frame in which an interface and a CPU are connected to each other so as to perform a predetermined operation and provided in a circuit, and a personal image of each participant in the conference is closed up from the video input of the entire landscape of the video conference captured by the video camera. A function to set and store, and a function to call the data in the screen frame set according to the input from the speaker determination device, enlarge the image in the screen frame, synthesize it with the original whole screen, and output the image With the above, it is possible to display a plurality of speaker screens from a video signal of one fixed type television camera without using a preset type electrically conductive swivel base or a zoom lens as in the past, that is, without performing a complicated camera operation. It can be set with a simple operation that only encloses it in a frame, and since it is instant from when the image is called to when the speaker is displayed on the screen, the conventional remote control operation and the operation after voice detection are performed. In which can realize the video conference image signal processing device and the speaker screen magnifier synthesizing apparatus excellent no conformability of poor due to the operation time of the base lens.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

[FIG. 2] FIG. 2A is a diagram showing an image of the entire conference participants. FIG. 2B is a diagram showing a close-up screen frame of each conference participant. Figure showing the enlarged and combined screen of the close-up image

FIG. 3 is a diagram illustrating a pixel interpolation operation when enlarging a preset frame screen.

FIG. 4 is a block diagram showing a configuration of a camera control device of a conventional video conference system.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 TV camera 2 Monitor TV 3 Operation device 4 Image signal processing device 5 Speaker determination device 401 Video signal decoder 402 Pulse generator 403 A / D converter 404 Line memory 405 Video digital signal processor 406 RAM 407 Character generator ROM 408 D / A converter 409 Video signal encoder 410 Keyboard interface 411 CPU

Claims (2)

[Claims] 1. A video signal decoder, a pulse generator, and A
/ D converter, line memory, video digital signal processor, its external RAM, character generator ROM, D / A converter, video signal encoder, keyboard interface, CP
It is equipped with U by connecting to U so as to have a predetermined operation,
A function to set and store a screen frame that closes up individual images of each participant in the conference from the video input of the entire scene of the video conference captured by the TV camera, and a screen set according to the input from the speaker determination device. Call the data in the frame,
An image signal processing apparatus for a video conference, comprising a function of enlarging an image in the screen frame, synthesizing the original whole screen, and outputting the image. 2. An overall view of a video conference imaged by the video camera, comprising the image signal processing device for video conference according to claim 1, a TV camera, a monitor TV, an operating unit, and a speaker judging device. From the video input of, the screen frame that closes up the individual images of the conference participants in advance is set and stored by the operation device, and the image of the speaker's image frame specified by the speaker determination device is enlarged and the original full view is displayed. An apparatus for enlarging and composing a speaker screen for a video conference, which is characterized by synthesizing with an image and outputting the image.