JPH04321396A - Video image synthesizer by television telephone set - Google Patents

Abstract

PURPOSE:To monitor 3rd information such as a character pattern and a still picture at the preset priority. CONSTITUTION:A received video signal from an opposite party is inputted to a multiplex processing section 29 as a main video picture element signal and the sent video signal is subject to reduction processing and the result is inputted to a memory section 19, read by a memory control section 25 in response to the mapping position in the display pattern and inputted to the multiplex processing section 29 as a sub video picture element signal, a character pattern is outputted from a character pattern generating section 26 synchronously with the received video image and inputted to the multiplex processing section 29. A predetermined switching signal is generated by a switching signal generating section 31 in response to the three input states of the multiplex processing section 29 and the multiplex processing section 29 is controlled by the switching signal and one of the three input states is selected and outputted.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video compositing device used in a video telephone device to combine and display a received video signal from the other party, a transmitted video signal, etc. on the same display screen.

0002

2. Description of the Related Art FIG. 7A shows an image synthesizing section in a conventional video telephone device or the like. Video synthesis in a call between terminal A and terminal B is performed as follows. A video signal of a speaker on the terminal A side is input from the video input unit (camera) 11 on the terminal A side, the video signal is video encoded in the encoding unit 12, and is sent to the terminal B side via the transmission unit 13. sent to.

On the other hand, video data sent from terminal B and received by transmission section 13 is decoded by decoding section 14, and the decoded video signal is sent to video input section 1 by video synthesis section 15.
The speaker video signal on the terminal A side input from 1 is combined,
The image is supplied to the video output section 16 and displayed. As shown in FIG. 8, the video synthesis unit controls the video signal so that the video signals of the speaker on the terminal B side and the video of the speaker on the terminal A side constitute one frame so that the speaker video on the terminal B side and the speaker video on the terminal A side are displayed in combination on the same display screen. Synthesize in step 15.

[0004] By synthesizing such images,
In addition to viewing the other party's video, the speaker's own video is also displayed, making it possible to check, for example, whether the speaker is correctly positioned in front of the camera, and to check the content and quality of the transmitted video. can. Here, the conventional configuration of the video synthesis section 15 is as shown in FIG. 7B. The input signals are two video inputs: a main video signal that is a received video signal and a sub video signal that is a transmitted video signal.
In step 7, reduction processing is performed. The reduction processing unit 17 reduces the size of the sub-video in the vertical and horizontal directions on the TV screen.
Pixel thinning in the horizontal line direction and line thinning in the vertical direction are performed. The reduced sub-picture signal is sent to the inversion processing section 18, where the left and right sides of the television screen display image are inverted as necessary. Normally, when self-monitoring the image input by a camera, the user sees an image reflected in a mirror, but by performing inversion processing, it is possible to self-monitor the same image that the other party sees. For this reversal process, either reversal or non-reversal can be selected as necessary.

[0005] In order to be able to synthesize the reduced sub-picture signal at any position on the television screen, the reduced sub-picture signal is temporarily stored in a memory section. The stored sub-picture signal is read out from the memory section 19 in accordance with a read signal from the memory control section 21, and is combined with the main video signal in the multiplexing section 22. This multiplexing process is performed in response to read output from the memory unit 19.

[0006]

[Problems to be Solved by the Invention] Recently, studies have been progressing to use video telephone devices to hold conferences over long distances, and the importance of transmitting meeting materials in addition to speakers is increasing. In this case, the problem is which video should be displayed on the screen.

[0007]

[Means for Solving the Problems] According to the present invention, a third information signal such as a character pattern information signal or a still image information signal,
A third information generator is provided that generates data in synchronization with the received video signal, and a predetermined switching is performed based on the input state of the third information signal and the input of the reduced transmitted video signal. A signal is generated from a switching signal generating section, the multiplexing processing section is controlled by the switching signal, and one of the received video signal, the transmitted video signal read from the memory, and the third information signal is switched and output. Ru.

[0008] By simply changing the relationship between the input state of each signal in the switching signal generating section and the switching signal, the priority of displayed images and how they are combined within one screen can be changed without changing the configuration of the multiplexing processing section. You can change the type of video and the order of overlay.

[0009]

[Embodiment] An embodiment of the present invention is shown in FIG. The input selection unit 23 selectively inputs two video signals from a plurality of input video signals based on an instruction of an input selection command. For example, the main video and sub-video targets are selected and input from among the transmitted video to be sent to the other party, the received video received from the other party's terminal, the own terminal input video other than the transmitted video, etc.

[0010] In the reduction processing section 17, reduction processing for generating a sub-picture is performed on the selectively inputted sub-picture signal. In order to reduce the size of the TV screen in the vertical and horizontal directions, the reduction processing unit 17 thins out pixels in the horizontal line direction of the input signal (horizontal subsample processing) and thins out lines in the vertical direction (vertical subsample processing). I do. The reduction processing unit 17 may include a filter that performs band limitation before thinning out in order to remove Nyquist folding frequency components at the time of sub-sampling. There are two types of band-limiting filters: one for thinning out pixels in the horizontal line direction (horizontal filter) and one for thinning out pixels in the vertical direction (vertical filter). In addition, these filter processes are performed on the Y signal (
This is performed for each of the luminance signal), Cb, and Cr signals (two types of color difference signals). Sub-sample processing is performed in accordance with the display size of the sub-video. For example, if the display size of the sub video is 1/3 of the screen size of the main video in both the vertical and horizontal directions, then in the horizontal direction, every 3 pixels will be thinned out by 2 pixels, and in the vertical direction, every 3 lines will be thinned out by 2 lines. do.

The output of the reduction processing section 17 is supplied to the inversion processing section 18. The inversion processing unit 18 is composed of two buffer memories that store data for one horizontal line, and reverses the writing order of data input to the buffer memory and the order of reading data from the buffer memory for each line. Read and output with the last number at the beginning. When one of these two buffer memories is in a writing state, the other is in a reading state, and by repeating such a writing/reading relationship line by line, the inversion processing unit 18 An inverted video signal with a delay of one line is continuously output. Normally, when self-monitoring the video input from a camera, you look at the image reflected in a mirror, but by performing inversion processing, you can self-monitor the same image that the other party sees, and the transmitted video It is effective for checking etc. Note that the inversion processing section 24 can similarly perform inversion processing on the main video signal as necessary. The reversal processing units 18 and 24 each have a different reversal designation/
Performs reversal processing or does not perform reversal processing in response to a release command. In the inverted state, no writing is performed to the buffer memory, and the signal is simply passed through the bypass.

The output of the inversion processing section 18 is stored in a memory section 19. This stored video signal is usually one television frame worth, and is read out from the memory section 19 in accordance with a read signal from the memory control section 25. FIG. 2 shows the positional relationship between the main video and the sub-video on the display screen. The starting point of the television frame is defined as the origin (0,0) using the main video screen as a coordinate system. At this time, the pixel number in the horizontal direction is the coordinate value X of the horizontal axis, and the line number is the coordinate value Y in the vertical direction. Therefore, the parameter set of the window that maps the sub-video onto the frame is set as {(X0, Y0
), LX, LY} (where (X0, Y0) is the origin of the window, and LX, LY are the width and height of the window, respectively), the memory control unit 25 performs the following for each line in the horizontal direction. X coordinate value X in synchronization with the sampling clock
and compare it with X0, X0+LX, and in the vertical direction, count the Y coordinate value Y in synchronization with the line clock and calculate Y
0,Y0+LY, and the comparison result is the following two conditions X0
If ≦X≦X0+LX and Y0≦Y≦Y0+LY are satisfied at the same time, data is read from the memory unit 19. Here, the reading order corresponds to the writing order to the memory unit 19, and the reading speed matches the pixel frequency (sampling clock frequency) of the main video.

In this embodiment, a character pattern generating section 26 is provided as the third information generating section, and the character pattern generating section 26 outputs a binary pattern (0, 1 signal) in the direction of one horizontal line of the character. Characters are formed by repeating this pattern generation for multiple lines. This character pattern is the phase processing unit 2
7. The phase processing unit 27 is composed of a buffer memory that stores data for one horizontal line, and delays the character pattern signal by the phase difference between the main video line synchronization and the character pattern line synchronization so that the phases of the line synchronization and the line synchronization of the character pattern match. . The output of the phase processing section 27 is input to the color control section 28, and the color control section 28 outputs a character pixel signal consisting of the luminance value and color difference value of the character in synchronization with the input. The character pattern generation section 26 is comprised of a character generator, etc., and therefore can improve character quality compared to characters input directly with a camera.

The main video signal, sub-video signal, and character pixel signal are input to a multiplexing processing section 29 . Multiplexing processing unit 2
Reference numeral 9 denotes a switch circuit which selects and outputs one of the input signals based on a switching signal to synthesize a video signal. The input state of the sub-video signal and the character pixel signal is detected by the switching signal generating section 31, and a preset switching signal is generated based on the detection result, and the multiplexing processing section 29 is switched by this switching signal.

Multiplexing processing section 29 and switching signal generation section 31
is configured as shown in FIG. 3, for example. Multiplexing processing unit 2
Reference numeral 9 denotes a switch SW1 to which a character pixel signal is supplied, a switch SW2 to which a main video pixel signal is supplied, a switch SW3 to which a sub-picture pixel signal is supplied, and these switches SW1, SW.
It is composed of an OR gate 32 that outputs each output signal of W2 and SW3 using OR logic. Switch SW1, SW2,
SW3 is controlled on and off by switching signals S1, S2, and S3, respectively. Possible switching signals S1, S2, S3
There are three combinations of "0" and "1" as shown below.

{S1, S2, S3}={1,0,0},
{0, 1, 0}, {0, 0, 1} That is, for example, if switch SW1 is connected (on), switches SW2 and SW3 are in the open state (off), and among the three switches SW1 to SW3, One switch is always closed, and only the pixel signal from the closed switch is outputted via the OR gate 32.

The switching signal generating section 31 has a state pattern memory (which stores, as a state pattern, the presence or absence of a signal input to the multiplexing processing section 29 and the relationship between the switches to be opened and closed in the multiplexing processing section 29 according to the input state). ROM) 33, and a switching pulse generation circuit 34 that converts the output from the memory 33 into switching signals S1, S2, and S3. FIG. 4A shows an example of a state pattern of a switch to be opened or closed, which is stored in the state pattern memory 33. The signals input to the switching signal generator 31 are the character pattern signal M and the reading timing F.
and the sampling clock S, the character pattern signal M is synchronized with the character pixel signal, and when M=1 (0), the multiplexing processing unit 2
Indicates whether or not there is a character pixel signal input to 9. The reading timing F is synchronized with the sub-picture pixel signal input to the multiplexing processing section 29, and F=1 (0) indicates that the sub-picture pixel signal is input to the multiplexing processing section 29 (not present). The sampling clock S is synchronized with the main video pixel signal, and S=1 at all times. As a precondition for setting the state shown in FIG. 4A, the priority order of pixel signals to be output from the multiplexing processing unit 29 is set in the order of character pixel signal, sub-picture pixel signal, and main video pixel signal, and Prioritize poses. If the speaker does not want to display character pattern information, the character pattern generation section 26 outputs the character pattern when the speaker instructs the speaker to do so. Therefore, there are four output states for the input states of character pattern signal M, reading timing F, and sampling clock S as shown in FIG. 4A, and these four states are shown as switching designation results M0, F0, and S0. The result M0 is “1”
”, if F0 is “1” and S0 is “1”, the output results of the multiplexing processing unit 29 will be a character pixel signal, a sub-picture pixel signal, and a main picture pixel signal.In addition, the switching designation result M0
, F0, S0 and the output of the switching pulse generation circuit 34 is as shown in the timing chart of FIG. 4B. In other words, when M0 is “1”, S1 is “1”, and when F0 is “1”, S
2 becomes "1", S0 becomes "1", and S3 becomes "1".

Note that the main video signal and the sub video signal are multiplexed line by line, and when repeated for one frame, a main/sub composite screen is constructed. The position of the window can be changed as a parameter set, and it can be displayed at any position on the television frame. Further, the character pattern may be displayed on the entire screen, or may be displayed in a window together with the main video (received video).

[0019] In the video synthesis device according to the present invention, the third
An example of using still image information as information will be shown. That is, as shown in FIG. 5, instead of the character pixel signal generation section in FIG.
A still image memory section 35 and a still image memory control section 36 are provided, and the other configurations are the same as the embodiment shown in FIG. The still image stored in the still image memory section 35 is, for example, a buffered sub-picture pixel signal for one television frame that has been subjected to reduction processing, and the signal read out from the memory section 19 is stored in the still image memory section 35. . Still image memory control unit 36
controls the position of the window that maps the still image on the display surface. This window parameter is {(X′0,
Y'0), LX, LY} (here, (X'0, Y'0)
is the origin of the window), and the window width and height LX, LY are the same as the window specified value for the sub-video.

In the still image memory control unit 36, in the horizontal direction, the X coordinate value X is counted for each line in synchronization with the sampling clock and compared with X'0, X'0+LX, and in the vertical direction, the X coordinate value The Y coordinate value Y is counted in synchronization with the clock and compared with Y'0, Y'0+LY, and the comparison results meet the following two conditions: X'0≦X≦X'0+LX Y'0≦Y≦Y'0+LY If both conditions are satisfied, data is read from the still image memory section 35. Here, the reading speed is made to match the pixel frequency (sampling clock frequency) of the main image. Multiplexing processing section 29 and switching signal generation section 31 in this embodiment
is shown in FIG. 6 is basically the same as the configuration shown in FIG. 3, but in this example, a variable state pattern memory (RAM) 37 that can freely set the state pattern portion of the state pattern memory 33 of the switching signal generator 31 is used. This is the case. A sampling clock, still image reading timing, and sub-picture reading timing are input to the switching signal generating section 31, and based on these input states, a switching signal for synthesizing the main video, sub-picture, and still image is generated. . Such a switch SW1 of the switching signal generation section 31 and the multiplexing processing section 29
, SW2, and SW3 are performed in the same manner as in the case of superimposition shown in the embodiment of FIG. In addition, a variable state pattern memory (RAM) 3 of the switching signal generating section 31
The content change in step 7 is performed using an overlay designation signal, which defines the priority order when main video, sub video, and still images are combined on the display screen.

The signal format of the video signal input/output to the video synthesis apparatus shown in FIGS.
c. If the format is in accordance with the 601 Recommendation, it will be compatible with the standardized videophone video encoding system, and since the video signal processing is digital, it will be of high quality with no noise or the like. For still images, the input selection section 23 is selectively controlled, and still images from various still image sources such as VTRs, video discs, video memories, video cameras, and image processing devices including computers are processed by the reduction processing section 17 or inversion processing. The signal is input to the still image memory section 35 as a sub-picture pixel signal or a main picture pixel signal. The still image may also be displayed to fill the entire screen as necessary, and even if it is displayed in a reduced size, it does not have to be the same size as the sub-image. In addition to the video selection described above, the input selection unit 33 also selects the video of the speaker among the multiple parties in the case of a video conference for the received video.

[0022]

[Effects of the Invention] As explained above, according to the present invention,
In a videophone, it is possible to display not only received video and transmitted video, but also 3rd information such as text data and still images. A predetermined switching signal is generated from the switching signal generation section, and the multiplexing processing section is switched and synthesized using the switching signal, so multiple inputs can be displayed according to preset priorities. , the order can be changed by simply changing the switching signal generating section, and there is no need to change the multiplexing processing section. In particular, when the state pattern memory is configured with RAM as shown in FIG. 5, the priority order can be easily changed as necessary.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment of the invention.

FIG. 2 is a diagram showing an example of the format of a window on a display screen.

FIG. 3 is a block diagram showing a specific example of the multiplexing processing section 29 and the switching signal generation section 31 in FIG. 1.

4A is a diagram showing a storage example of the state pattern memory 33 in FIG. 3, and B is a time chart showing an example of the relationship between a switching designation result and a switching signal.

FIG. 5 is a block diagram showing another embodiment of the invention.

6 is a block diagram showing a specific example of the multiplexing processing section 29 and the switching signal generation section 31 in FIG. 5. FIG.

[Fig. 7] A block diagram showing images, etc. of a conventional videophone,
B is a block diagram showing a conventional video synthesis section 15.

FIG. 8 is a diagram showing a window format on a display surface.

Claims (3)

[Claims] 1. A reduction processing unit reduces a transmission video signal, stores the reduction-processed transmission video signal in a memory, and a memory control unit performs a display on which the transmission video signal subjected to the reduction processing is to be displayed. The video phone video synthesis device reads out the reduced transmitted video signal from the memory using the window position information on the screen, and switches and synthesizes the received video signal in the multiplexing processing unit, in synchronization with the received video signal. a third information generating section that generates a third information signal; a predetermined switching signal based on the states of the input of the third information signal and the input of the reduced transmission video signal; and a switching signal generating unit that generates a switching signal, and the switching signal generator is controlled by the switching signal to switch and output any one of the received video signal, the transmitted video signal read from the memory, and the third information signal. 1. A video phone video synthesis device, characterized in that it is provided with a multiplexing processing section. 2. The video telephone video composition apparatus according to claim 1, wherein the third information generating section is a character pattern generating section that generates a character pattern information signal as the third information signal. 3. The video telephone video synthesis apparatus according to claim 1, wherein the third information generating section is a still image information generating section that generates a still image information signal as the third information signal.