JPH0738868A - Terminal equipment for video conference - Google Patents

Abstract

PURPOSE:To record the speech and image of a communication opposite party. CONSTITUTION:A level detecting circuit 22 detects the speech level of the speech from the opposite party. It is recognized from the detected speech level which opposite party the speech is received from at present. The speech and image of the recognized opposite party whose speech is being received are recorded on a sound recording and reproducing device 24 and a video recording and reproducing device 34. When no communication is made, the speech and image recorded on the sound recording and reproducing device 24 and video recording and reproducing device are reproduced.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a terminal device in a video conference system for communicating multimedia (image, voice, data, etc.) with a plurality of parties via a communication line.

[0002]

2. Description of the Related Art Conventionally, a video conference system performs multimedia communication by multiplexing images, voice and data with a plurality of parties via a communication line, and mixes voices received from a plurality of parties into one. As shown in FIG. 3, it is configured to output to a speaker, display images received from a plurality of partners on a multi-window, and record and reproduce voices and images received from the plurality of partners. .

[0003]

However, in the conventional example,
Since the voices and images of all the other parties are recorded, the amount of information to be recorded becomes enormous, and there is a problem that the point of the storage device is insufficient.

An object of the present invention is to provide a video conference terminal device that solves such problems.

[0005]

A video conference terminal device according to the present invention is currently used in a video conference system for performing multimedia communication with a plurality of partners via a communication line, by detecting the voice level of the partner. A recognition means for recognizing from whom the voice is received, a recording means for recording the voice and image of the other party recognized as receiving the voice, and a reproduction means for reproducing the recorded voice and image. It is characterized by being provided.

[0006]

By means of the above means, only the voice and image of the other party receiving the voice need be recorded, so the amount of information to be recorded is reduced. Therefore, the capacity of the storage device can be saved.

[0007]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 shows a schematic block diagram of an embodiment of the present invention. 10 is a CPU for controlling the entire apparatus, 12
Is an RO that stores programs executed by the CPU 10.
M and 14 are R for storing data used by the CPU 10.
AM, 16 is an operating device including a keyboard and a touch panel for operating the device, and 18 is a TTU-T
(Former CCITT) Recommendation G. A voice coding / decoding circuit for performing coding and decoding of a voice signal according to 722;
Is a microphone for voice input, 22 is a level detection circuit for detecting the level of the received voice, 24 is a recording / playback device composed of a magnetic disk for recording and playing back the received voice, and 26 is one of the voices of a plurality of received parties Is a voice mixing circuit for mixing with, and 28 is a speaker for outputting voice.

Numeral 30 is TTU-T Recommendation H.264. An image encoding / decoding circuit that executes encoding and decoding of an image signal in accordance with H.261, 32 is a camera for inputting an image, 34 is a recording / reproducing apparatus including a magnetic disk for recording and reproducing the received image, and 36 is a plurality of devices. , A display control circuit for displaying the image of the other party on a multi-window, 38 is an LCD for displaying the image
Alternatively, the display device is a CRT or the like.

40 is CCITT recommendation H.264. A demultiplexing circuit that multiplexes and separates images, voices, and data according to 221 and a line control circuit 42 that controls connection and communication with a communication line.

FIG. 2 shows the structure of the audio frame supplied from the audio encoding / decoding circuit 18 to the audio mixing circuit 26. Voice signals from a plurality of partners are time-division multiplexed, and the voice signals of each partner are stored in time slots. In FIG. 2, 44 is a time slot for storing the voice signal of the partner # 1, 46 is a time slot for storing the voice signal of the partner # 2, 48 is a time slot for storing the voice signal of the partner # 3, and 50 is a partner #. 4 is a time slot for storing 4 audio signals. Figure 2 shows the case where there are four people,
The time slot 50 is followed by a time slot with the same repetition.

FIG. 3 shows the structure of an image frame supplied from the image encoding / decoding circuit 30 to the display control circuit 36. The image signals of a plurality of partners are time-division multiplexed, and the image signals of each partner are stored in time slots. In FIG. 3, reference numeral 52 is a time slot for storing the image signal of the partner # 1, 54 is a time slot for storing the image signal of the partner # 2, 56 is a time slot for storing the image signal of the partner # 3, and 58 is a partner #. 4 is a time slot for storing the image signal of No. 4. FIG. 3 shows a case where there are four opponents, and the time slot 58 is followed by a time slot with the same repetition.

FIG. 4 shows a display device 38 displayed during communication.
An example of the screen of is shown. In FIG. 4, reference numeral 60 denotes the display device 3.
8 is a screen, 62 is a window for displaying an image received from the other party # 1 in the screen 60, 64 is a window for displaying an image received from the other party # 2 in the screen 60, and 66 is a other party in the screen 60 A window for displaying the image received from # 3 and a window 68 for displaying the image received from the partner # 4 on the screen 60. Reference numeral 70 denotes a recording start menu for instructing the start of recording with the mouse of the operation device 16,
Reference numeral 72 denotes a recording stop menu for instructing to stop recording with the mouse of the operation device 16. FIG. 4 shows a case where there are four opponents.

FIG. 5 shows the structure of an audio frame supplied from the recording / reproducing device 24 to the audio mixing circuit 26 during reproduction. The reproduced voice is continuously stored in the time slot. 74, 76, 78 and 80 are time slots in which reproduced audio is stored.

FIG. 6 shows the structure of an image frame supplied from the recording / reproducing device 34 to the display control circuit 36 during reproduction. The reproduced images are continuously stored in the time slot. The 82, 84, 86, and 88th are time slots in which reproduced images are stored.

FIG. 7 shows a display device 38 displayed during reproduction.
An example of the screen of is shown. In FIG. 7, 90 is a reproduction window for displaying a reproduced image, 92 is a reproduction start menu for instructing the start of reproduction by the mouse of the operation device 16, and
Reference numeral 94 is a reproduction stop menu for instructing stop of reproduction by the mouse of the operation device 16. The reproduction window 90 is displayed only during reproduction. Further, the reproduction start menu 92 and the reproduction stop menu 94 are displayed except during communication.

FIG. 8 shows the memory configuration of the RAM 14.
100 is a variable N for storing the number of opponents, 102 is a variable I for storing the number of the opponent, 104 is a variable J for storing the number of the opponent having the highest voice level, and 106 is a variable for storing the voice level of the opponent I. V and 108 are variables X for storing the highest sound level.

FIG. 9 shows a flow chart of the recording program stored in the ROM 12. At the start of communication, the number of the other party is set in the variable N, the recording / reproducing device 24 is initialized, and the voice mixing circuit 26 is instructed to mix the voices for the number of persons indicated by the variable N and output the mixed voice to the speaker 28. It is assumed that the display control circuit 36 is instructed to display the images for the number of persons indicated by the variable N on the display device 38.

In addition, this program, during communication, clicks (instructs) the recording start menu 70 with the mouse of the operation device 16 and then clicks the recording stop menu 72 or ends the communication until the audio frame shown in FIG. It is activated every time.

First, the variable I is set to 2 (S1), and the variable J is set to 1 (S2). The level detection circuit 22 detects the voice level from the other party # 1 and stores it in the variable X (S
3). If the variable I is less than or equal to the variable N, the process proceeds to S5, and if not, the process proceeds to S10 (S4).

In S5, the level detection circuit 22 detects the voice level from the opponent I and stores it in the variable V. If the variable V is larger than the variable X, proceed to S7, otherwise S
9 (S6). The contents of the variable I are set in the variable J (S7), the contents of the variable V are set in the variable X (S8),
1 is added to the variable I and the process returns to S4 (S9).

In S10, the recording / reproducing device 24 is instructed to record the voice of the other party J for one time slot in the voice frame shown in FIG. Further, the recording / playback device 3
4 is instructed to record the image of the other party J for one time slot in the image frame shown in FIG. 3, and the process ends (S11).

FIG. 10 shows a flowchart of the reproduction program stored in the ROM 12. This program is repeatedly started while communication is not being performed. The display control circuit 36 is instructed to display the reproduction start menu 92 and the reproduction stop menu 94 while communication is not performed.

If the reproduction start menu 92 is clicked by the mouse of the operating device 16, the process proceeds to S22, and if not, the process ends (S21). In S22, the recording / playback device 2
4 is initialized so that the voice can be reproduced from the beginning.
Further, the recording / reproducing device 34 is also initialized so that the image can be reproduced from the beginning (S23).

The audio mixing circuit 26 is instructed to output the reproduced sound without mixing the sounds to the speaker 28 (S
24) Instruct the display control circuit 36 to display the reproduction window 90 on the display device 38 (S25).

When the state of the recording / reproducing apparatus 24 is the reproduction end, the process proceeds to S30, and if not, the process proceeds to S27 (S2
6). Playback stop menu 9 with the mouse of the operating device 16
If 4 is clicked, the process proceeds to S30, and if not, the process proceeds to S28 (S27).

At S28, the sound is reproduced by the recording / reproducing device 24.
It is instructed to reproduce the time slot and output it to the audio mixing circuit 26. After that, one image is displayed on the recording / reproducing device 34.
An instruction is given to reproduce the time slot and output to the display control circuit 36, and the process returns to S26 (S29).

In S30, the display control circuit 36 is instructed to erase the reproduction window 90 of the display device 38,
finish.

In the above embodiment, the ROM 12 and the RAM 14 are used as a storage device for storing programs and data, but a floppy disk, a hard disk or a memory card may be used.

In the above embodiment, the TTU-T recommendation H.264 is used as the image encoding / decoding method. 261 is adopted, MP
EG (Moving Picture Coding)
Other image coding such as Experts Group)
A decoding method may be used.

In the above embodiment, a moving image is used as the image, but it may be a still image or a continuous still image.

In the above embodiment, TTU-T Recommendation H.264 is used as the multimedia multiplexing method. 221 is used,
A method of allocating a plurality of calls to images, voices and data may be used.

In the above embodiment, the voices and images of a plurality of partners are multiplexed by the time division multiplexing method, but frequency multiplexing, phase difference multiplexing and packet multiplexing may be used.

In the above embodiment, the magnetic disk is used for the recording / reproducing device 24 and the recording / reproducing device 34, but a magnetic tape or a magneto-optical disk may be used.

In the above embodiment, the person receiving the voice is recognized by finding the maximum voice level from one voice frame. However, the voice level in a plurality of voice frames is determined for a certain period of time. The maximum voice level may be found after averaging.

In the above embodiment, only the voice of one person recognized as receiving the voice is recorded, but the voices of all the other persons, the voices of all the persons having a certain voice level or higher, or It is also possible to record the voices of a certain number of opponents having a higher voice level. Since voice has less information content than images, it does not occupy as much storage capacity as images.

In the above-described embodiment, only the image of one person who is recognized as receiving voice is recorded, but the image of all the persons having a certain voice level or higher, or the voice level is high. It is also possible to record a certain number of images of the other party.

In the above embodiment, the microphone 20 and the camera 3
The voice encoding / decoding circuit 18 and the image encoding / decoding circuit output their own voices and images input from 2 to the level detection circuit 22 and the recording / reproducing device 34, respectively, so that their own voices and images can be recorded.

In the above embodiment, the recording and reproducing device 24 and the recording and reproducing device 34 can store only one voice and image, but a plurality of voices and images can be stored by giving a file name.

In the above embodiment, each menu 70, 72,
The operation is performed by clicking 92 and 94 with a mouse, but the operation can also be performed by inputting a command from the keyboard or pressing the touch panel.

[0041]

As can be easily understood from the above description, according to the present invention, since only the voice and image of the other party who is receiving the voice need be recorded, the amount of information to be recorded is reduced and the memory is stored. The capacity of the device can be saved.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of an embodiment of the present invention.

FIG. 2 is a frame structure diagram of a voice frame during communication according to the present exemplary embodiment.

FIG. 3 is a frame structure diagram of an image frame during communication according to the present exemplary embodiment.

FIG. 4 is an example of a display screen during communication.

FIG. 5 is a frame structure diagram of an audio frame during reproduction according to the present embodiment.

FIG. 6 is a frame structure diagram of an image frame during reproduction according to the present embodiment.

FIG. 7 is an example of a display screen during reproduction.

FIG. 8 is a memory configuration of a RAM 14.

9 is a flowchart of a recording program stored in ROM 12. FIG.

FIG. 10 is a flowchart of a reproduction program stored in the ROM 12.

[Explanation of symbols]

10: CPU 12: ROM 14: RAM 16: Operating device 18: Voice encoding / decoding circuit 20: Microphone
22: Level detection circuit 24: Recording / reproducing device 26: Audio mixing circuit 28: Speaker 30: Image encoding / decoding circuit 32: Camera 34: Recording / reproducing device 36: Display control circuit 38: Display device 40: Demultiplexing circuit 4
2: Line control circuit 44, 46, 48, 50, 52, 5
4, 56, 58: Time slot 60: Display screen 6
2, 64, 66, 68: Window 70: Recording start menu 72: Recording stop menu 7
4, 76, 78, 80, 82, 84, 86, 88: Time slot 90: Playback window 92: Playback start menu 94: Playback stop menu 100: Variable N 102 for storing the number of the other party N: The number of the other party is stored Variable I 104: Variable J 106 for storing the number of the other party having the highest voice level V 106: Variable V 108 for storing the voice level of the other party I 108: Variable X for storing the highest voice level

Claims (1)

[Claims] 1. A recognition means for recognizing from which party a voice is currently received by detecting the voice level of the other party in a video conference system which performs multimedia communication with a plurality of parties via a communication line. And a recording means for recording a voice and an image of the other party recognized as receiving the voice, and a reproducing means for reproducing the recorded voice and the image, the video conference terminal device.