JPH0813127B2 - Electronic conference system - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to an electronic conferencing system for holding a conference at a remote location, and more particularly to detecting speaker and absentee and detecting moving image data of the speaker as speaker identification information. The present invention relates to an electronic conferencing system in which an input is added to the system.

[Technical Background of the Invention] Conventionally, an electronic conference system has been known as a system for performing conference communication between conference unit systems installed in remote locations.

As a conference unit system adopted in such an electronic conference system, for example, one shown in FIG. 11 is considered. As shown in the figure, this conference unit system includes a microphone 1, a speaker 2, a television camera 3 for capturing an image of a conference room, a television camera 4 for capturing a document, a television monitor 5, a facsimile device 6, and a hard copy device 12 as a printer. , An audio unit 7 for encoding / decoding the audio signals of the microphone 1 and the speaker 2, an image unit 8 for encoding / decoding an image signal, a control unit 9 for controlling the image unit, an instruction of the control unit 9 It comprises an operation unit 10 for inputting, and a transmission unit 11 for transmitting and receiving audio signals and image signals.

[Problems of Background Art] However, in such an electronic conference system, the audio unit 7 encodes and decodes the audio signal, and the image unit 8 encodes and decodes the image signal. , The audio unit 7 and the image unit 8 must be provided with encoding / decoding functions for the maximum number of input / output devices of audio / image systems assumed in the entire system, and There is a problem that a large-scale system and a small-scale system with a small number of input / output devices are wasteful.

[Object of the Invention] The present invention has been made in consideration of such a point, and a moving image input system having a particularly large amount of data is automatically configured to have a minimum necessary amount, and speaker identification information is added to a moving image of a speaker. The purpose is to resist the electronic conferencing system that additionally compensates for the difficulty of speaker determination due to lack of resolution.

SUMMARY OF THE INVENTION That is, the present invention is an electronic conferencing system that transmits moving image data and audible sound data based on a conference attendee between a plurality of conference unit systems via a line, and has at least two voice input means. By correlating the signals input to these two voice input means, the speaker is determined from the plurality of conference attendees present in the conference room, and the speaker identification signal (ID ), An ultrasonic oscillator for generating ultrasonic waves, an ultrasonic detector for detecting the ultrasonic waves reflected by a conference participant or the wall of the conference room, and an output signal from the ultrasonic detector. A state detecting device which is provided with a judging means for judging presence / absence of an attendee, and outputs the output of the judging means as a state detecting signal (SDET) representing a seating / seating state of the conference attendee; A control device having means for generating a speaker switching signal (PID) from a signal (ID) and the state detection signal (SDET),
A plurality of moving picture input means for individually inputting the moving pictures of the plurality of conference attendees, and among the plurality of moving picture input means by the speaker switching signal (PID) generated in the control device. A video input device that selects one video input means to which a video corresponding to the speaker is input and that invalidates the output of the video input means corresponding to the conference attendees who have left the meeting. An electronic conferencing system characterized by the provision of.

Embodiments of the Invention Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a block diagram showing a conference unit system according to an embodiment of the electronic conference system of the present invention.

The control device 900 is connected with input devices of drawing system, still image system, moving image system, and audio system, and connected with output devices of magnetic storage system, storage output system, and audio system. The signals from these input devices and the signals to the output devices are subjected to processing such as selection, multiplexing, etc. by some control information signals, and are also transmitted.

As the input device, a plurality of drawing input devices 100 for inputting handwritten characters and graphics, a still image input device 200 for inputting document originals before or during a meeting, a background of a meeting attendee or a meeting place. A moving image input device 300 for inputting an image and a voice input device 400 for inputting voice are connected.

Further, the output device is displayed on a plurality of flat panel display devices 150 for displaying the input information of its own terminal, the drawing system transmitted, and the still image system, the graphic display device (1) 160, and the graphic display device (1) 160. A hard copy device 250 for storing and outputting (copying) based on information, a graphic display device (2) 350 for displaying the input information of its own terminal and the moving image that is transmitted, and a voice that outputs the input information of the transmitted audio system. The output device 450 is connected.

The storage device 600 records and reads the input / output information, and the transmission device 700 transmits the input / output information to other two or more conference unit systems at remote locations.

Transmission and reception of the above input / output information is controlled by signals from devices such as an operation input device 850, a state detection device 800 for detecting the presence / absence of a seat of a conference attendant, and a speaker identification device 510 for determining a speaker from voice. ing.

Each device forming the main part of this embodiment will be described in more detail below.

Video Input Device 300 FIG. 2 is a block diagram showing an example of the video input device 300.

The moving image input device 300 is for displaying a speaker by a plurality of TV cameras, selecting one of the TV cameras, and sending the displayed image information to the control device 900.

The moving image input device 300 includes a plurality of moving image input units 310a, 310
b, 310c, 310d, and a moving picture selector 320 for selecting one moving picture input section from the plurality of moving picture input sections based on a speaker switching signal (PID) from the control device 900, and the selected moving picture input signal A video encoding unit 330 for compressing and encoding (PIN), and an identifier adding unit 340 for adding a frame identifier (FID) and a talker switching signal (PID) to the encoded video data (PSD). It

 The operation of the moving image input device 300 will be described below.

Moving images, such as conference attendees and the background of the conference hall, are converted into electric signals by a plurality of moving image input units 310a, 310b, 310c, 310d and sent to a moving image selector unit 320.

Considering that the transmission capacity of the line is set to 64 kbps and that data such as audio must be transmitted to this line, it is impossible to simultaneously input a plurality of moving images. Therefore, the video selector 32
0 selects one corresponding moving image input unit so that the speaker is copied by the speaker switching signal (PID) of the control device 900, and sends this moving image input signal (PIN) to the moving image encoding unit 300. .

As will be described later, the speaker switching signal (PID) is a speaker identification signal (ID) generated by the speaker identification device 510 and a state detection signal (SDE) generated by the state detection device 800.
T) is generated by the conference support unit 991 of the control device 900,
When the attendee leaves the seat in the middle of the meeting, the speaker switching signal (PID) disconnects the moving-picture input section of the leaving person during the meeting.

The moving picture coding unit 330 compresses the moving picture input signal (PIN) to 32 kbps by sampling processing using a frame dropping method, predictive coding processing using an interframe coding method, direct transcoding processing using a cosine transform coding method, and the like. And sends it to the identifier adding unit 340 as moving image data (PSD).

The identifier adding unit 340 includes a frame identifier (FID) indicating moving image data and a speaker switching signal (PI) indicating which moving image input unit has been selected from the plurality of moving image input units.
D) and are added to the video data (PSD) and the transmission data (SX
D) to the control device 900.

 Next, the moving picture coding unit 330 will be described.

Video Coding Unit 330 An example of the video coding unit 330 is shown in FIG.

The moving picture coding unit 330 includes an input processing unit 331, a transmission memory unit 332, a cosine transform unit 333, and a coding unit 334.

The operation of the moving picture coding unit 330 will be described below.

First, the moving image input signal (P
IN) is sent to the input processing unit 331, and after the color burst and the sync signal are removed from the moving image input signal, 8-bit A / D conversion is performed. Furthermore, the number of pixels in each frame is reduced from 256 × 256 to 12
The frame frequency is also reduced from 30 frames / sec to 10 frames / sec to 8 × 128.

Next, the data of each frame subjected to the input processing is transmitted to the transmission memory unit 332, and is stored in a working frame memory (not shown). The data of the immediately preceding rewritten frame is stored in a reference frame memory (not shown), and the current frame and the reference frame are compared in block units of 8 × 8 pixels to determine the block to be rewritten. When a certain block is largely changed, the reference memory is rewritten and the block information is input to the cosine transform unit 33. Blocks that do not change much are not processed or rewritten.

Further, the cosine transform unit 333 removes the correlation between the pixels by two-dimensional cosine transform, and sends the obtained transform coefficient to the encoding unit 334.

Since many of these conversion coefficients take zero, bit 0
Instead of sending a long sequence of.

The coding unit 334 then selects the actual code in the Huffman coding table according to the prediction of the local energy of the image for the transform coefficients. Also, information about the block is encoded.

This encoded data enters a transmission buffer (not shown),
From here, it is sent as moving image data (PSD) to the identifier adding unit 340 at a predetermined transmission speed.

Also, the encoding unit 334 performs transmission rate control to prevent overflow or underflow of transmission data.

In this way, the moving picture coding unit 330 combines three compressions in terms of spectrum, space and time.

That is, on the spectrum, the sub-sampling is performed for the color component filtering with respect to the redundancy of the color component, and on the space, the sub-sampling in both the horizontal and vertical directions and the correlation between pixels are excluded with respect to the redundancy in the luminance or amplitude. 2D cosine transform coding is performed, and temporally, frame interlacing, interpolation, and conditional rewriting are performed on the redundancy in the value between frames.

Speaker Identification Device 510 FIG. 4 is a block diagram showing an example of the speaker identification device 510.

This speaker identification device 510 is a device for identifying a speaker using a code correlation when there is a plurality of conference participants who speak, and microphones 512 and 514 and amplifiers 516 and 515. , Code detectors 520 and 522, 64-tap shift register 524, 32-tap shift register 526, 64 exclusive OR circuits 528 as correlation detectors, and 64 up / down counters 530-1 to 530 -64, coder 532, OR circuit 534 and latch 536.

FIG. 5 shows an example of installation of the speaker identification device 510 when there are five conference participants A to E. Microphones 512 and 514 are installed on the left and right sides of the device as shown in FIG. Has been done.

The microphones 512 and 514 convert the voice uttered by the speaker into electric signals, and the amplifiers 516 and 515 amplify the converted first channel audio signal and second channel audio signal to appropriate levels.

Next, the sign detectors 520 and 522 set the sign to “1” if the signals output from the amplifiers 516 and 515 are (+), and sign “0” if the signals are (−), SIGNR (k) and SIGNL (k ) Is output.

However, in the case of this embodiment, the audio signal is about 2 bands.
In the double frequency, for example, 0 to 3.4 kHz band, it is represented by a signal at the time of k samples sampled at 8 kHz.

Next, the 64-tap shift register 524 shifts the code of the second channel every 8 kHz. The 32-tap shift register 526 shifts the code of the first channel and delays it by 32 samples.

Then, the exclusive OR circuit 528 outputs the output SIGNR (k−i +) of the i-th stage of the shift register 524 as shown in the equation (1).
1) and the output of the sample shift register 526, SIGNL
Sign correlation is obtained with (k−32).

Zi = SIGNR (k-i + 1) + SIGNL (k-32) (1) The up / down counters 530-1 to 530-64 count up this correlation output Zi when Zi = 1, and when Zi = 0. Time is counted down for averaging, and when this counter overflows, the tap position with the highest correlation is detected.

That is, the coder 532 is the up / down counter 530-1.
Find the overflow position of 530-64, encode the position information into 6 bits, and latch the upper 3 bits
Output to 536.

Further, the OR circuit 534 clears each of the up / down counters 530-1 to 530-64 when an overflow occurs and sends a signal to the latch 536.

The latch 536 captures the 3-bit information when receiving this signal and sends it to the control device 900 as a speaker identification signal ID.

6 (A), (B), (C), (D), and (E) are impulse responses when the conference participants A, B, C, D, and E shown in FIG. 5 speak. Since the voice arrival time to each of the left and right channels differs depending on the speech of each conference participant, the speaker identification can be performed by correlating the input signals of the left and right channels.

That is, the left and right microphones are installed as shown in FIG. 5, and when the impulse response is obtained from the transfer characteristics between the channels, as shown in FIG. 6, the voice arrival time difference between the left and right channels is the speaker position. Depends on

In the case of Fig. 5, since the delay is within ± 4msec, the correlation detector sets the voice band to 3.4kHz and one sample is 125μse.
If it is c, it is sufficient if it can process 32 samples before and after the total of 64 samples.

Therefore, the correlation detector of FIG. 4 can implement a three-party teleconference at 64 sumps.

State Detection Device 800 The state detection device 800 detects the presence or absence of conference attendees, and an example thereof is shown in FIG.

This state detection device 800 has n seats 802-1 to 802-n.
N detection units 804-1 to 804-n corresponding to
It comprises a coder 806 which encodes the output codes 4-1 to 804 to n and sends it as a state detection signal SDET to the conference support unit 991 in the control device 900 described later.

This detection unit 804-i is an oscillator (OSC) that generates ultrasonic waves.
808, a detector (DET) 810 for detecting the ultrasonic waves reflected by conference participants and walls, and a measuring circuit 812.

FIG. 8 is a detailed circuit diagram of the detection unit 804-i. The measurement circuit 812 includes a pulse generation circuit (PG) 814 that generates the pulse shown in FIG. 9 and an inverter that inverts the signal of the detector 810.
816, an AND circuit 818 that ANDs the output of the pulse generation circuit 814 and the output of the inverter 816, and a timer (TIMER) 820 that measures the time when the output of the AND circuit 818 is "1".
And a preset value T for the output signal C of the timer 820
When C <T, the output signal STi is set to "1", and the comparator (COMP) 822 indicates that the user is seated.

 Next, the operation of this state detecting device 800 will be described.

First, when a pulse as shown in FIG. 9 (a) is generated from the pulse generating circuit 814, the oscillator 808 oscillates only when the pulse is “1”, and the ultrasonic wave as shown in FIG. 9 (b) is generated. Generate.

This ultrasonic wave is reflected by the conference participant when the conference participant is sitting in the seat, is reflected by the wall when the conference participant is not sitting in the seat, and is delayed by the delay time τ as shown in FIG. 9 (c). It arrives at the detector 810 later.

When the detector 810 detects this ultrasonic wave, its output signal becomes "1" (Fig. 9 (d)). This is inverted by the inverter 816, and the logical sum of the output signal of the pulse generation circuit 814 and the output signal of the inverter is obtained by the AND gate 818. Therefore, the output of the AND gate 818 is as shown in FIG. 9 (e). Becomes

That is, the output signal of the AND gate 818 becomes a signal which becomes "1" only during the delay time τ. The delay time τ is small when a conference attendee is seated in the seat, and is large when there is no conference attendee.

Therefore, the output signal of the AND gate 818 of the timer 820 is "1".
Is measured and is input to the comparator 822 as the signal C.

The comparator 822 compares the signal C with the set value T. The set value T is determined in advance so that the signal C is smaller than T when the conferees are seated, and the signal C is larger than T when the conferees are not seated. Therefore, when C <T, the output signal STi of the comparator 822 becomes "1", and when C> T, the output signal STi of the comparator becomes "0". I know whether or not.

In FIG. 7, the coder 806 includes the detection units 804-1 to 804.
The output signal STi (i = 1 to n) sent from -n is encoded and the state detection signal SDET is sent to the conference support unit 991.

Control Device 900 FIG. 10 is a block configuration diagram showing the configuration of the control device 900.

Data from each input device and data to each output device are transmitted via the internal data bus 901.

That is, drawing data from the drawing input devices 100a, 100b ... Is multiplexed by the multiplexing unit 911 and sent to the data bus 901 via the input interface unit 912 and the data buffer 913. Here, the alphabetical letters a, b ... Show that a plurality of devices having the same function are connected (the same applies hereinafter).

The still image data from the still image input device 200 is sent to the data bus 901 via the input interface unit 922 and the data buffer 923.

The moving image data from the moving image input device 300 is supplied to the data bus 90 via the input interface unit 932 and the data buffer 933.
Sent to 1.

Audio data from the audio input device 400 is transferred to the data bus 90 via the input interface unit 942 and the data buffer 943.
Sent to 1.

To these input data, a frame identifier (FID) indicating from which input device the data frame originated is added.

For data output, first, the still image data is data bus 90
1 to the hard copy device 260 via the output interface unit 926.

Similarly, still images and drawing data are transmitted from the data bus 901 through the output interface unit 916 to the flat panel display device 150.
are sent to a, b, ... And to the graphic display device (1) 160 via the output interface unit 917, and the moving image data is sent to the graphic display device (2) 3 via the output interface unit 936.
The audio data is sent to the output interface unit 946.
Is sent to the audio output device 450 via.

Further, a still image and drawing data can be sent from the graphic display device (1) 160 to the data bus 901 via the input / output interface unit 917.

The stored data from the storage device 600 is sent to the data bus 901 via the file interface 961, and conversely, the data is sent from the data bus 901 to the storage device 600 via the file interface 961.

The frame controller 990 switches these data transmission destinations, selects data, and multiplexes.

When sending the data group collected in the frame controller 990 to the line, it is sent to the transmission device 700 via the transmission interface 972. The data group sent from the line via the transmission device 700 is the transmission interface 97
2 to the frame controller 990.

Here, the data buffers 913, 923, 933, and 943 are used when the transmission to the data bus is temporarily interrupted due to the switching of the data transmission destination or the selection of the data, or when the transmission line capacity is transiently exceeded due to the multiplexing. It is for accumulating data when doing.

The conference support unit 991 controls the frame controller 990. The conference support unit 991 is a speaker identification device 51
0, and instructs the frame controller 990 to select data or to send to the frame controller 990 by signals from the state detection device 800, the operation input device 850, and the like.

With the above data amount, the conference support unit 991 advances the conference.

 Hereinafter, the operation of control device 900 will be described.

First, a speaker identification signal (ID) indicating who the speaker is is input from the speaker identification device 510. Based on this signal, the conference support unit 991 outputs a speaker switching signal (PID) to the moving image input device 300 so as to capture the speaker.

These moving image data are multiplexed by the frame controller 990 and sent to the transmission device 700 via the transmission interface 972. A speaker switching signal (PID) indicating a speaker is added to the data frame in the moving image data.

At the same time, the video data is output to the output interface 93
It is sent to the graphic display device (2) 350 via 6 and displayed.
This moving image data is data compressed by a method such as frame correlation. The speaker can determine that he / she is selected because he / she is displayed on the graphic display device (2) 350.

A state detection signal (SDET), which indicates a signal such as the number of conference attendees or an abandoned person during a conference, is transmitted from the state detection device 800 to the conference support unit
Entered in 91. Based on this signal, the conference support unit 991 instructs the frame controller 990 to select data input from each input device, and also generates a talker switching signal (PID) based on this signal to input a moving image. The device 300 is instructed to disconnect the corresponding moving image input unit 310, or the display of the graphic display device (2) 350 is instructed to be erased by the speaker switching signal (PID).

As described above, in this embodiment, the conference support unit 991 generates a speaker identification signal (PID) from the speaker identification signal detected by the speaker identification device 510 and the state detection signal detected by the state detection device 800. Video input device 300
It is input to the video selector unit 320 of the
Video input section 31 corresponding to the conference attendees who are speaking by 20
Since 0 is selected and the moving image input unit 310 corresponding to the conference attendee who has left his / her seat is disconnected, the moving image data of the conference attendee who is speaking is automatically input to the present system, and the operator is not required.

[Effects of the Invention] As described above, according to the present invention, a talker identification signal is generated from a speaker identification signal and a state detection signal, and the talker identification signal is used to select and disconnect the moving picture input unit of the moving picture input device. Since the video data of the speaker is automatically input to the present system, the operator is not required and the entire system can be downsized.

[Brief description of drawings]

1 is a block configuration diagram of an electronic conference system, FIG. 2 is a block configuration diagram of a moving image input device, FIG. 3 is a block configuration diagram of a moving image encoding unit, FIG. 4 is a block configuration diagram of a speaker identification device, FIG. 5 is a diagram showing an installation example of a speaker identification device, FIG. 6 is an impulse response diagram, FIG. 7 is a block configuration diagram of a state detection device, FIG. 8 is a block configuration diagram of a detection unit, and FIG. FIG. 10 is a waveform diagram of signals of respective parts of the state detection device, FIG. 10 is a block configuration diagram of a control device, and FIG. 11 is a block configuration diagram of a conventional electronic conference system. 300: Video input device 510: Speaker identification device 800: State detection device 900: Control device

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Ryoichi Nakamura 3-1-1 Asahigaoka, Hino-shi, Tokyo Inside the Toshiba Hino Plant (72) Inventor Hirokazu Sakano 3-1-1 Asahigaoka, Hino-shi, Tokyo Toshiba Corporation Hino Factory (72) Inventor Masayoshi Aihara 3-1-1 Asahigaoka, Hino-shi, Tokyo Toshiba Corporation Hino Factory

Claims (2)

[Claims] 1. An electronic conferencing system for transmitting moving image data and audible sound data based on attendees between a plurality of conference unit systems via a line, comprising at least two voice input means, and these two voices. A talk that determines a speaker from among a plurality of conference attendees present in the conference room by correlating the signals input to the input means and generates a speaker identification signal (ID) indicating this speaker. Person identification device, an oscillator that generates ultrasonic waves, an ultrasonic detector that detects the ultrasonic waves reflected by the conference participant or the wall of the conference hall, and the output signal from the ultrasonic detector is input to the attendees. A state detection device that includes a determination unit that determines presence / absence, and outputs the output of the determination unit as a state detection signal (SDET) indicating a seating state of a conference attendee, the speaker identification signal (ID), and the state A control device having a means for generating a talker switching signal (PID) from an outgoing signal (SDET), and a plurality of moving image input means for individually inputting the moving images of the plurality of conference attendees. From the plurality of moving picture input means, one moving picture input means to which a moving picture corresponding to the speaker is inputted is selected by the speaker switching signal (PID) generated in An electronic conferencing system comprising: a moving image input device having a selection unit for invalidating the output of the moving image input unit corresponding to an attendee. 2. The plurality of judging means, a pulse generating circuit for generating a pulse in a predetermined cycle, an inverter for inverting the polarity of an output signal from the ultrasonic detector, an output of the pulse generating circuit and the A logical product circuit for taking a logical product of the outputs of the inverter, a timer for measuring the time when the output of the logical product circuit is at the first level, and an output signal C of the timer.
Is compared with a preset value T, and when C <T, a state detection signal of a first level indicating that the user is seated, and when C> T, the first state indicating signal is detected.
2. The electronic conferencing system according to claim 1, further comprising a comparator that outputs a state detection signal of a second level different from the level of 1.