JPH0771275B2 - Electronic conference system - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to an electronic conference system for holding a conference between remote points, and more particularly to an electronic conference system for displaying attendees who are speaking in a moving image.

[Technical Background of the Invention] This type of electronic conference system is composed of a plurality of conference unit systems connected via lines. As the conference unit system, for example, one shown in FIG. 35 has been developed. ing. As shown in the figure, the conference unit system includes a microphone 1, a speaker 2, a television camera 3 for capturing an image of a conference room, a television camera 4 for capturing an original document,
A television monitor 5, a facsimile device 6, a hard copy device 12 as a printer, an audio unit 7 for encoding / decoding an audio signal of the microphone 1 and the speaker 2, an image unit 8 for encoding / decoding an image signal, It comprises a control unit 9 for controlling the image unit, an operation unit 10 for inputting instructions from the control unit 9, and a transmission unit 11 for transmitting and receiving audio signals and image signals.

The video signal of the conference hall obtained from the TV camera 3 is also continuously transmitted to another conference unit system via the line by the transmission unit 11, and conversely the video signal transmitted from the other conference unit system is transmitted. It is received by the unit 11 and reproduced on the television monitor 5. Therefore, the image of the conference hall of the other conference unit system is displayed on the television monitor 5.

[Background Art Problems] However, in such an electronic conference system, since the conference hall is imaged, there is a problem that the speaker cannot clearly grasp. Therefore, it is conceivable to perform a zoom operation so as to magnify and image only the speaker, but in that case, this zoom operation becomes troublesome and a special operator is required. In order to eliminate these inconveniences, it is considered that the TV cameras 3 are prepared by the number of speakers, the operator operates the TV cameras 3 for the speakers who are making a statement, and the TV monitor 5 displays them. However, even in this case, the operator needs to perform an operation such as switching the television camera 3 each time the speaker changes, which causes various problems such as deterioration of the functional service for proceeding the conference.

[Object of the Invention] An object of the present invention is to provide an electronic conferencing system capable of automatically identifying a speaker and displaying the speaker and other attendees separately in order to solve the above problems. .

[Summary of the Invention] In order to achieve the above object, the present invention has a plurality of image pickup means for picking up a moving image of each of a plurality of conference attendees and a plurality of audible sound input means for inputting an audible sound. In an electronic conferencing system in which a plurality of conference unit systems for transmitting the moving image data and the audible sound data are connected via a line, an identifying means for determining a speaker from the plurality of conference attendees, and an identifying means Generating the identification information indicating the speaker determined by the, transmitting means for transmitting this identification information through the line, receiving means for receiving the identification information from the line, one screen is divided into a plurality of areas, Based on the moving picture data of the speaker corresponding to the identification information received by the receiving means, the speaker is displayed as a moving picture in a predetermined area of the divided screen, and the moving picture data of the conference attendees other than the speaker are displayed. Display means for displaying the other attendees of the conference as a still image on the other area of the divided screen based on the information.

The second invention has a plurality of image pickup means for picking up a moving image of each of the plurality of conference attendees and a plurality of audible sound input means for inputting an audible sound, and stores the moving image data and the audible sound data from each means. In an electronic conferencing system in which a plurality of conference unit systems for transmission are connected via a line, an identification means for determining a speaker from the plurality of conference attendees and a speaker determined by the identification means are shown. Transmitting means for generating the identification information and transmitting the identification information through the line;
A receiving means for receiving the identification information from the line and one screen divided into a plurality of areas, and a predetermined area of the divided screen based on the moving picture data of the speaker corresponding to the identification information received by the receiving means. Along with displaying the video in the area, based on the video data of the conference attendees other than the speaker and the final video data when the input of the identification information is cut off, the other conference attendees of the divided screen are displayed. Display means for displaying a still image in another area is provided in the conference unit system.

Embodiments of the Invention Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a block diagram showing a conference unit system according to an embodiment of the electronic conference system of the present invention.

The control device 900 is connected with input devices of drawing system, still image system, moving image system, and audio system, and is connected with output devices of magnetic storage system, recording output system, and audio system. The signals from these input devices and the signals to the output devices are subjected to treatment such as selection, multiplexing, etc. by some control information signals, and are also transmitted.

As the input device, a plurality of drawing input devices 100 for inputting handwritten characters and graphics, a still image input device 200 for inputting document originals before or during a meeting, a background of a meeting attendee or a meeting place. Video input device for inputting images
300 and a voice input device 400 for inputting voice are connected.

Further, the output device is displayed on a plurality of flat panel display devices 150 for displaying the input information of its own terminal, the drawing system transmitted, and the still image system, the graphic display device (1) 160, and the graphic display device (1) 160. A hard copy device 260 for recording and outputting (copying) based on information, a graphic display device (2) 350 for displaying input information of its own terminal and moving image system transmitted, a voice for outputting transmitted audio system input information. The output device 450 is connected.

Further, the storage device 600 records and reads out these input / output information. The transmission device 700 is connected to two or more conference unit systems at remote locations.

The above input / output information is transmitted by a signal from a device such as an operation input device 850, a state detection device 800 that detects the presence / absence of a seat of a conference attendee, a speaker identification device 510 that determines a speaker from the voice of a speaker. Is controlled.

A more detailed description will be given below for each device.

Drawing Input / Output Device 100 FIG. 2 is a block diagram of the drawing input device 100.

This drawing input device 100 detects coordinate position information of a handwritten figure, and when pressure is applied to the illustrated writing input surface, drawing of a pressure detection method that outputs coordinate data of the point where the pressure is applied The input section 101 and the drawing entrance section 101
A sampling unit 103 for sampling analog coordinate data connected to, a pen touch detection unit 104 for detecting a handwriting input, an input processing unit 105 connected to the sampling unit 103, and an encoding unit 106 for encoding drawing data. , And an identifier adding unit 109 that frames the encoded drawing data.

The input pen 102 is for applying pressure to the writing input surface by drawing a figure or the like on the paper placed on the writing input surface of the drawing input unit 101.

Next, the operation of the drawing input / output device 100 will be described.

First, when pressure is applied to the writing input surface of the drawing input unit 101 (when there is a pen touch), it is detected from the signal output from the drawing input unit 101 that the writing input (pen touch) is performed. When a pen touch is detected, the pen touch detection unit 10
The sampling unit 10 outputs a control signal for instructing the sampling unit 103 and the input processing unit 105 to start operation.
3, and the input processing unit 105 is made operable.

This sampling unit 103 samples the analog coordinate data output from the drawing input unit 101 at a predetermined cycle,
It is digitized and output to the input processing unit 105.

The input processing unit 105 receives the coordinate data output from the sampling unit 103, checks whether this coordinate data is appropriate data as coordinate values, and the coordinate data is a suddenly discrete value. Then, based on the previous coordinate data, interpolation processing or the like for correcting the coordinate data is performed. Further, when a control signal according to a command indicating partial erasure is given from the input analysis unit 108 described later, based on the coordinate data of the closed curve output from the sampling unit 103, it will be described later enclosed by this closed line. All coordinate data of the area in the frame memory unit 152 is logically “0” (“0”).
Represents the data that becomes the background color when displayed), and creates the drawing data that is the basis for rewriting it, and outputs this.

Next, the encoding unit 106 encodes the drawing data output from the input processing unit 105 and outputs it to the identifier adding unit 109.

Further, the identifier adding unit 109 adds a frame identifier (FID) indicating that it is drawing data to the drawing data and outputs it to the control device 900.

Note that reference numeral 107 indicates a switch input unit, and the operator
With this switch input unit 107, commands such as cursor display, partial erasing, and full erasing can be input.

In addition, the input analysis unit 108 connected to the switch input unit 107
By analyzing the data output from the switch input unit 107,
Which instruction the data indicates is analyzed, and the corresponding control signal is output to the input processing unit 105 and the encoding unit 106 based on the analysis result.

Flat Display Device 150 FIG. 3 is a block diagram of the flat display device 150.

The plane display device 150 displays drawing data or still image data, and includes a plane display unit 151 including an LCD (Liquid Crystal Display), a frame memory unit 152 for storing image data, and a drawing or still image display unit. A drawing reproduction processing unit 153 for performing reproduction, a still image reproduction processing unit 154,
A drawing decoding unit 155 that decodes drawing data or still image data based on image information, a still image decoding unit 156, and an identifier detection unit 157 that identifies control information.

The operation of the flat panel display device 150 will be described below.

First, the control device 900 sends control information and image information to the identifier detection unit 157.

This control information is made up of a frame identifier (FID), a device address (SAD), page control (PAGE), etc., and there are two types of image information, drawing data and still image data. In the drawing data, for example, a command for moving the cursor is incorporated in the coordinate position information. Still image data is data in which the run length is compressed by a method using modified read (MR) code, for example.

Next, the identifier detection unit 157 determines whether the drawing data or the still image data is based on the frame identifier (FID) in the control information.
5, and the still image data is sent to the still image decoding unit 156.

The drawing data decoded by the drawing decoding unit 155 is sent to the drawing processing unit 153, interpolation processing of the coordinate data necessary for graphic display is performed, and the drawing data is sent to the frame memory unit 152.

The still image data decoded by the still image decoding unit 156 is sent to the still image reproduction processing unit 154, and image processing such as linear density conversion, enlargement and reduction necessary for graphic display is performed, and the frame memory unit 152. Sent to.

The image data output from the drawing reproduction processing unit 153 or the still image processing unit 154 is written in the frame memory unit 152. The image data stored in the frame memory unit 152 is
It is read and displayed on the flat display unit 151 in a predetermined cycle.

Graphic display device (1) 160 The configuration of the graphic display device (1) 160 is shown in FIG.

The graphic display device (1) 160 includes an identifier detection unit 161, which identifies control information, a drawing decoding unit 162a which decodes drawing data or still image data from image information, and a still image decoding unit 162b. A drawing reproduction processing unit 163a and a still image reproduction processing unit 163b that perform image processing on the generated image data.
And a frame memory unit 164 for storing image data,
A graphic display unit (1) 165 on which this image data is displayed,
A still image transmission processing unit 166 that reads out image data from the frame memory unit 164 and performs image processing, a still image encoding unit 167 that encodes the transmitted still image data, and an identifier addition that frames the encoded still image data. And part 168.

The operation of the graphic display device (1) 160 will be described below.

First, the control device 900 sends the control information and the image information to the identifier detection unit 161.

This control information consists of a frame identifier (FID), a device address (SAD), page control (PAGE), etc., and there are two types of image information: drawing data and still image data.

Next, in the identifier detection unit 161, it is determined whether the data is drawing data or still image data based on the frame identifier (FID) of the control information, and further, according to the content of the page control (PAGE), a transmission request for the still image data described later. It is determined whether or not.

Further, the image information determined to be drawing data by the frame identifier (FID) is sent to the drawing decoding unit 162a, and the image information determined to be still image data is sent to the still image decoding unit 162b.

The drawing data decoded by the drawing decoding unit 162a is sent to the drawing reproduction processing unit 163a, subjected to interpolation processing necessary for graphic display, and sent to the frame memory unit 164.

On the other hand, the still image data decoded by the still image decoding unit 162b is sent to the still image reproduction processing unit 163b, and image processing such as linear density conversion, enlargement and reduction necessary for graphic display is performed, and the frame memory It is sent to the unit 164.

In this way, the frame memory unit 164 includes the drawing reproduction processing unit 1
The image data output from 63a or the still image reproduction processing unit 163b is written. The image data stored in the frame memory unit 164 is read by the graphic display unit (1) 165 in a predetermined cycle and displayed on the graphic display unit (1) 165.

If a request to send still image data is specified by page control (PAGE), the stored image data is read from the frame memory unit 164, and the still image sending processing unit 16
At 6, image conversion such as linear density conversion, enlargement, and reduction is performed, and the result is sent to the still image encoding unit 167.

The still image data encoded by the still image encoding unit 167 is added with a frame identifier (FID) by the identifier adding unit 168 and is sent to the control device 900.

Still image input device 200 The still image input device 200 is an input for optically reading the conference material or the like in order to display the conference material or the like on the graphic display device (1) 160 or send it to the other terminal in the electronic conference system. It is a device.

The configuration of the still image input device 200 is shown in FIG.

The still image input device 200 includes a reading unit 201 that optically reads a document, a buffer unit 202 that temporarily stores the read still image data, an encoding unit 203 that encodes the still image data, and a frame identifier. It is composed of an identifier adding unit 204 for adding (FID) and is connected to the control device 900.

The operation of the still image input device 200 will be described below.

First, the manuscript reading unit 201 is instructed by a not-shown operation input means for a reading operation, a still image size, a linear density, etc., reads the manuscript, and transfers the read data to the buffer unit 202.

Next, the buffer unit 202 absorbs the difference between the transfer speed of the still image data from the reading unit 201 and the data input speed of the encoding unit 203, and only a part of the data from the reading unit 201 is enlarged or large. In the case of characters, image conversion is performed to reduce the amount of information by roughening the line density.

Then, the encoding unit 203 encodes the read data,
The encoded data is sent to the identifier adding unit 204. This encoded data is transferred at a rate of 16K bps.

If you read an A4 size document at a line density of 8 dots / mm x 7.7 lines / mm, the data amount will be about 480 Kbytes and the transfer time will be about 240 seconds. If a redundancy suppression code such as the Hidman Huffman (MH) method or the modified read (MR) method is used, the data amount is about 30 Kbytes, and the transfer time is compressed to about 15 seconds.

Finally, the identifier adding unit 204 adds a frame identifier (FID) indicating that the image is still image data,
It is sent to the control device 900.

Hard copy device 260 The hard copy device 260 is a device for obtaining a recording output based on the information on the screen displayed on the graphic cover device (1) 160, and the recording method is a heat sensitive head and a heat sensitive device used in a facsimile or the like. A thermal recording method using recording paper is used.

The configuration of the hard copy device 260 is shown in FIG.

The hard copy device 260 includes an identifier detection unit 261 that identifies control information and a decoding unit 262 that decodes still image data.
And a buffer unit 263 for temporarily storing this still image data.
And a recording unit 264 for recording on the thermal recording paper by the thermal head.
And is connected to the control device 900.

The operation of this hard copy device will be described below.

First, control information and image information are sent from the control device 900 to the identifier detection unit 261.

The control information is frame identifier (FID), device address (SA
D), page control (PAGE), etc., and the image information is only one type of still image data.

Next, when the identifier detection unit 261 detects that it is still image data by the frame identifier (FID) in the control information,
The still image data is sent to the decoding unit 262.

Further, the decoding unit 262 decodes the still image data and transfers the decoded data to the buffer unit 263.

The buffer unit 263 absorbs the difference between the transfer speed of the decoded data from the decoding unit 262 and the speed of the recording data to the recording unit 264, and performs image processing such as linear density conversion, enlargement and reduction,
It is sent to the recording unit 264.

Finally, the recording unit 264 operates the thermal head, the drive motor, and the like based on the recording data from the buffer unit 263, and records information based on the recording data on the thermal recording paper.

Video Input Device 300 FIG. 7 is a block diagram of the video input device 300.

The moving image input apparatus 300 is for displaying a speaker by a plurality of TV cameras, selecting a predetermined TV camera among the speakers, and sending image information displayed to the control apparatus 900.

This video input device 300 includes a plurality of video input units 310a, 310b, 3
10c, 310d, and a speaker switching signal (PI
D), a moving picture selector section 320 for selecting one moving picture input section from the plurality of moving picture input sections, and a moving picture coding section 330 for compressing and coding the selected moving picture input signal (PIN).
And an identifier adding unit 340 that adds a frame identifier (FID) and a talker switching signal (PID) to the encoded moving image data (PSD).

The operation of the moving image input device 300 will be described below.

Moving images such as the background of conference attendees and conference halls are converted into electric signals by a plurality of video input units 310a, 310b, 310c, 310d,
It is sent to the video selector unit 320.

Considering that the transmission capacity of the line must be 64 kbps and data such as voice must be sent to this line, it is impossible to input multiple moving images at the same time. So the video selector 320
Selects one of the moving picture input units corresponding to the sender switching signal (PID) of the control device 900 so as to capture the caller, and sends this moving picture input signal (PIN) to the moving picture coding unit 330.

Note that the speaker input signal (PID) can be used to disconnect the moving image input unit of the person leaving the seat during the conference.

The moving picture coding unit 330 compresses the moving picture input signal (PIN) to 32 kbps by sampling processing by frame dropping method, predictive coding processing by frame coding method, direct exchange coding processing by cosine transform coding method, etc. And sends it to the identifier adding unit 340 as moving image data (PSD).

The identifier adding unit 340 includes a frame identifier (FID) indicating that the data is moving image data, and a speaker switching signal (PID) indicating which moving image input unit is selected from a plurality of moving image input units.
And are added to the moving image data (PSD) and sent to the control device 900 as transmission data (DXD).

Next, the moving picture coding unit 330 will be described.

Video Coding Unit 330 The configuration of the video coding unit 330 is shown in FIG.

The moving picture coding unit 330 includes an input processing unit 331, a transmission memory unit 332, a cosine conversion unit 333, and a coding unit 334.

The operation of the moving picture coding unit 330 will be described below.

First, the video input signal (PI
N) is sent to the input processing unit 331, and after the color burst and the synchronizing signal are removed from the moving image input signal, A / D conversion is performed with 8 bits. Furthermore, the number of pixels in each frame is changed from 256 × 256 to 128 by a digital filter (not shown).
To 128, the frame frequency is also reduced from 30 frames / sec to 10 frames / sec.

Next, the data of each frame subjected to the input processing is sent to the transmission memory unit 332 and is stored in the working frame memory (not shown). The data of the previous rewritten frame is stored in the reference frame memory (not shown),
The current frame and the reference frame are compared in a block unit of 8 × 8 pixels to determine a block to be rewritten. When a block is largely changed, the reference memory is rewritten and the block information is input to the cosine transform unit 333. Blocks that do not change much are not processed or rewritten.

Further, the cosine transform unit 333 removes the correlation between pixels by the two-dimensional cosine transform, and sends the obtained transform coefficient to the coding unit 334.

Since many of these transform coefficients take zero, they are compressed by run-length coding instead of sending a long string of bits 0.

The coding unit 334 then selects the actual code in the Huffman coding table according to the prediction of the local energy of the image for the transform coefficient. It also encodes information about blocks.

This encoded data enters a transmission buffer (not shown), from which it is transmitted as moving image data (PSD) to the identifier adding unit 340 at a predetermined transmission rate.

Graphic Display Device (2) 350 FIG. 9 is a block diagram of the graphic display device (2) 350.

This graphic display device (2) 350 displays the input information of its own terminal and the moving image system that is transmitted, and it receives the device address (SA) from the received data (RXD) output from the control device 900.
D), frame identifier (FID) and speaker switching signal (PI
D) for detecting the identifier, moving image decoding units 370a and 370b for decoding the moving image data (PRD), and a frame memory unit for temporarily storing and synthesizing the decoded moving image signal (POUT). 380, a display control unit 385 for controlling display of a plurality of moving images and moving images in a still state in a plurality of divisions, and a graphic display unit 390 for displaying moving images.

The operation of the graphic display device (2) 350 will be described below.

The reception data (RXD) sent from the control device 900 is composed of control information and image information, and the control information includes a frame identifier (FID), a device address (SAD), and a talker switching signal (PI).
D) etc., and the image information is only moving image data.

First, the identifier detection unit 360 detects that the frame identifier (FID) is moving image data, then detects the device address (SAD), and determines whether the own terminal is the partner terminal. Further, it determines from which moving picture input section the moving picture data is based on the speaker switching signal (PID).

Next, the video data (PRD) from which the frame identifier (FID), the device address (SAD) and the talker switching signal (PID) are removed by the identifier detection unit 360 is the video decoding unit 370a or 370b.
The video data that has been compressed and encoded to 32 kbps is decoded.

The video decoding units 370a and 370b have the same configuration so that they can be correctly decoded without delay even if the own terminal and the other terminal, or the other terminal and the other terminal simultaneously input, corresponding to the device address (SAD). It has two systems.

The decoded moving image signal (POUT) is temporarily stored in a predetermined area of the frame memory unit 380, and the display screen is combined.

Further, when the decoded video signal (POUT) is not input to the predetermined area, the frame memory unit 380 keeps the memory of the finally obtained video signal until a new video signal is input to the corresponding area. To do. Therefore, a still image is displayed except for the speaker.

The device address (SAD) detected by the identifier detector 360
And the talker switching signal (PID) is sent to the display control unit 385, and a plurality of moving images of the own terminal and any two terminals of the two partner terminals and still images are displayed on the same screen with respect to the frame memory unit 380. Display control for split display is performed.

Then, the moving image signal is read from the frame memory unit 380 in a predetermined cycle and displayed on the graphic display unit 390.

Next, the moving picture decoding unit 370a (370b) will be described.

Video Decoding Unit 370a (370b) FIG. 10 shows the configuration of the video decoding unit 370a.

The moving picture decoding unit 370a includes a decoding unit 371 that decodes moving picture data in block units, and an inverse cosine transform unit 37 that performs inverse cosine transform on the decoded transform coefficients.
2, a reception memory unit 373 having a current frame memory and a reference frame memory, and an output processing unit 374 for outputting a moving image frame from the current frame memory and the reference frame memory.

The operation of the moving picture decoding unit 370a will be described below.

First, the video data (PRD) output from the identifier detector 360
Is placed in a reception buffer (not shown) of the decoding unit 371. Next, the block information is decoded, and the decoding step size is determined to obtain the transform coefficient using the same table as the Huffman coding table, and the position of the block to be rewritten is determined.

The inverse cosine transform unit 372 then transforms the decoding unit 371.
2D inverse cosine transform is applied to the transform coefficient reproduced in
Output block unit data.

Then, the reception memory unit 373 rewrites the corresponding area in the current frame memory (not shown) in block units of 8 × 8 pixels based on the position information of the block to be rewritten.

In addition, the reference frame memory (not shown) stores the data of the previous frame that has been rewritten. It is used to compare with the next frame.

Further, the output processing unit 374 uses the current frame memory and the reference frame memory of the reception memory unit 373 to return the frame frequency from 10 frames / second to 30 frames / second, and the pixel level in the reproduced frame temporally. The frame is interpolated by calculation so that it changes smoothly.

Finally, the video signal (POU
T) is sent to the frame memory unit 380.

Voice Input Device 400 The voice input device 400 is for converting the voice of a conference attendee into an electric signal and incorporating it into the electronic conferencing system, and FIG. 11 is a block configuration diagram thereof.

As shown in the figure, this voice input device 400 has four microphones 402-1 to 402-4 arranged so as to be able to collect the voices of all attendees, and in a direction different from that of the conference attendees. Microphone 402-5 arranged with directivity
, A microphone amplifier (MAMP) 403 that amplifies the output signals of the microphones 402-1 to 402-5, a room noise removal unit (NCAN) 404 that removes room noise, and a dereverberation unit (E) that removes reverberation.
CAN) 405, variable gain unit (AGC) 406 that removes noise from the microphone of a speaker who is not busy, mixer unit (VMIX) 407 that synthesizes output signals of each microphone, and voice that compresses and encodes voice. And a coding unit (VCOD) 408.

The four microphones 402-1 to 402-4 are arranged in consideration of directivity so that the voices of all the attendees can be collected, and pick up the voices of the conference attendees. Also a microphone
402-5 picks up the background noise signal.

A microphone amplifier (MAMP) 403 amplifies the output signal of each microphone to an appropriate level.

Room noise eliminator (NCAN) 404 The room noise eliminator 404 is a circuit that removes background noise that is always present in the conference room, and is picked up by the microphone 402-5 whose directionality is different from that of the conference attendees. Signal VB- with background noise removed from microphone sounds VA-1 to VA-4 by a noise canceller using the background noise signal as a reference signal
1 to VB-5 are generated.

FIG. 12 is a circuit diagram of the room noise elimination unit 404.

As shown in the figure, the room noise elimination unit 404 includes adaptive filters (NC1 to NC4) 404-1a to 404-1d and a subtractor.
404 to 2a to 404-2d and a learning prohibition control circuit 409.

The adaptive filters 404-1a to 404-1d generate pseudo noises Y _{N1 to} Y _N4 using the output signal VA-5 of the microphone 402-5 as a reference signal. Subtractors 404-2a to 404-2d are microphones 402-1 to 40-40 amplified by the microphone amplifier 403.
2-4 output signals VA-1 to VA-4 from the pseudo noise Y _N1
~ Y _N4 and remove the noise to remove residual signal VB-1
~ Outputs VB-4.

Further, the adaptive filters 404-1a to 404-1d are successively learned by, for example, a well-known learning identification method so that the correlation between the reference signal and the residual signal is eliminated. However, learning is disturbed if there is a transmitted voice or speaker output during this learning, and this is prevented by the learning prohibition control circuit 409 in the broken line in the figure.

This learning prohibition control circuit 409 includes microphones 402-1 to 40-40.
2-4 Level detection circuits (NLV1 to NLV5) for detecting the levels of the output signals VA-1 to VA-4 after amplification 409-1a to 409-1d
And level detection circuits (NLVR, NLVL) 409-2 and 409-3 for detecting the levels of the left and right speaker output signals SR and SL,
Maximum value detection circuit (MAX1) 409-4 for detecting the maximum value among the level detection circuits 409.1a to 409.1d, and the level detection circuit 4
An adder 409-5 that generates a sum signal of 09-2 and 409-3;
The maximum value detection circuit (MAX) that outputs the larger of the output signal of the maximum value detection circuit 409-4 and the output signal of the adder 409-5
2) 409-6 is compared with the output signal of the level detection circuit 409-7 and the output signal of the maximum value detection circuit 409-6, and the output signal of the level detection circuit 409-7 is the maximum value detection circuit 409-6. And a comparator 403-8 which outputs a control signal NADP which becomes “1” when the output signal is larger than the output signal of the comparator 403-8.

In the learning inhibition control circuit 409, the maximum value detection circuit 4
09-4 selects the maximum output signal of the microphones 402-1 to 402-4 after amplification, and the maximum value detection circuit
409-6 selects the larger signal between the output signal of the maximum value detection circuit 409-4 and the sum signal of the output signals SR and SL of the left and right speakers.

Thus, the comparator 409-8 determines that the level of the reference signal VA-5 output from the level detection circuit 409-7 is such that the transmitted voice VA-1.
~ The control signal NADP which becomes "1" only when it is larger than the maximum level of VA-4 and the level of the sum of the left and right speaker output signals SR and SL is output to the adaptive filters 404-1a to 404-1d, and the adaptive filter is output. 404-1a to 404-1d learn only when the control signal NADP is "1".

Echo Reverberation Unit (ECAN) 405 The dereverberation unit 405 causes howling and voice quality degradation due to reverberation due to echo generated by the sound emitted from the speaker being reflected by the wall or window in the conference room. The left and right speaker signals SR and SL are input and two-channel echo cancellers are used to generate reverberant signals VC-1 to VC-4.

FIG. 13 is a circuit diagram of this dereverberation unit.

As shown in the figure, the dereverberation unit 405 includes echo cancellers 405-1a to 405-1d and subtractors 405-2a to 405-2d.
And a double-talk detector 410.

The echo cancellers 405-1a to 405-1d generate pseudo reverberation signals Ye1 to Ye4 from the output signals SR and SL of the left and right speakers.

FIG. 14 is a circuit diagram showing the configuration of this echo canceller (for example, 405-1a). As shown in FIG. 14, this echo canceller 405-1a has two adaptive filters (AF1, AF2) 405-3, 405. -4 and an adder 405-5, the right speaker signal SR is input to the adaptive filter 405-3, the left speaker signal SL is input to the adaptive filter 405-4, and the two adaptive filters 405-3 , 405-4 output signals are added by the adder 405-5 to generate pseudo reverberation Ye1.

In FIG. 13, subtractors 405-2a to 405-2d subtract the pseudo-reverberation signals Ye1 to Ye4 from the signals VB-1 to VB-4 output from the room noise removal unit 404, and remove the reverberation residuals. The difference signals VC-1 to VC-4 are generated.

In addition, learning of adaptive filters 405-3 and 405-4
For example, by the well-known learning identification method, the reference signal SR,
This is realized by performing sequential learning so as to eliminate the correlation between SL and the residual signals VC-1 to VC-4.

Further, the double-talk detector 410 detects the double-talk state and prohibits the learning, because the learning is disturbed when the learning is performed during the double-talk in which both the speaker output and the transmitted voice exist.

The double-talk detector 410 includes level detectors (EVL1 to EVL4) 410-1a to 410-1d for detecting the levels of the output signals VB-1 to VB-4 of the room noise eliminator 404 and left and right speaker signals SL. , Level detector for detecting SR level (EVLL, EVL
R) 410-2, 410-3 and level detectors 410-1a to 410-1
It is composed of a maximum value detection circuit (MAX) 410-4 that removes and outputs the maximum of the output signals of d, an adder 410-5, and a comparator 410-6.

In this double root detector 410, the level detectors 410-1a to 410-1d are first detected by the maximum value detection circuit 410-4.
Of the output signals of the
5, the sum of the output levels of the left and right speaker signals SL and SR is calculated, and the output signal of the maximum value detection circuit 410-4 and the adder 410 are added.
The level of the output signal of -5 is compared by the comparator 410-6.

This comparator 410-6 generates a signal EADY that becomes "1" when the output signal of the adder 410-5 is smaller than the output signal of the maximum value detection circuit 410-4, and this signal EADY is echo canceller 405-1. Adaptive filter 405-1 in 1a to 405-1d
Input to 3, 405-4, ....

The learning of the adaptive filters 405-3, 405-4, ... Is prohibited when the signal EADY is “1”, that is, when the output level of the speaker is smaller than the maximum level of the voice output from each microphone.

Variable gain unit (AGC) 406 and mixer unit (VMIX) 407 The variable gain unit (AGC) 406 uses the speaker identification code TID provided from the speaker identification device 510 described later to generate four input signals V.
Of the C-1 to VC-4, it selectively amplifies the signal from the microphone whose direction is direct to the speaker currently speaking and removes unnecessary noise from the microphone. )
Reference numeral 407 is for adding the output signals of the variable gain unit 406.

FIG. 15 is a circuit diagram of the variable gain section 406 and the mixer section 407. The variable gain section 406 is a variable gain amplifier 406-1 to 406-4.
And a control unit 406-5, and the mixer unit 407 is an adder 407-1.

When the speaker identification signal TID given from the speaker identification device 510 is input, the control unit 406-5 operates as a variable gain amplifier other than the variable gain amplifier (for example, 406-1) corresponding to the speaker who is currently speaking. Lower the gain.

The adder 407-1 of the mixer unit 407 is a variable gain amplifier 406-1.
The output signals of 406-4 are added.

Voice Coding Unit (VCOD) 408 The voice coding unit (VCOD) 408 outputs the output signal V of the mixer unit 407.
In response to the selection signal VRUL of the coding / decoding algorithm sent from the control device 900, the data is compressed and coded according to the transmission speed, and an identifier is further added and output to the control device 900.

FIG. 16 is a circuit diagram of the voice encoding unit 408.

As shown in the figure, the voice encoding unit 408 includes a compression algorithm storage unit 408-1 in which a compression algorithm program is stored, a selection switch 408-2 for selecting a compression algorithm, and compression for selecting voice data. It is composed of an encoding unit 408-3 that compresses and encodes with an algorithm and an identifier adding unit 408-4 that adds control information.

The compression algorithm storage unit 408-1 includes a 64K ADPCM coding program 408-1a and a 32K ADPCM coding program 408.
-1b, 16KAPCAB encoding program 408-1c, and 8KAPC
The AB encoding program 408-1d is stored.

First, the selection switch 408-2 is responsive to the selection signal VRUL sent from the control device 900 to output the compression algorithm storage unit 408.
Select one encoding program in -1.

Next, the encoding unit 408-3 uses the encoding program selected by the selection switch 408-2 to generate the mixer unit 407.
The output signal V of is encoded.

Then, the identifier adding unit 408-4 adds the talker identifier (TID) and the device address (SAD) to the encoded signal, and outputs it as the transmission signal VTX to the control device 900.

Voice output device 450 The voice output device 450 converts an electric signal into a voice signal by a speaker and outputs the voice signal. As shown in FIG. 17, an identifier detection unit 451 for identifying control information and a voice for decoding voice data. A decoding unit (VDEC) 452, a stereo sound generation unit (STGEN) 453 that outputs stereo sound with a delay,
It is composed of a right amplifier (RAMP) 454 and a right speaker 455 for amplifying and outputting the right side sound, and a left amplifier (LAMP) 456 and a left speaker 457 for amplifying and outputting the left side sound.

The identifier detecting unit 451 identifies a voice frame from the frame identifier FID and identifies the partner terminal from the device address SAD.

FIG. 18 shows the structure of the voice decoding unit (VDEC) 452. As shown in FIG. 18, the voice decoding unit 452 is a compression algorithm storage unit 45 in which a decoding program is stored.
8 and a selection switch 459 for selecting a decryption program,
A frame decomposing unit 460 that decomposes a frame, and a decoding unit 461 that encodes audio data using a selected algorithm.
And RAMI46 that stores the decrypted data corresponding to the terminal
2 and RAMII463, and a switch 464.

The compression algorithm storage unit 458 includes a 64K ADPCM decoding program 458-1, a 32K ADPCM decoding program 458-
2, 16 KAPCAB decoding program 458-3 and 8 KAPCAB decoding program 458-4 are stored.

In addition, the selection switch 459 responds to the selection signal VRUL sent from the control device 900, by the compression algorithm storage unit 45.
Select one decryption program in 8.

RAMI462 and RAMII463 store data used for decoding according to the partner terminal.

First, the frame decomposing unit 460 decomposes the frame to extract the partner terminal identification data CR, and outputs this to the switch 464.

Next, the switch 464 sets the RAMI
And RAMII463, and outputs the data used for decoding to the decoding unit 461.

Further, the decoding unit 461 performs decoding by the compression algorithm (that is, the decoding program) selected via the selection switch 459, generates the audio data VRX, and outputs it to the stereo audio generation unit 453.

Now, the stereo voice generator (STGEN) 453 is a voice signal VRX.
Which produces more stereo audio signals SR and SL.
Figure 19 shows its configuration. As shown in the figure, the stereo sound generation unit 453 includes shift registers 470 to 47 for delay.
5 and switches 476 and 477 for selecting the delay amount.

The delay shift registers 470 to 475 delay the audio signal VRX. The switches 476 and 477 select voice signals having different delay amounts according to the speaker identification signal TID sent from the control device 900, and output left and right voice signals.

The left and right audio signals SL and SR generated by the stereo audio generation unit 453 are output to the left amplifier (LA) as shown in FIG.
Speaker 45 via MP) 456 and right amplifier (RAMP) 454
It is input to 7,455 and the audio signal is reproduced.

Speaker Identification Device 510 FIG. 20 is a block diagram of the speaker identification device 510.

This speaker identification device 510 is a device for identifying the speaker using code correlation when there is a plurality of people who speak among them, and includes microphones 512 and 514 and amplifiers 516 and 5
18, code detectors 520 and 522, 64-tap shift register 524, 32-tap shift register 526, 64 exclusive OR circuits 528 as correlation detectors, and 64 up / down counters 530-1 to 530-1. 530-64, coder 532, OR circuit 534 and latch 536.

FIG. 21 shows this speaker identification device 51 when there are five humans.
0 shows an example of installation, and as shown in the figure, microphone 5
12, 514 are installed on the left and right of the device.

The microphones 512 and 514 convert the voice uttered by the speaker into electric signals, and the amplifiers 516 and 518 amplify the converted first channel voice signal and second channel voice signal to appropriate levels.

Next, the sign detectors 520 and 522 set the sign to "1" if the signals output from the amplifiers 516 and 518 are (+), and sign the sign "0" if they are (-). ) Is output.

However, in the case of this embodiment, the audio signal is represented as a signal at the time of k samples sampled at 8 kHz in a frequency about twice the band, for example, in the band of 0 to 3.4 kHz.

Next, the 64-tap shift register 524 shifts the code of the second channel every 8 kHz. The 32-tap shift register 526 shifts the code of the first channel and delays it by 32 samples.

Then, the exclusive OR circuit 528 outputs the output SIGNR (k−i +) of the i-th stage of the shift register 524 as shown in the equation (1).
1) and the output of the sample shift register 526, SIGNL
Sign correlation is obtained with (k−32).

Zi = SIGNR (k-i + 1) + SIGNL (k-32) (1) The up / down counters 530-1 to 530-64 count up this correlation output Zi when Zi = 1 and when Zi = 0. The time is counted down for averaging, and this counter overflows to detect the tap position with the highest correlation.

That is, the coder 532 has up / down counters 530-1 to 53-53.
Find the position of overflow 0-64, encode the position information into 6 bits, and latch the upper 3 bits 53
Output to 6.

Also, the OR circuit 534 clears each of the up / down counters 530-1 to 530-64 when an overflow occurs and sends a signal to the latch 536.

The latch 536 captures the 3-bit information when receiving this signal and sends it to the control device 900 as a speaker identification signal TID.

22 (A), (B), (C), (D), and (E) are 21st.
It shows an impulse response when the speakers A, B, C, D, and E shown in the figure speak, and the voice arrival time to the left and right channels differs depending on the person's speech. The speaker can be identified by correlating the input signals.

That is, the left and right microphones are installed as shown in Fig. 21,
When the impulse response is obtained from the transfer characteristics between the channels, the difference in the arrival time of the voice to the left and right channels varies depending on the position of the speaker, as shown in FIG.

In the case of FIG. 21, since the delay is within ± 4 msec, if the correlation detector sets the voice band to 3.4 kHz and one sample is 125 μsec, it suffices to process a total of 64 samples of 32 samples before and after.

Therefore, the correlation detector of FIG. 20 may be 64 samples long.

Storage Device 600 This electronic conferencing system includes a storage device 600 for storing conference materials and the like. As the storage device 600, a floppy disk device or the like is used in consideration of recording, erasing, and portability.

The configuration of the storage device 600 is shown in FIG. The storage device 600 includes a control unit 611 that controls writing and reading, an identifier detection unit 612 that identifies frame control information, a writing buffer unit 613, a storage unit 614, and a reading buffer unit 615. , And an identifier adding unit 616 for checking frame control information.

The operation of the storage device 600 will be described below.

First, the control device 900 sends control information and image information to the identifier detection unit 612. This control information includes frame identifier (FID), device address (SAD), page control (PAGE)
However, the image information is only one type of still image data.

Next, the identifier detection unit 612 confirms whether the still image data is the still image data by the frame identifier (FID) in the control information, and then detects whether the still image data is written or read based on the content of the page control (PAGE). Then, the screen number is detected and the instruction content is sent to the control unit 611.

Then, the control unit 611 controls each unit according to the instruction content of the identifier detection unit 612 and controls writing and reading of still image data.

In the case of writing still image data, the still image data is sent from the identifier detecting unit 612 to the buffer unit 613 with control information such as page control (PAGE) attached. This buffer unit 613
Transmits and writes still image data to and from storage unit 614 while absorbing the difference between the transfer rate of still image data and the writing speed of storage unit 614.

In the case of reading the still image data, the still image data of the screen number designated by page control (PAGE) is read from the storage unit 614 and sent to the buffer unit 615.

The buffer unit 615 absorbs the difference between the read speed of the storage unit 614 and the transfer speed of the still image data, and the storage unit 614 reads the still image data and sends it from the buffer unit 615 to the identifier adding unit 616.

The identifier adding unit 616 checks whether the control information and the still image data read from the storage unit 514 have the designated screen number, and further confirms that the control information such as the frame identifier (FID) is correctly added. , To the control device 900.

The usage of the storage device 600 is to send the conference material to the other terminal prior to the conference so that both parties can store the same material and retrieve the material according to the instruction of the own terminal or the other terminal, or to correct the correction during the conference. It is also possible to accumulate the results of the above and keep them as a minutes.

Transmission control device 700 The frame information generated in each device in the electronic conference system is packetized by the transmission control device 700 as shown in FIG. 24 and transmitted to another conference unit system. A frame transmitted from another conference unit system is received by the transmission control device 700 and distributed to each device in the system.

The electronic conference system of this embodiment is 2B + D (B; 64 kbps,
It is connected to the ISDN network by basic access (D; 16 kbps). That is, the transmission control device 700 has a 4-wire home bus interface, which is being standardized by CCITT, and can access the home bus through an information outlet.

FIG. 25 is a block diagram of the transmission control device 700.

This transmission control device 700 includes a line terminating device (NT1) 701 which is an interface with an ISDN network and a physical layer interface (INF) 70 which is an interface with a home bus.
2. Packet generation / decomposition circuits (PAD) 703 and 704 for generating and disassembling packets, a telephone interface (TEL) 705 to which a handset is connected, a signaling control unit (SIG) 706 for controlling call connection, etc. Input / output buffers (BUF) 707 to 710, handset 711, interface (IF) 712 connected to the system bus, mode switching switch 713, OR circuit 714 for detecting buffer overflow, and keyboard It consists of 715 and. Reference numeral 716 represents ISDN network codes 717 and 718 represent T lines and R lines of a four-wire home bus.

A 4-wire home bus composed of T lines 717 and 718 is connected to an ISDN network via a line terminating device (NT1) 701.

The physical layer interface (INF) 702 is an R line 71.
The clock is extracted from the received signal of 8 and the frame synchronization is established, and the data of B1, B2 and D channels are received. At the same time, the transmission data for the B1, B2, and D channels are input and transmitted to the T line 717.

Further, the switch 713 is switched between a normal telephone mode and a teleconference mode by a control signal MODE from the conference support unit 991, which will be described later. In the telephone mode, B1,
B2 channel via the telephone interface 705,
Connected to handset 711. This telephone has two lines B1 and B2, and is assigned two telephone numbers #A and #B.

Then, the signaling control unit (SIG) 706 receives information such as normal telephone call setup and cancellation from the keyboard (KEY) 715, and this information is sent to the ISDN network 716 via the D channel, and the B1 and B2 channels. The line connection is made.

On the other hand, when switch 713 is in teleconference mode, B
The data of each channel of 1 and B2 is sent to the packet generation / decomposition circuit (PAD) 703 and 704, and after the header and the error detection code are removed, it is passed through the buffers (BUF) 707 to 710 and the interface (IF) 712. , Conference unit system sent to internal bus.

The buffers 707 and 708 are prohibited from outputting the buffer when the output device in the conference unit system is busy. In addition, if these buffers are full, control signal C
1 and C3 inform the packet generation / decomposition circuits (PAD) 703, 704, and the packet generation / decomposition circuits (PAD) 703, 704
A management packet indicating unreceivable is generated, sent to another terminal, and the buffer full is notified.

On the other hand, when a packet that cannot be received is sent from the partner terminal, the packet disassembly generation circuits (PAD) 703 and 704
The output of the buffers 709 and 710 is prohibited. When this buffer becomes full, the control signal TBUSY is notified to the conference support unit 991 described later via the OR circuit 714, and the conference support unit 991
Based on this information, the input device in the electronic conference system is instructed to prohibit input.

Next, the state of call setting of this electronic conference system will be described with reference to FIGS. 26 and 27.

In FIG. 26, the broken line is the line of a normal PCM telephone,
The solid line is a line for packet communication via the B1 and B2 channels performed in the teleconference mode. At the initial stage, as shown in FIG. 26 (a), the electronic conference system A,
Call setups are made with ordinary telephones, each using the B1 channel. Then, as shown in FIGS. 26 (b) and 26 (c), the new participating system C sets up the call with the A and B systems using the same ordinary telephone (B2 channel), and then all the systems are set up for teleconference. By switching to the mode (shown in FIG. 26 (d)), teleconference by packet data is started. This switching is performed by exchanging a supervisory frame indicating the start of packet communication between the terminals. The termination of the teleconference mode is also performed by exchanging a supervisory frame indicating the end of packet communication.

Another procedure for call setup is shown in FIG. In this case, it is assumed that a teleconference using the B1 channel has started between the systems A and B, as shown in FIGS. At this time, the new participation system C makes a normal call to the B2 channel telephones of A and B, and after the line is set up, a three-party teleconference becomes possible.

As described above, this electronic conference system can easily carry out a three-party teleconference without preparing a multipoint control unit on the network side.

State Detection Device 800 The state detection device 800 detects the presence or absence of conference attendees, and its configuration is shown in FIG.

This state detecting device 800 includes n detecting units 804-1 to 804-n corresponding to n seats 802-1 to 802-n and a detecting unit 804.
A coder that encodes the outputs of −1 to 804 to n and sends them as a control signal SDET to the conference support unit 991 in the control device 900 described later.
It consists of 806 and.

This detection unit 804-i is an oscillator (OSC) 80 that generates ultrasonic waves.
8, a detector (DET) 810 for detecting the ultrasonic waves reflected by a person or a wall, and a measuring circuit 812.

FIG. 29 is a detailed circuit diagram of the detection unit 804-i. The measurement circuit 812 includes a pulse generation circuit (PG) 814 that generates the pulse shown in FIG. 30 and an inverter 81 that inverts the signal of the detector 810.
6, an AND circuit 818 that ORs the output of the pulse generation circuit 814 and the output of the inverter 816, and a timer (TIMER) 820 that measures the time when the output of the AND circuit 818 is "1".
And a preset value T for the output signal C of the timer 820
When C <T, the output signal STi is set to "1", and the comparator (COMP) 822 indicates that the user is seated.

Next, the operation of this state detecting device 800 will be described.

First, when the palace generating circuit 814 generates a palace as shown in FIG. 30 (a), the oscillator 808 outputs the pulse of "1".
Only at the time of oscillating, ultrasonic waves as shown in FIG. 30 (b) are generated.

This ultrasonic wave is reflected by the person when the person is sitting on the seat and is reflected by the wall when the person is not sitting on the seat, and is delayed by the delay time τ to the detector 810 as shown in FIG. 30 (c). arrive.

When the detector 810 detects this ultrasonic wave, its output signal becomes "1" (Fig. 30 (d). This is the inverter.
Since it is inverted by 816, and the output signal of the pulse generation circuit 814 and the output signal of the inverter are ORed by the AND gate 818, the output of the AND gate 818 is as shown in FIG. 30 (e). .

That is, the output signal of the AND gate 818 becomes a signal which becomes "1" only during the delay time τ. The delay time τ is small when a person is seated in the seat and is large when there is no person.

Therefore, the timer 820 measures the time when the output signal of the AND gate 818 is “1”, and inputs it as the signal C to the comparator 822.

The comparator 822 compares the signal C with the set value T. This set value T is determined in advance so that the signal C is smaller than T when a person is seated, and the signal C is larger than T when a person is not seated. When <T, the output signal STi of the comparator 822 becomes "1", and when C> T, the output signal STi of the comparator becomes "0", and this output signal STi indicates whether or not a person is seated.

In FIG. 28, the coder 806 includes the detection units 804-1 to 804-
The output signal STi (i = 1 to n) sent from n is coded and sent to the conference support unit 991.

Operation Input Device 850 FIG. 31 is a block diagram of the operation input device 850.

This operation input device 850 includes an operation input unit 851 for inputting each operation instruction, an input analysis unit 852 for analyzing the operation instruction, and an input / output interface unit 853 for connecting this device and the control device 900. And a display analysis unit 854 for checking display contents and a display unit 855.

Hereinafter, this operation input will explain the operation of the device 850.

The input analysis unit 852 analyzes various operation switch inputs from the operation input unit 851, assembles them into code signals, and sends them as operation signals (OPSG) to the control device 900 via the input / output interface unit 853.

From the control device 900, a response signal to this operation input and a signal indicating the operating state of this conference system are sent to the transmission interface section 853. Next, the display analysis unit 8
The signal encoded at 54 is analyzed and displayed 855
Sent to and displayed.

FIG. 32 shows the operation input unit 851 and the display unit 855 of the operation input device 850.
3 shows an example of the appearance of the.

"POW" of the operation input unit 851 is the device power switch, "Meeting"
Is a switch for switching between the telephone mode and the teleconference mode, which are the operation modes of the device, and "Record" is the graphic display device (1)
A switch to instruct the storage device 600 to record the display content of
"Read" is a switch for instructing reproduction display from the storage device 600 to the graphic display device (1) 160, "Copy" is a switch for instructing copy output of the display contents of the graphic display device (1) 160 to the hard copy device 260, "A" to "D" are switches for instructing from which terminal the device address (SAD) is set and which data is input at the time of "reading", and "1" to "0" 10
Each of the numeral keys is a page number indicating switch at the time of “read”. The display showing the response to each operation input is the 32nd
In the example shown in the figure, an illuminated switch in which an indicator LED is incorporated in the switch can be considered.

The display unit 855 is composed of an LED, and has a speaker switching signal (PID) and a speaker switching signal (TI) indicating the state of input data from each input device.
D) and monitor the connection status of the line.

Control Device 900 FIG. 33 is a block configuration diagram showing the configuration of the control device 900.

Data from each input device and data to each output device are transmitted via the internal data bus 901.

That is, the drawing data from the drawing input devices 100a, 100b is multiplexed by the multiplexing unit 911 and the input interface unit 9
12, sent to the data bus 901 via the data buffer 913. Here, the alphabetical letters a, b ... Indicate that a plurality of devices having the same function are connected (the same applies hereinafter).

Still image data from the still image input device 200 is transferred to the data bus via the input interface unit 922 and the data buffer 923.
Sent to 901.

The moving picture data from the moving picture input device 300 is transferred to the data bus 901 via the input interface section 932 and the data buffer 933.
Sent to.

The voice data from the voice input device 400 is transferred to the data bus 901 via the input interface unit 942 and the data buffer 943.
Sent to.

A frame identifier (FID) indicating which input device the data frame is from is added to these input data.

For the data output, first, the drawing data is transmitted from the data bus 901 through the output interface unit 916 to the flat panel display device 150.
sent to a, b ...

Similarly, still images and drawing data are sent from the data bus 901 to the hard copy device 260 via the output interface unit 926.
To the graphic display device (1) 160 via the output interface unit 917, the moving image data is sent to the graphic display device (2) 350 via the output interface unit 936, and the audio data is output to the output interface unit. It is sent to the audio output device 450 via 946.

Furthermore, the graphic display device (1) 160 can also send still images and drawing data to the data bus 901 through the input / output interface unit 917.

Further, the stored data from the storage device 600 is sent to the data bus 901 via the file interface 961, and conversely, when stored, it is sent from the data bus 901 to the storage device 600 via the file interface 961.

The frame controller 990 switches these data transmission destinations, selects data, and multiplexes.

When sending the data group collected in the frame controller 990 to the line, it is sent to the transmission device 700 via the transmission interface 972. The data group sent from the line via the transmission device 700 is the transmission interface 972.
To the frame controller 990 via.

Here, the data buffers 913, 923, 933, and 943 are used when the transmission to the data bus is temporarily interrupted due to the switching of the data transmission destination or the selection of the data, or when the transmission line capacity is transiently exceeded due to the multiplexing. It is for storing data when you are doing.

The conference support unit 991 controls the frame controller 990. The conference support unit 991 is a speaker identification device 51.
Signals from 0, the state detection device 800, the operation input device 850 and the like are used to instruct the frame controller 990 such as data selection and destination.

The amount of data input to this control device 900 is as follows.

First, the drawing data from the drawing input devices 100a, b ... Is 300bps.
Because of the low speed, even if multiple devices are input at the same time, the data speed is about several kbps. Therefore, the drawing data can be input at the same time, and are multiplexed and taken in by the multiplexing unit 911.

Further, the still image data from the still image input device 200 is about 16 kbps. The video data from the video input device 300 is about 32 kbps, and if the transmission capacity of the line is about 64 kbps, it is impossible to input multiple units at the same time considering transmission of other data such as voice to send to the line. , Any one is selected by a speaker switching signal (PID) from the conference support unit 991.

Furthermore, the audio data from the audio input device 400 is μ-LAWPC.
It is used by switching as needed from 64kbps to 8kbps by methods such as M, ADPCM, and APCAB. Switching is performed by the conference support unit 991 as a voice coding rule signal (VRUL) by the voice input device 400.
Be instructed to. This signal is output by the conference support unit 991 depending on the amount of other data.

The input / output devices with alphabets a, b ... Are installed in the hands of the attendees of the conference.

With the above data volume, the conference support unit 991 advances the conference.

The operation of the control device 900 will be described below.

First, a speaker identification signal (ID) indicating who is the speaker is input from the speaker identification device 510. Based on this signal, the conference support unit 991 outputs a transmission switching signal (TID) to the voice input device 400 so as to select the voice of the speaker. Similarly, a speaker switching signal (PID) is output to the moving image input device 300 so as to capture the speaker.

Audio data and moving image data from these input devices are multiplexed by the frame controller 990 and sent to the transmission device 700 via the transmission interface 972. A speaker switching signal (PID) indicating the speaker is added to the data frame in the video data, and a voice coding rule signal (VRUL) and a speaker identification signal (TID) are added to the data frame in the audio data. Has been done.

Along with this, the video data is output in the interface section 936.
It is sent to and displayed on the graphic display device (2) 350 via.
This moving image data is data compressed by a method such as frame correlation. The speaker can determine that he / she is selected because he / she is displayed on the graphic display device (2) 350.

The document original for the conference is input from the still image input device 200. The still image data read here is data compressed by a method such as frame correlation. The still image data is stored in the storage device 600 via the file interface 961 before the conference, or sent to the other terminal through the line while being stored in the storage device 600 during the conference.
A page control signal (PAGE) is added when storing.

Further, during the conference, the stored data is read from the storage device 600 via the file interface 961 and is output through the output interface unit 916 and the input / output interface unit 917 to the flat panel display devices 150a, 150b and the graphic display device (1) 160, respectively. Is displayed.

These instructions are given from the operation input device 850 and input as an operation signal (OPSG).

The conference support unit 991 receives the signal from the frame controller 9
Instruct 90 to send data and add page control signal (PAGE).

When instructing the partner terminal to display the stored data via the transmission interface 972, the page control signal (PAGE) is used. When the data is stored, the contents of the document original are displayed on the graphic display device (1) 160 and the drawing input devices 150a, 150b, and the operator can perform the work while checking.

When corrections are added to the displayed information such as figures and characters, the drawing input devices 100a, 100b ... When the drawing input device 100a is used now, the operator determines the input position by relying on the display of his own flat display device 150a.

This drawing data is sent to the data bus 901 via the multiplexing unit 911, the input interface unit 912, and the data buffer 913. The frame controller 990 outputs the drawing data to the front surface display device 1 via the output interface unit 916.
50a, 50b, ... And then to the graphic display device (1) 160 via the input / output interface unit 917. At the same time, the data is sent to the transmission device 700 via the transmission interface 972.

Since the drawing data has a small amount of information and can be sent at a low speed, a plurality of operators can simultaneously input to the drawing input devices 100a, 100b.

When the information displayed on the graphic display device (1) 160 is output to the hard copy device 260 or stored in the storage device 600, the output interface unit 926 and the file interface are input from the input / output interface unit 917 via the data bus 901. This is done by sending display data to the 961.

The status detection signal (SDET), which indicates the number of participants in the meeting and the number of people leaving the meeting during the meeting, is sent from the status detector 800 to the meeting support unit 991.
Entered in. In the conference support unit 991, the frame controller 990 selects data input of each input device based on this signal.
To the moving image input device 300, or to instruct the moving image input device 300 to erase the display of the person leaving the seat from the display of the graphic display device (2) 350.

The above has described the operation when data is input from each input device. The operation when data is sent from the partner terminal (there may be a plurality) is as follows.

The data group from the transmission device 700 is the transmission interface 972.
To the frame controller 990 via. Here, it is distributed according to each data. The audio data is sent to the audio output device 450 through the output interface unit 946, and the drawing data and the still image data are both output through the output interface unit 916 and the plane display devices 150a, b ... It is sent to the display device (1) 160. The moving image data is sent to the graphic display device (2) 350 via the output interface unit 936. In the case of the accumulation instruction, the still image data is stored in the file interface 961.
To the storage device 600 via.

Furthermore, what is sent from the line via the transmission device 700 is the above-mentioned data group and information regarding these controls. Control information is attached to the beginning of each data frame, and includes a frame identifier (FID), device address (SAD), page control signal (PAGE), talker switching signal (PID), and voice coding rule signal (VRUL). , A transmission switching signal (TID), etc.

For transmission control, the following signals are exchanged between the conference support unit 991 and the transmission device 700 via the transmission interface 972.

First, a mode signal (MODE) for switching from the telephone mode to the remote conference mode is passed from the conference support unit 991 to the transmission device 700. When the buffer in the transmission device 700 is full, a transmission data transfer prohibition signal (TBUSY) is output to the conference support unit 991.

The device address (SAD) added to the data frame to be transmitted is attached by the frame controller 990 according to the instruction from the conference support unit 991.

The data buffers 913, 923, 933, and 943 work in the following cases.

Drawing input device 10
It has 0a, b ... And there is a possibility that multiple inputs can be made at the same time. At the time of ordering these, the drawing is stored in the data buffer 913. In particular, when partially erasing the display contents when correcting it, it is necessary to temporarily suppress other inputs, and at this time, they are stored in the data buffer 913.

Further, the still image input may be stored in the storage device 600 while being displayed on the graphic display device (1) 160. For example, when the display of the figure display device (1) 160 is copied during a conference and a new original is displayed subsequently, when both operations are performed at the same time, the copy of the still image data is temporarily performed in order to give priority to the copy. It will be stored in buffer 723.

On the other hand, when someone speaks, the voice data and video data of that person will be sent to the partner terminal. At this time, the video data is 32
About kbps, voice data by ADPCM is about 32 kbps, and the line capacity is full. However, if this is a message during still image input, the line capacity will be exceeded due to the still image data. Therefore, it becomes necessary to slow down the voice data by using the voice coding rule signal (VRUL). Audio data is stored in the data buffer 943 during the transition period until the switching to the low speed.

In addition, at least two people may have a conversation during a teleconference. At this time, the graphic display device (2) 35
At 0, the video data of both the speaker of the own terminal and the speaker of the other terminal are sent and displayed. Therefore, both data will be sent in a time division manner, and the own terminal moving image data will be accumulated in the data buffer 933.

The transmission data transfer prohibit signal (TBUSY) from the transmission device 700
Is output, the data buffers 913,923,933,
The 943 accumulates data when there is an input from the corresponding input device.

FIG. 34 shows a frame structure of data input / output to / from the control device 900. There are five types of data frames: still image frame (1) 992, still image frame (2) 993, drawing frame 994, moving image frame 995, and audio frame 996. Each data frame is composed of control information at the head and data subsequently.

The head of the control information is a frame identifier (FID), which is added by each input device and input to the control device 900.
The subsequent device address (SAD) is not added in each input device, and this part is sent as an empty (NULL) and added by the frame controller 990 when transmitting.
The third and subsequent control information is added as input information or in the control device 900 as needed.

The structure of each data frame will be described below along with the data flow.

The still image frame (1) 992 is sent from the still image input device 200 through the frame controller 990 to the graphic cover device (1) 160, the flat panel display devices 150a, 150b, ... And the storage device 600. At this time, the page control signal (P
AGE) is added. The page control signal (PAGE) is incremented in page number every time still image data is stored in the storage device 600 for one page of the document original. At the same time, it is sent to the partner terminal by the transmission device 700. At the partner terminal, the transmitted still image frame (1) 992 is displayed on the graphic display device (1) 1
60 and the plane display devices 150a, 150b and the storage device 600.

When reading and displaying the stored still image data, the frame controller 990 sends the still image frame (2) 993 to the storage device 600 to instruct the reading of the data. As a result, the corresponding still image frame (1) 992 is read from the storage device 600, and passes through the frame controller 990 and the graphic display device (1) 160 and the flat display devices 150a, 150b, ...
Sent to. At this time, the device address (SAD) and the page control signal (PAGE) indicate which page of the still image data input from which terminal. In order to display the same on the other party's terminal, the still image frame (2) 993 is transmitted by the transmission device 700.
Sent through.

When the display contents of the graphic display device (1) 160 are accumulated, the still image frame (2) 993 is sent to the graphic display device (1) 160 to instruct the data transfer. As a result, the display content of the graphic display device (1) 160 is sent to the frame controller 990 as a still image frame (1) 992, where the device address (SAD) and page control signal (PAGE) are added to the storage device 600. Sent. Similarly, the still picture frame (2) 993 is transmitted via the transmission device 700 to be stored in the partner terminal.

Display contents of graphic display device (1) 160 to hard copy device 2
When outputting to 60, data transfer is instructed in still image frame (2) 993. As a result, the display content is sent to the frame controller 990 as a still image frame (1) 992, and transferred from here to the hard copy device 260.

In the above case, the page control signal (PAGE) is set to 4 by writing and reading the still image frame (1) 992 to the storage device 600, reading from the graphic display device (1) 160, and page number instruction.
Contains one piece of information.

The drawing frame 994 is sent from the drawing input devices 100a, b ... To the graphic display device (1) 160 and the flat display devices 150a, b. At the same time, it is sent to the partner terminal by the transmission device 700. The drawing frame 994 sent from the partner terminal is the graphic display device (1) 160 and the flat display device 15
0a, b ... The data portion of the drawing frame 994 includes drawing reproduction, cursor, partial erasing, and full erasing as commands indicating the processing contents of the drawing together with the coordinates, and the display reproduction is performed according to this command.

The moving picture frame 995 is sent from the moving picture input device 300 through the frame controller 990 to the graphic display device (2) 350, and at the same time, sent to the partner terminal via the transmission device 700. A speaker switching signal (PID) indicating a speaker is added to the moving image frame 995 as control information, and the signal and the device address (SAD) indicate the change point of the display content. Similarly, on the partner terminal, the video frame 9 is displayed on the graphic display device (2) 350.
95 is sent.

The voice frame 996 is sent from the voice input device 400 to the partner terminal via the frame controller 990 and the transmission device 700. The partner terminal sends the transmitted voice frame 996 to the voice output device 450. In the voice frame 996, a speaker identification signal (TID) indicating who is the speaker and a voice encoding rule signal (VRUL) indicating which voice encoding method the data of this voice frame is encoded. Has been added. The audio output device 450 of the partner terminal controls the sense of position of the stereo sound source at the time of audio reproduction and selects the decoding system at the time of reproduction, based on these control information.

[Effects of the Invention] As described in detail above, according to the present invention, one speaker among a plurality of conference attendees present in a conference room is displayed as a moving image in one area on one screen divided into a plurality of areas. By doing so, the speaker and the other attendees can be automatically distinguished and displayed, and the operator is not required.

[Brief description of drawings]

1 is a block configuration diagram of an electronic conference system, FIG. 2 is a block configuration diagram of a drawing input device, FIG. 3 is a block configuration diagram of a flat display device, and FIG. 4 is a block configuration diagram of a graphic display device (1). FIG. 5 is a block diagram of a still image input device, FIG. 6 is a block diagram of a hard copy device, and FIG.
FIG. 8 is a block configuration diagram of a moving image input device, FIG. 8 is a block configuration diagram of a moving image encoding unit, FIG. 9 is a block configuration diagram of a graphic display device (2), and FIG. 10 is a block configuration diagram of a moving image decoding unit. , FIG. 11 is a block configuration diagram of a voice input device, FIG. 12 is a block configuration diagram of a room noise removal unit, FIG. 13 is a block configuration diagram of a reverberation removal unit, FIG. 14 is a block configuration diagram of an echo canceller, FIG. 15 is a circuit configuration diagram of a variable gain unit and a mixer unit, FIG. 16 is a block configuration diagram of a speech encoding unit, FIG. 17 is a block configuration diagram of an audio output device, and FIG. 18 is a block configuration of a speech decoding unit. Fig. 19, Fig. 19 is a block configuration diagram of a stereo sound generation unit, Fig. 20 is a block configuration diagram of a speaker identification device,
FIG. 21 is a diagram showing an installation example of a speaker identification device, FIG. 22 is an impulse response diagram, FIG. 23 is a block configuration diagram of a storage device, and FIG.
FIG. 24 is a configuration diagram of a transmission packet, FIG. 25 is a block configuration diagram of a transmission identification control device, FIGS. 26 and 27 are explanatory diagrams showing a state of call setting, and FIG. 28 is a block configuration diagram of a state detection device. FIG. 29 is a block configuration diagram of a detection unit, FIG. 30 is a signal waveform diagram of each unit of the state detection device, FIG. 31 is a block configuration diagram of the operation input device, and FIG. 32 is an external view of the operation input device. FIG. 33 is a block configuration diagram of a control device, FIG. 34 is a data frame format diagram, and FIG. 35 is a block configuration diagram of a conventional electronic conference system. 100: Drawing input device 150: Flat display device 160: Graphic display device (1) 200: Still image input device 260: Hard copy device 300: Video input device 350: Graphic display device (2) 400 …… Voice input device 450 …… Voice output device 510 …… Speaker identification device 600 …… Storage device 700 …… Transmission device 800 …… State detection device 850 …… Operation input device 900 …… Control device

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Ryoichi Nakamura 3-1-1 Asahigaoka, Hino-shi, Tokyo Inside the Toshiba Hino Plant (72) Inventor Hirokazu Sakano 3-1-1 Asahigaoka, Hino-shi, Tokyo Toshiba Corporation Hino Factory (72) Inventor Masayoshi Aihara 3-1-1 Asahigaoka, Hino-shi, Tokyo Toshiba Corporation Hino Factory

Claims (3)

[Claims] 1. A plurality of image pickup means for picking up a moving image of each of a plurality of conference attendees and a plurality of audible sound input means for inputting an audible sound, and transmission of moving image data and audible sound data from each means. In an electronic conference system in which a plurality of conference unit systems are connected via a line, identification means for determining a speaker from among the plurality of conference attendees and identification information indicating the speaker determined by the identification means are provided. Transmission means for generating and transmitting this identification information via a line, receiving means for receiving the identification information from the line, and dividing one screen into a plurality of areas and corresponding to the identification information received by the receiving means Based on the video data of the speaker, the speaker is displayed as a video in a predetermined area of the divided screen, and the other attendees of the conference are displayed based on the video data of the conference attendees other than the speaker. An electronic conferencing system, wherein the conference unit system is provided with display means for displaying a still image in another area of the divided screen. 2. The electronic conference system according to claim 1, wherein the moving image data and the identification information input to the display means are obtained from another conference unit system via a line. 3. A plurality of image pickup means for picking up a moving picture of each of the plurality of conference attendees and a plurality of audible sound input means for inputting an audible sound, and moving picture data and audible sound data from each means are transmitted. In an electronic conference system in which a plurality of conference unit systems are connected via a line, identification means for determining a speaker from among the plurality of conference attendees and identification information indicating the speaker determined by the identification means are provided. Transmission means for generating and transmitting this identification information via a line, receiving means for receiving the identification information from the line, and dividing one screen into a plurality of areas and corresponding to the identification information received by the receiving means The speaker is displayed as a video in a predetermined area of the divided screen based on the video data of the speaker, and the input of the video data and the identification information of the conference attendees other than the speaker is interrupted. An electronic conferencing system, wherein the conferencing unit system is provided with display means for displaying the other conferees as still images in another area of the divided screen based on the final moving image data. .