JP2008061060A - Conference system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a conference system which forms a pseudo environment close to a face-to-face conference environment, has a presence, and can materialize efficient conference communications. <P>SOLUTION: A plurality of conference devices S, A, and B to M connected to an IP network NT comprise a main processor 2 for controlling an operation of the entire device, a speech/image data processor 3 connected to a picture/speech input device 7 and a speech output device 8 for processing speech data and picture data, a display data controller 4 connected to a display device 6 for controlling to display data, and a response auxiliary device 5 for executing an input and output of an expression of intent. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a conference system that is connected to an IP network and performs audio conferences and video conferences between conference devices. In particular, additional information such as the setting status and operation status of conference devices participating in the conference can be confirmed. It is an object of the present invention to provide a conference system with a sense of presence by enabling participants to express their intentions regarding the content of the conference.

  In the conventional conference system, when the receiving conference apparatus receives the audio signal from the transmitting conference apparatus, the conference system generates audio level information indicating the audio level of the audio signal, and the audio level information is transmitted to the transmitting side via the network. Japanese Patent Application Laid-Open No. 2004-133867 discloses a configuration in which audio level information is received by the conference apparatus on the transmission side and the audio level is visually displayed by the conference apparatus on the transmission side.

  As a result, the speaker of the conference device on the sending side can directly confirm that his / her voice is transmitted to the other party (the conference device on the receiving side) from the visualized voice level. It can be done.

  In addition, the conventional conferencing system confirms in real time with the conferencing device on the transmitting side how the audio / image data transmitted from the conferencing device on the transmitting side has reached the conference device on the receiving side. When the conference device receives the voice / image data, it generates quality information indicating the reception state of the voice / image data (the voice data is the output level from the speaker, and the image data is the number of packets). “Patent Document 2” discloses a composite image generated by combining an image and transmitted to a conference device on the transmission side via a network.

  “Patent Document 2” further describes that not only the quality information but also a composite image obtained by combining the intention display of the conference participant with an image using an icon or a fixed message can be transmitted.

As a result, the speaker of the conference device on the sending side can recognize from the screen display of quality information that his / her voice and image are transmitted to the other party (the conference device on the receiving side). The intention expression can also be confirmed.
JP 2004-186870 A JP 2005-269498 A

  In communication using a conference device, there is always anxiety about whether the voice of the speaker (including images in the case of a video conference) is transmitted to the other party, and the number of participants in the conference increases (multiple locations). This tendency becomes even more pronounced.

  Therefore, in order to eliminate such anxiety, the meeting organizer must confirm in advance whether all the participants can hear the sound or see the image before entering the meeting. Don't be.

  The conventional conference system disclosed in "Patent Document 1" or "Patent Document 2" generates an audio level from audio data on the transmission side received on the reception side, or generates quality information from audio data and image data. By sending the audio level and quality information to the sender and displaying them on the conference device on the sender side, the sender confirms that his / her voice and images have reached the receiver (conference device) reliably. However, there are cases in which the receiver on the receiving side (conference device) does not always recognize voice and images properly.

  For example, audio data or image data has arrived at the receiving conference device, but in the case of audio, the speaker is turned off, the volume control volume position is inappropriate, and the audio cannot be heard or is difficult to hear There is.

  In addition, although the image data has arrived at the receiving conference device, there are cases where it is difficult to see the image due to inappropriate brightness and color tone of the display device.

  In such a case, in the conventional conference system, there is a problem that causes an inappropriate situation in order to execute the conference in the entire system even if the audio level and quality information are normal.

  Also, when taking a vote on the agenda of a meeting, if there are many meeting participants (multiple locations), you must confirm the pros and cons with the participants at each point, and confirm the pros and cons of each one Has a problem that it takes a lot of time and the progress of the conference is not efficient.

  Furthermore, the reaction of the conference participants may not be obtained, and there is a possibility that the conference is less fruitful and less fruitful than a conference where faces are face to face.

  In addition, when there is a large amount of data on the communication path between the conference devices, the communication delay status cannot be grasped even though the communication delay is long. There is a problem that the visualization data is displayed and gives a sense of incongruity.

  The present invention has been made to solve such a problem, and its purpose is to create a simulated environment close to a meeting environment in which faces are face to face, and to realize realistic and efficient conference communication. Is to provide a reliable conference system.

  In order to solve the above problems, a conference system according to the present invention connects at least two or more conference terminals via an IP network, and audio data for audio conference communication or audio data and video for video conference communication between the conference terminals. A conference system configured to transmit and receive data, wherein the conference terminal multiplexes the operation state of the conference terminal on its own side as additional data into audio data or video data, and the multiplexed data is transmitted to the other party via the IP network. A decomposing unit for decomposing the additional data from the received multiplexed data when receiving the multiplexed data transmitted via the IP network by the transmitting unit of the other conference terminal , Display that displays the operation status of the other party conference terminal based on the additional data decomposed by the disassembling means Characterized by comprising a stage.

  The conference system according to the present invention connects at least two or more conference terminals via an IP network, and transmits / receives audio data of audio conference communication or audio data and video data of video conference communication between the conference terminals. Each of the conference terminals receives a conference signal transmission means for transmitting a conference start signal of the conference terminal on the own side, and a conference start signal transmitted from the conference terminal on the other side. A reply means that immediately returns a received signal corresponding to the received conference start signal to the other party's conference terminal, and a communication delay time to the other party's conference terminal when received from the other party's conference terminal based on the received signal received A delay time calculating means for calculating the delay time, a delay time notifying means for notifying a communication delay time calculated by the delay time calculating means to the conference terminal on the other side, and a delay time Characterized by comprising a time-stamp management means for managing the unified time stamp based on the notified communication delay time knowledge means.

  Each conference terminal of the conference system according to the present invention multiplexes the operation state of the conference terminal on its own side as audio data or video data as additional data, and transmits the multiplexed data to the other conference terminal via the IP network. And means for decomposing the additional data from the received multiplexed data when the multiplexed data transmitted via the IP network is received by the transmitting means of the other party's conference terminal, and the addition that is decomposed by the decomposing means Display means for displaying the operation state of the other party's conference terminal based on the data, so that the conference participant can operate the other party's conference terminal, such as the microphone on / off state and the speaker volume position. The conference can be executed while checking on the screen, and in a simulated environment close to the actual conference, a realistic audio conference or TV It is possible to realize the discussions.

  In addition, each conference terminal according to the present invention receives the conference start signal transmitted from the conference signal transmitting means for transmitting the conference start signal of the own conference terminal and the conference start signal transmitted from the other conference terminal. A response means for immediately returning a received signal corresponding to the other party's conference terminal, and a delay time for calculating a communication delay time to the other party's conference terminal based on the received signal when receiving from the other party's conference terminal Management unit, delay time notifying unit for notifying the other party's conference terminal of the communication delay time calculated by the delay time calculating unit, and managing a unified time stamp based on the communication delay time notified from the delay time notifying unit The meeting organizer terminal can manage the unified time stamp by calculating the communication delay time of all participating conference terminals. It is possible to have a sense of reality to grasp the conference status.

  The conference system according to claim 1 connects at least two or more conference terminals via an IP network, and transmits / receives audio data of audio conference communication or audio data and video data of video conference communication between the conference terminals. In the conference system configured as described above, the conference terminal multiplexes the operation state of the conference terminal on its own side as audio data or video data as additional data, and the multiplexed data is transmitted to the other conference terminal via the IP network. A transmission unit for transmitting, and when receiving the multiplexed data transmitted via the IP network by the transmission unit of the other party conference terminal, a decomposition unit for decomposing the additional data from the received multiplexed data, and a decomposition unit Display means for displaying the operating state of the other party conference terminal based on the decomposed additional data It was.

  Therefore, according to each conference terminal of the conference system according to claim 1 of the present application, the operating state of the conference terminal on the local side is multiplexed as additional data with audio data or video data, and the multiplexed data is transmitted to the other side via the IP network. A disassembling means for disassembling the additional data from the received multiplexed data when receiving the multiplexed data transmitted via the IP network by the transmitting means of the counterpart conference terminal, and having a transmitting means for transmitting to the conference terminal; Display means for displaying the operating state of the other party's conference terminal based on the additional data decomposed by the disassembling means, so that the conference participant can operate the other party's conference terminal, for example, on / off of the microphone. You can run a conference while checking the status of the speaker volume and the position on the screen. In a simulated environment close to the actual conference, It is possible to realize a voice conference or video conference with a place feeling.

  In addition, each conference terminal according to claim 2 of the present application includes storage means for storing in a state management table the operating states of all the conference terminals on the other side connected to the IP network, and the additional data decomposed by the disassembling means. Based on this, the state management table is updated sequentially.

  Therefore, according to each conference terminal according to claim 2 of the present application, the additional data decomposed by the disassembling means is provided with storage means for storing in the state management table the operating states of all the conference terminals connected to the IP network. The status management table is updated sequentially based on the above, so that the operating status of each conference terminal that changes every moment can be confirmed on the screen, and the conference environment with a sense of reality can be created by changing the operating status. Can be formed.

  Furthermore, the operation state as additional data stored in the state management table according to claim 3 of the present application is the level of audio data or the level of video data received from the counterpart conference terminal or communication delay time information.

  In addition, setting information of each conference terminal is included as additional data stored in the state management table according to claim 4 of the present application.

  Therefore, according to the additional data stored in the state management table according to claims 3 and 4 of the present application, the operation state is the level of the audio data or the level of the video data received from the partner conference terminal or the communication delay time information. In addition, since the setting information of each conference terminal is included, it is possible to memorize and display various audio conference and video conference settings, operation information, and operation status, and to make the conference more realistic. Can do.

  Furthermore, in the conference system according to claim 5, at least two or more conference terminals are connected via an IP network, and audio data of audio conference communication or audio data and video data of video conference communication are transmitted between the conference terminals. A conference system configured to transmit and receive, wherein each conference terminal receives a conference signal transmission unit that transmits a conference start signal of the conference terminal on its own side, and a conference start signal transmitted from the conference terminal on the other side Then, a reply means that immediately returns a received signal corresponding to the received conference start signal to the other party's conference terminal, and communication from the other party's conference terminal to the other party's conference terminal when received from the other party's conference terminal. A delay time calculating means for calculating a delay time, a delay time notifying means for notifying a communication delay time calculated by the delay time calculating means to the conference terminal on the other side, And to have a time-stamp management means for managing the unified time stamp based on the notified communication delay time from the time notifying means.

  Therefore, according to each conference terminal according to claim 5 of the present application, each conference terminal transmits a conference start signal transmitted from the conference terminal on the other side, and a conference start signal transmitted from the conference terminal on the other side. , A reply means that immediately returns a received signal corresponding to the received conference start signal to the other party's conference terminal, and when receiving from the other party's conference terminal, to the other party's conference terminal based on the received signal received. A delay time calculating means for calculating the communication delay time, a delay time notifying means for notifying a communication delay time calculated by the delay time calculating means to the conference terminal on the other side, and a communication delay time notified from the delay time notifying means. And a time stamp management means for managing a unified time stamp based on this, the conference host conference terminal calculates and unifies the communication delay time of all participating conference terminals. It is possible to manage the time stamp, it is possible to have a sense of reality to grasp the conference status.

  Further, each conference terminal according to claim 6 of the present invention includes a visualization means for visualizing audio data received from the other party's conference terminal, and voice visualization data from the other party's conference terminal visualized by the visualization means. Voice visualization data transmission means for adding and transmitting communication delay time information to the other conference terminal, and receiving the voice visualization data of the conference terminal on the other side from the conference terminal on the other side, up to the conference terminal on the other party A delay state is determined based on the communication delay time information, and a visualization voice display means for displaying the judgment result together with the voice visualization data is provided.

  Therefore, according to each conference terminal described in claim 6, the visualization means for visualizing the voice data received from the other party's conference terminal, and the voice visualization data from the other party's conference terminal visualized by the visualization means, The voice visualization data transmission means for adding and transmitting communication delay time information to the conference terminal on the other side, and receiving the voice visualization data of the conference terminal on the other side from the conference terminal on the other side, Since the delay situation is determined based on the communication delay time information to the terminal and the determination result is displayed together with the voice visualization data, the voice sender transmits the voice to the other side. Meeting and confirming the communication delay time of the data while confirming the display screen of the voice visualization data. It can be current.

  Further, the visualized voice display means according to claim 7 of the present application displays the determination result as an alarm display when the communication delay time exceeds a predetermined time.

  Therefore, according to the visualized voice display means according to claim 7 of the present application, when the communication delay time exceeds the predetermined time, the determination result is displayed as an alarm display, so that the voice transmitting side stops the voice. However, when voice visualization data is displayed, it can be warned that the communication delay time is long and a time lag occurs, and the voice visualization data is displayed when the voice is stopped. The feeling of strangeness can be relieved.

  Further, each conference terminal according to claim 8 of the present application captures an image of the conference terminal on its own side and outputs image data that is captured, and video that reproduces the video data received from the conference terminal on the other side Reproduction means, still image extraction means for extracting still image data at a fixed period from video data reproduced by the video reproduction means, and adding communication delay time information to the still image data extracted by the still image extraction means Multiplexing transmission means for multiplexing and transmitting multiplexed data to the video data captured by the means, and a still image that displays the communication delay time added together with the still image data when the multiplexed data is received from the other party conference device Display means.

  Therefore, according to each conference terminal according to claim 8 of the present application, the video of the conference terminal on the own side is captured, the imaging means for outputting the captured video data, and the video data received from the conference terminal on the other side are reproduced. Video reproduction means, still image extraction means for extracting still image data from the video data reproduced by the video reproduction means, and communication delay time information is added to the still image data extracted by the still image extraction means. When the multiplexed data is received from the other party conference device, the communication delay time added together with the still image data is displayed. Since the video transmission side has a still image display means, the video transmission side performs a video conference while confirming in a still image how the video transmitted to the other party is displayed. Nau it can, it is possible to appeal a further sense of realism.

  Further, the still image display means according to claim 9 of the present application displays the still image data as a continuous slide.

  Therefore, according to the still image display means according to claim 9 of the present application, since still image data is displayed as a continuous slide, a moving image can be displayed with still image data far less than moving image data. Can increase the sense of reality.

  In addition, each conference terminal according to claim 10 of the present application is input from an input unit for inputting input information from an input device and an input unit for displaying an intention display of a conference participant by a voice message, a text message, or a pictograph. A command transmission unit that converts the input information into a communication command based on the conversion table and transmits the communication command; and an output unit that outputs the output information corresponding to the communication command when the communication command is received.

  Therefore, according to each conference terminal of claim 10 of the present application, in order to display the intention display of the conference participant with a voice message, a text message, or a pictogram, the input means for inputting input information from the input device, and the input means A command transmission unit that converts input input information into a communication command based on the conversion table and transmits the command, and an output unit that outputs output information corresponding to the communication command to the output device when the communication command is received. As a result, the participants of the conference can promptly express approval or disapproval of the content of the conference, and can proceed with an efficient conference with a sense of reality close to that of the actual conference.

  Furthermore, the input means according to claim 11 of the present application is provided with an interface with a personal computer constituting the input device, and input information is input from the personal computer.

  Therefore, according to the input means described in claim 11 of the present application, since it is provided with an interface with a personal computer constituting the input device and input information is input from the personal computer, it is simple to carry out an audio conference or a video conference. With a simple operation, it is possible to input highly complicated and advanced input information, and a meeting environment close to an actual meeting can be formed.

  The output means according to claim 12 comprises voice message storage means for storing a voice message in response to a communication command, and when the communication command is received, the corresponding voice message is read from the voice message storage means and audible. The sound is output to a sound output device as a display, and the sound is output via the sound output device.

  Further, the output means according to claim 13 includes character message storage means for storing a character message corresponding to the communication command, and when the communication command is received, the corresponding character message is read from the character message storage means and visible. The data is output to a display device as a display device, and characters are output via the display device.

  Therefore, according to the output means of claims 12 and 13 of the present application, the output means includes voice message storage means for storing a voice message corresponding to the communication command, and when the communication command is received, the corresponding voice message is sent to the voice message. Read from storage means, output to audio output device as audible display, and output sound through output device, or read corresponding character message from character message storage means, display device as visual display Since the text is output via the display device, it is possible to convey the intentions of the conference participants with a standard voice message or text message set in advance with a simple operation, and appealing the improvement of convenience be able to.

  The output means according to claim 14 includes power generation means for generating a driving force in response to a communication command, and when the communication command is received, the power generation means converts the power generation means into a driving force and is connected thereto. , And various display panels are moved (for example, the display panel is raised or tilted) to display.

  Therefore, according to the output means of claim 14 of the present application, the power generation means for generating the driving force in response to the communication command is provided, and when the communication command is received, the power generation means is converted into the driving force and connected. Since the power unit is driven and various display boards are moved (for example, standing or tilting the display board) and displayed, the participants of the conference are moved by the movement of the power unit (for example, standing or tilting the display board). This makes it possible to give a sense of fun to the movement of a power unit (for example, a display board of pictograms) and provide a fun and peaceful audio conference and video conference.

(Embodiment)
Embodiments of the present invention will be described below with reference to the accompanying drawings. The present invention is a conference system that performs communication for audio conferences using audio data or video conferences using audio data and video data between a plurality of conference terminals connected to an IP network. In order to check whether or not your voice or video is transmitted to the other party, to display the voice data of your voice or the still image data of your video sent from the other party on the screen, The other party's setting data and status data, and the communication delay time of the communication path to the other party are displayed on the screen.

  In addition, the present invention quickly displays the intention of a conference participant by voice message, text message, or power display with a simple operation.

  FIG. 1 is a system configuration diagram of an embodiment of a conference system according to the present invention. In FIG. 1, the conference system 1 is composed of an IP network NT and conference devices S, A, B to M that constitute a plurality of conference terminals connected to the IP network NT, and the conference devices S, A, A voice conference of voice data communication or a video conference of voice data communication and video data communication is performed between B to M.

  The conference device S is used as a conference host terminal and the conference devices A and B to M are participating in the conference. The conference host terminal is in charge of any conference device S, A, or B to M. can do. Further, if there are two or more conference devices S, A, B to M, a conference can be held with an arbitrary number.

  FIG. 2 is a block diagram showing the main part of an embodiment of the conference apparatus according to the present invention. The conference apparatuses S, A, and B to M have the same configuration and the same function, and therefore the conference apparatus S will be described. In FIG. 2, a conference device S is connected to an IP network NT and is connected to a main processing unit 2 that controls the operation of the entire device, a video / audio input device 7 and an audio output device 8, and receives audio data and video data. The voice / image data processing unit 3 to be processed, a display data control unit 4 connected to the display device 6 for controlling display of display data, and a response assisting device 5 for executing input / output of intention display.

  The main processing unit 2 includes a device operation input device 9 for performing a conference start and speaker volume setting operation of the conference device S, a system control unit 10 for controlling the operation of the entire device, and transmitting audio data, video data, and the like. The transmission / reception data multiplexing / decomposing unit 11 that multiplexes data and performs processing for decomposing the received multiplexed data, stores the speaker status, microphone status, participant information, etc. of the conference device participating in the conference, and updates Setting / operation status management table 12 (state management table), conference time management table 13 for storing communication delay time of communication paths between conference devices, interface with IP network NT, and various data to IP network NT IP network communication I / F control unit 14 that transmits and receives from IP network NT, and is supplied from response assisting device 5 Response support for controlling communication for transmitting a communication command converted from the input information of the conference participants to the IP network NT, receiving communication command from the IP network NT, and controlling communication for transmitting to the response assisting device 5 A device communication control unit 15 is provided.

  The audio / image data processing unit 3 is connected to a microphone / camera video / audio input device 7 and a speaker audio output device 8, encodes video data captured by the video / audio input device 7 (camera), and performs main processing. A video data processing unit (encoding / decoding) 16 for decoding the encoded data of the video data supplied from the main processing unit 2 and supplying it to the display data control unit 4; The audio data picked up by the device 7 (microphone) is encoded and supplied to the main processing unit 2, and the encoded data of the audio data supplied from the main processing unit 2 is decoded and output from the audio output device 8 (speaker). Audio visualization data is generated from the audio data decoded by the audio data processing unit (encoding / decoding) 17 and the audio data processing unit (encoding / decoding) 17 The still image data obtained by extracting the video data decoded by the audio visualization data generating unit 18 and the video data processing unit (encoding / decoding) 16 supplied to the main processing unit 2 at a predetermined cycle is supplied to the main processing unit 2 An image data extraction unit 19 is provided.

  The display data control unit 4 is connected to the display device 6, the audio visualization data control unit 20 that controls the audio visualization data supplied from the main processing unit 2, and the video data and still image data supplied from the main processing unit 2. Display data is generated from the audio / visualization data supplied from the video / still image data control unit 21 and the audio visualization data control unit 20 to be controlled, and the video data and still image data supplied from the video / still image data control unit 21; The display data control part 22 displayed on the display apparatus 6 is provided.

  The response assisting device 5 outputs the intention display of the conference participant as a voice message, a text message, or a mechanical power display. The intention display input unit 23 inputs the intention input information of the intention display to the other party and the other party. The intention display output unit 24 that outputs the intention output information of the intention display from the user is input, and when the intention display of the conference participant is input to the intention display input unit 23 as the intention input information from the console or personal computer, the intention input information is communicated. It is converted into a command and transmitted to the IP network NT via the main processing unit 2 (response assisting device communication control unit 15), and a communication command for intention display from the IP network NT is sent to the main processing unit 2 (response assisting device communication control unit). 15), the intention display output unit 24 converts the message into a voice message, text message, or mechanical power intention output information. Are, speaker, display, sound from the mechanical device, gestures in character or mechanical movement is made.

  FIG. 3 is a diagram showing input / output data (information) related to an embodiment of the main processing unit according to the present invention. In FIG. 3, the main processing unit 2 receives from the IP network NT, and via the system control unit 10, setting / operation status information from the other party, received voice visualization data + communication delay time information, received still image data + Communication delay time information and audio / video output data are supplied to the audio / image data processing unit 3 or the display data control unit 4 as display reproduction data.

  Further, the main processing unit 2 sends the output audio visualization data, the still image data extracted from the output video, and the audio / video input data to the other party via the system control unit 10 as transmission data. Send to.

  FIG. 4 is an input / output data (information) related diagram of an embodiment of a sound / image data processing unit according to the present invention. In FIG. 4, an audio data processing unit (encoding / decoding) 17 encodes audio (data) input from the microphone MC, supplies the encoded input audio data to the main processing unit 2, and performs main processing. The encoded output audio data supplied from the unit 2 is decoded to generate output audio from the speaker SP.

  The voice visualization data generation unit 18 uses, for example, an equalizer characteristic having an amplitude level corresponding to a frequency band (voice frequency) or an amplitude level level characteristic voice from the output voice from the voice data processing unit (encoding / decoding) 17. Visualization data is generated, and output voice visualization data is supplied to the main processing unit 2.

  The video data processing unit (encoding / decoding) 16 encodes the input video (data) captured by the camera CCD, and supplies the encoded input video data to the main processing unit 2 and from the main processing unit 2. The encoded output video data is decoded and the output video data is supplied to the display data control unit 4.

  The still image data extraction unit 19 periodically extracts the output video data from the video data processing unit (encoding / decoding) 16 and supplies the still image data extracted from the output video data to the main processing unit 2.

  FIG. 5 is an input data (information) related diagram of an embodiment of the display data control unit according to the present invention. In FIG. 5, the voice visualization data control unit 20 includes setting / operation state information supplied from the main processing unit 2, received voice visualization data + communication delay time information, and output supplied from the voice / image data processing unit 3. Voice visualization data is taken in, processed and controlled, and supplied to the display data control unit 22.

  The video / still image data control unit 21 receives setting / operation state information supplied from the main processing unit 2, received still image data + communication delay time information, and output video data supplied from the audio / image data processing unit 3. Is supplied to the display data control unit 22 after being processed and controlled.

  The display data control unit 22 generates display data based on the data supplied from the audio visualization data control unit 20 and the video / still image data control unit 21, supplies the display data to the display device 6, and the audio visualization data. And communication delay time information and setting / operation status data are displayed together with still image data, and video data is displayed.

  FIG. 6 is a block diagram of a main part of one embodiment of the response auxiliary block according to the present invention. In FIG. 6, the response auxiliary block 25 includes a response auxiliary device 5 having an intention display input unit 23 and an intention display output unit 24, a response auxiliary device communication control unit 15 of the main processing unit 2, a system control unit 10, and transmission / reception data. The multiplexing / decomposition unit 11 and the IP network communication I / F control unit 14 are configured.

  The intention input information representing the intention display of the conference participant input to the intention display input unit 23 is converted into a communication comment, and is transmitted via the response assisting device communication control unit 15, the system control unit 10, and the transmission / reception data multiplexing / decomposition unit 11. Audio / video input data, output audio visualization data + communication delay time information and still image data extracted from output video + communication delay time information, and multiplexed data is transmitted from the transmission / reception data multiplexing / decomposition unit 11 to the IP. Sent to network NT.

  On the other hand, the multiplexed data including the communication comment indicating the intention of the conference participant received from the IP network NT by the IP network communication I / F control unit 14 passes through the transmission / reception data multiplexing / decomposition unit 11 and the system control unit 10. Audio / video output data, received audio visualization data + communication delay time information, received still image data + communication delay time information, communication comments, and communication comments are displayed via the response assistant communication control unit 15 The output unit 24 converts it into intention output information and outputs it.

  The intention display input unit 23 is connected to an operation console or a personal computer, and is operated by a conference participant to receive intention input information and convert it into a communication comment corresponding to the intention input information. Supply to part 2.

  The intention display output unit 24 converts a communication comment supplied from the response assisting device communication control unit 15 into a voice message, a text message, or mechanical power that is preset intention output information, and performs a speaker, a display, a mechanical An intention display is realized from the device.

  Next, based on the configuration requirements of the conference device S described above, additional data and state management table, unified clock (time stamp) management between conference devices and communication delay time between conference devices, visualization of audio data, video data A still image and response assisting device will be described.

  First, the additional data of the conference device includes the speaker volume position of all conference devices participating in the conference, microphone mute ON / OFF, conference participant information (number of people, etc.), communication delay time information, voice visualization data, stationary Image data, etc., which are multiplexed with audio data and video data and transmitted / received.

  FIG. 7 is a block diagram of the main part of an embodiment of the conference apparatus for transmitting / receiving additional data according to the present invention. In FIG. 7, the conference terminal S multiplexes the operation state in the conference terminal on its own side as additional data with audio data or video data (audio / video data) by the multiplexing means 43, and the multiplexed data is transmitted via the IP network NT. Transmission means 41 for transmitting to the other party conference terminal, and when the multiplexed data transmitted via the IP network NT by the transmission means 41 of the other party conference terminal is received by the reception means 42, the received multiplexed data Decomposition means 44 for disassembling the additional data, and display means 45 for displaying the operating state of the other party conference terminal based on the additional data decomposed by the decomposition means 44. The decomposed additional data is stored in the state management table TB1 of the storage means 46.

  The multiplexing means 43 that multiplexes the additional data into the audio data or the video data is constituted by the transmission / reception data multiplexing / decomposition section 11 of the main processing section 2, and the transmission means 41 that transmits the multiplexed data to the IP network NT. The IP network communication I / F control unit 14 is configured.

  The receiving means 42 for receiving multiplexed data from the IP network NT is constituted by the IP network communication I / F control unit 14, and the decomposing means 44 for decomposing additional data from the multiplexed data is a transmission / reception data multiplexing / decomposing unit. 11.

  Further, the display means 45 for displaying the operation state based on the additional data is constituted by the display data control unit 4 and the display device 6.

  As a result, the conference participant can execute the conference while confirming the operation state of the other device, for example, the on / off state of the microphone and the position of the speaker volume, on the screen, and it can simulate the actual conference. Realistic audio and video conferences can be realized in the environment.

  In addition, the conference device S includes storage means 46 having a state management table TB1 for storing the operation state of each conference device, and based on the additional data decomposed by the decomposition means 44 (transmission / reception data multiplexing / decomposition unit 11) The management table TB1 is sequentially updated, and the display means 45 (the display data control unit 4 and the display device 6) displays the operation state based on the state management table TB1, so that each conference device that changes every moment is displayed. The operating state during the conference can be confirmed on the screen, and a realistic conference environment can be formed by the change in the operating state.

  The additional data stored in the state management table TB1 is such that the operation state is the level of audio data or the level of video data received from the partner conference terminal or communication delay time information, and includes setting information of each conference terminal. Therefore, it is possible to store and display various kinds of voice conferences and video conferences, setting information, and operation states, and to make the conference more realistic.

  FIG. 8 is an operation flowchart of the main part of an embodiment of the conference apparatus state management table creation method according to the present invention. In FIG. 8, the conference is started (step S1), and the setting / operation status information of the own terminal (conference device S) is transmitted to each terminal (conference devices A, B to M) (step S2).

  Next, the setting / operation status information of each terminal (conference devices A, B to M) is received (step S3), and a setting / operation status management (status management) table is created (step S4).

  Subsequently, it is determined whether or not the setting / operation status information is updated at any time (step S5), and when it is updated (YES), steps S2 to S4 are repeated. On the other hand, if not updated (NO), the conference is terminated (step S6).

  FIG. 9 is a conceptual diagram of an embodiment of the setting / operation status management table according to the present invention. In FIG. 9, the setting / operation status management table (status management table) of the conference device S stores the settings / operation status and status of the conference devices S, A, B to M.

  Similarly, the setting / operation status management tables of the conference apparatuses A, B to M store the setting / operation status and status of the conference apparatuses S, A, B to M, respectively.

  Next, unified clock management between conference devices and communication delay time between conference devices will be described. The unified clock management between the conference devices is to match the clocks of all the conference devices with the conference time (clock) of the conference device that holds the conference. The communication delay time between the conference devices is the conference time. Indicates the data transmission delay time of the communication path between devices.

  FIG. 10 is a block diagram of a main part of an embodiment of a conference apparatus for transmitting and receiving communication delay time information according to the present invention. In FIG. 10, when the conference start button is operated, the conference device S (sponsoring terminal) generates a conference start signal from the signal generation unit 47 and transmits the conference start signal generation from the conference signal transmission unit 49 to the IP network NT. . When the conference start button is operated, the unified clock WH starts.

  When the conference apparatus S is a participating conference apparatus, when any of the other conference apparatuses A and B to M is the host terminal and receives the conference start signal by the receiving means 51 via the IP network NT, the conference apparatus S immediately receives the received signal. Is generated from the signal generating means 47, and the received signal is transmitted from the reply means 50 to the IP network NT. In the case of a participating conference device, the unified clock WH is started simultaneously with receiving the conference start signal.

  Further, when the conference device S (host terminal) receives a reception signal in response to the conference start signal from the other conference devices A and B to M (participating terminals) by the reception means 51 via the IP network NT, The communication delay time is calculated by the delay time calculation means 52 from the delay time notifying means 53 to the transmission means 48 and the IP network NT. Devices A, B to M (participating terminals) notify each conference terminal.

  Further, a time stamp unified based on the calculated communication delay time is stored and managed in the time management table TB2 of the management means 54.

  The signal generating unit 47, the unified clock WH, the delay time calculating unit 52, the delay time notifying unit 53, and the managing unit 54 are configured by the system control unit 10 of the main processing unit 2. The transmission unit 48 and the reception unit 51 including the conference signal transmission unit 49 and the reply unit 50 are configured by the IP network communication I / F control unit 14 of the main processing unit 2.

  The unified clock WH that measures the conference time is built in the system control unit 10 and includes a time management table TB2 (conference time management table 13) that manages and rewrites the communication delay time between the conference devices.

  Here, with reference to FIG. 1 and FIG. 2, formation of a unified clock (time adjustment) of the conference apparatuses S, A, B to M and acquisition of a communication delay time between the conference apparatuses S, A, B to M explain. First, a conference start signal (first trigger) is transmitted from all the conference devices A and B to M from the conference device S that hosts the conference, and simultaneously, the clock of the conference device S is started.

  Upon receiving the conference start signal (first trigger) from the conference device S, the conference devices A and B to M immediately return the received signal to the conference device S and simultaneously start the clocks of the conference devices A and B to M, respectively. . At this time, the clocks of the conference apparatuses A and B to M are delayed from the clock of the conference apparatus S by different communication delay times between the conference apparatus S and the conference apparatuses A and B to M, respectively.

  When the conference apparatus S receives the reception signals returned from the conference apparatuses A and B to M, the time from transmission of the conference start signal (first trigger) to reception of the reception signal is received for each of the conference apparatuses A and B to M. Measure the communication delay time TDS-A, TDS-B,..., TDS-M from the conference device S to the conference devices A, B to M based on the measured time interval, and calculate the calculated communication delay time. Is stored in the meeting time management table 13. The communication delay times TDS-A, TDS-B,..., TDS-M are ½ of the time interval from the transmission of the conference start signal (first trigger) to the reception of the received signal. Calculated as / 2.

  Subsequently, the conference device S transmits the communication delay times TDS-A, TDS-B,..., TDS-M as communication delay time information to the conference devices A, B to M, and receives the communication delay time information. A and B to M correct their own clocks based on the received communication delay times TDS-A, TDS-B,..., TDS-M, respectively. For example, the conference apparatus A can match the clock time of the conference apparatus S by adding and correcting the communication delay time TDS-A to the clock time of its own apparatus.

  This process forms a unified time for all the clocks of the conference apparatuses S, A, and B to M, and communication delay times from the conference apparatus S to the conference apparatuses A and B to M as a conference time management table 13. Can be stored.

  Next, the communication delay time between the conference apparatus A and the conference apparatuses B to M is determined by transmitting the second trigger from the conference apparatus A to the conference apparatuses B to M and receiving the reception signals from the conference apparatuses B to M. , Measure the time interval from the second trigger transmission to reception of the received signal, and calculate the communication delay times TDA-B, TDA-C,..., TDA-M of the conference apparatuses A and B to M based on the time interval, respectively. The conference device S is notified as communication delay time information, and the conference device S stores the communication delay time information notified from the conference device A in the conference time management table 13.

  Thereafter, in the same manner as described above, the conference device B and the conference devices C to M,..., Calculate the communication delay time between the conference device M-1 and the conference device M, and notify the conference device S of the communication delay time information. The conference device S stores the communication delay time information in the conference time management table 13. In addition, the signal transmission / reception is periodically performed between the conference apparatuses S, A, and B to M, the communication delay time between the conference apparatuses S, A, and B to M is calculated, and the data transmission amount between the communication paths is calculated. If the communication delay time changes due to the above, the communication delay time information in the conference time management table 13 is updated.

  FIG. 11 is an operation flowchart of one embodiment of a communication delay time acquisition method for a conference system according to the present invention. In FIG. 11, when the conference is started (step S11), a first trigger signal (conference start signal) is transmitted from the sponsor terminal (for example, conference device S) to the participating terminals (conference devices A, B to M) ( Step S12). The conference participation terminals (conference devices A, B to M) immediately return to the sponsor terminal after receiving the first trigger signal (step S13).

  Subsequently, the sponsor terminal acquires communication delay time information between the sponsor and the participating terminals (step S14). On the other hand, when the number of participating terminals is two or more, the conference participating terminal transmits a second trigger signal to the terminal after its own terminal + 1 (step S15). After receiving the second trigger signal, the second trigger signal receiving terminal immediately transmits (received signal) to the transmission source terminal (step S16).

  The conference participation terminal acquires the communication delay time information of each terminal (step S17), and transmits the communication delay time information of each terminal to the host terminal (conference device S) (step S18).

  Subsequent to step S14 and step S18, the host terminal (conference device S) creates conference time management table information (step S19), and the conference time management table information is transmitted to all participating terminals (conference devices A, B to M). Sharing (step S20). In addition, it is determined whether or not the information is updated (step S21), and the communication delay time acquisition process is terminated.

  FIG. 12 is an operation flow chart of one embodiment of a unified clock information acquisition method of the conference system according to the present invention. In FIG. 12, when the conference is started (step S31), a first trigger signal (conference start signal) is transmitted from the sponsor terminal (for example, the conference device S) to the participating terminals (conference devices A, B to M). The unified clock is started (step S32).

  The conference participation terminals (conference devices A and B to M) start a unified clock upon receiving the first trigger signal (step S33). All terminals (conference devices S, A, B to M) share the conference time management table information (step S34).

  The conference participation terminals (conference devices A, B to M) reflect the communication delay time between the sponsor and the participation terminals in the unified clock of the own terminal from the conference time management table information (step S35).

  All terminals (conference devices S, A, B to M) create unified clock information (step S36). It is determined whether or not information is updated (step S36), and the unified clock information acquisition process is terminated.

  FIG. 13 is an image diagram of an embodiment of the conference time management table according to the present invention. All communication delay times are covered for all communication paths between the conference apparatuses S, A, B to M. Further, the reception level situation (for example, level attenuation, etc.) can be stored from the exchange of the conference start signal, the first trigger signal (including the second trigger signal) and the received signal.

  As a result, the conference hosting conference device (for example, conference device S) can manage the unified time stamp by calculating the communication delay time of all participating conference devices (conference devices S, A, B to M). , Can grasp the meeting situation and give a sense of reality.

  Subsequently, visualization of audio data of the conference apparatus will be described. FIG. 14 is a block diagram of a main part of an embodiment of a conference apparatus for transmitting and receiving voice visualization data according to the present invention. In FIG. 14, the conference apparatus S receives the visualization means 55 for extracting and visualizing the voice data (output voice) received from the other party's conference apparatus, and the voice visualization data of the own party's conference apparatus from the other party's conference apparatus. Then, communication delay time information to the other party conference device stored in the time management table TB2 is added to the voice visualization data from the other party conference device visualized by the visualization means 55 and transmitted to the IP network NT. When the voice visible data receiving means 57 receives the voice visible data to which the communication delay time information is added from the IP network NT, the voice visible data receiving means 57 determines the delay state based on the added communication delay time information. Visualization voice means 58 for displaying the determination result together with the voice visualization data.

  The visualizing means 55 for visualizing the audio data is configured by the audio visualization data generating unit 18 of the audio / image data processing unit 3. The time management table TB2 for adding the communication delay time information to the voice visualization data is added by the system control unit 10 of the main processing unit 2 by searching the conference time management table 13 for the communication delay time information.

  The voice visible data transmission means 56 and the voice visible data reception means 57 are configured by the IP network communication I / F control unit 14 of the main processing unit 2, and the visualization voice means 58 is configured by the system control unit 10 of the main processing unit 2. And the display data control unit 22 of the display data control unit 4 and the display device 6.

  As a result, the voice sender can hold the conference while checking the voice visualization data display screen to confirm that the voice is transmitted to the other party and the communication delay time of the data. A meeting can be realized.

  Further, when the communication delay time exceeds a predetermined time, the visualizing voice unit 58 displays the determination result with an alarm display. Note that it is the system control unit 10 of the main processing unit 2 that determines whether or not the communication delay time has exceeded a predetermined time.

  As a result, when the voice visualization data is displayed even though the voice transmission side has stopped the voice, it is possible to warn that the communication delay time is long and a time lag has occurred. It is possible to relieve the uncomfortable feeling that the voice visualization data is displayed when the voice is stopped.

  Next, a description will be given of the conversion of video data into a still image. FIG. 15 is a block diagram of the main part of an embodiment of a conference apparatus for transmitting and receiving still image data according to the present invention. In FIG. 15, the conference apparatus S captures an image of its own apparatus and outputs image data, an image reproduction means 62 that reproduces image data received from the other party's conference apparatus, and an image reproduction means. Still image extraction means 61 for extracting still image data from the video data (output video) reproduced in 62 at a fixed period, and communication stored in the time management table TB2 in the still image data extracted by the still image extraction means 61 When delay time information is added, multiplexed transmission means 60 that multiplexes the video data imaged by the imaging means 59 and transmits multiplexed data, and when the multiplexed data is received by the decomposing reception means 63 from the conference apparatus on the other side. And still image display means 64 for displaying the communication delay time added together with the still image data.

  The image pickup means 59 for picking up an image of the device itself and outputting the image data is a camera CCD, and the image reproduction means 62 for reproducing the image data is a video data processing section (encoding / coding) of the audio / image data processing section 3. 16) Still image extraction means 61 for extracting still image data, still image data extraction section 19, adds communication delay time information to still image data, and multiplexes and multiplexes with video data imaged by the imaging means Multiplex transmission means 60 and decomposition reception means 63 for transmitting data are the system control unit 10, transmission / reception data multiplexing / decomposition unit 11, conference time management table 13, and IP network communication I / F control unit 14 of the main processing unit 2. The still image display means 64 includes the display data control unit 4 and the display device 6.

  Still image display means 64 displays still image data in a continuous slide.

  FIG. 16 is an operation flowchart of one embodiment of the video / audio transmission method of the conference system according to the present invention. In FIG. 16, when video / audio is input (step P1), encoding of the input video / audio is executed (step P2), and unified clock information at the time of transmission is multiplexed on the encoded video / audio (step P3). The encoded video / audio + unified clock information is transmitted (step P4). In addition, the transmission terminal side (for example, conference apparatus S) performs the step of step P1-step P4.

  After receiving the encoded video / audio + unified clock information (step P5) and decomposing the encoded video / audio and the unified clock information (step P6), the encoded video / audio is decoded and communication delay time information is obtained. Generate (step P7). Note that generation of communication delay time information is referred to FIG.

  Still image data of the reproduced moving image is extracted, reproduced audio visualization data is generated (step P8), the moving image data / audio data is reproduced, and the audio visualization data is displayed (step P9). Subsequently, the communication delay time information is added to the still image data / audio visualization data (step P10), and the still image data / audio visualization data + communication delay time information is transmitted (step P11). In addition, the receiving side terminal (for example, conference apparatus A) performs the step of step P5-step P11.

  Still image data / sound visualization data + communication delay time information is received (step P12), display data is generated from still image data / sound visualization data, setting / operation state information, and communication delay time information (step P13). The generated display data is displayed (step P14). Note that the steps of Step P12 to Step P14 are executed by the transmission terminal side (for example, the conference device S).

  FIG. 17 is a communication delay time information generation flowchart according to the present invention. In FIG. 17, the unified clock information is acquired from the received multiplexed data (step P21), the received unified clock information is compared with the current terminal unified clock information (step P22), and the communication delay time is calculated (step P22). Step P23).

  The conference time management table is compared with the calculated communication delay time (step P24), and communication delay time information is generated (step P25).

  Accordingly, the video transmission side can perform a video conference while confirming in a still image how the video transmitted to the other party is displayed, and can further enhance the sense of reality.

  In addition, moving images can be displayed with a continuous slide of still image data far less than moving image data, and the sense of reality can be enhanced.

  18 and 19 are display screens of an embodiment of the display device according to the present invention. In the figure, the display screen shows the screen of the conference apparatus S. For example, the upper part of the screen shows the video of the conference apparatuses A and B and the audio visualization data of the conference apparatus S, and the lower part of the screen shows, for example, the left side. Shows the still images of the conference device S and the conference device B being played back by the conference device A and the audio visualization data of the conference device A, and the conference device S being played back by the conference device B on the right side. The still image of the conference apparatus A and the voice visualization data of the conference apparatus B are displayed. In addition, in the still image and the audio visualization data, information such as a communication delay time, normality or abnormality thereof, and volume setting is also displayed.

  Therefore, the speaker of the conference apparatus S can view his / her video and audio received at the conference apparatus A and the conference apparatus B as still images and voice visualization data while listening to the video and conversation captured by the conference apparatus A and the conference apparatus B. In addition to the visual confirmation, the communication delay time information and the setting / operation status can also be visually confirmed, and the atmosphere of the pseudo conference environment close to the actual conference environment can be experienced.

  Next, a response assisting device for intention display will be described. FIG. 20 is a block diagram of a main part of an embodiment of a conference apparatus for transmitting and receiving information for intention display according to the present invention. In FIG. 20, the conference apparatus S has an input means 65 for inputting input information from an input device (for example, a personal computer) in order to display the intention display of the conference participant by a voice message, a text message, or a pictograph. Command transmission means 67 that converts the input information input from 65 into a communication command based on the conversion table 66 and transmits it, and output means 69 that outputs the output information corresponding to the communication command when the communication command is received. .

  The input means 65 has an interface with a personal computer, and input information is input from the personal computer.

  The output unit 69 includes a voice message storage unit (memory) that stores a voice message in response to the communication command. When the communication command is received, the output unit 69 reads the corresponding voice message from the voice message storage unit, and serves as an audible display. The sound is output to a sound output device (for example, a speaker), and the sound is output through the sound output device (speaker).

  The output means 69 includes character message storage means (memory) for storing a character message corresponding to the communication command. When the communication command is received, the output means 69 reads the corresponding character message from the character message storage means, and displays the visual message. Are output to a display device (for example, a display device), and characters are output via the display device (display device).

  Furthermore, the output unit 69 includes a power generation unit that generates a driving force in response to the communication command. When the communication command is received, the power generation unit converts the driving unit into a driving force and drives the connected power unit, Various display boards are moved (for example, the display board is raised or lowered) and displayed.

  The input means 65 is configured by the intention display input unit 23 of the response assisting device 5, the output means 69 is configured by the intention display output unit 24 of the response assisting device 5, and the conversion table 66 includes the intention display input unit 23 and the intention display output unit. 24 is constituted by a common memory built in the unit 24. A voice message storage means for storing a voice message, a character message storage means (memory) for storing a character message, and a power generation means (power conversion section) for converting to a driving force are memories built in the intention display output section 24. Constitute. The command transmission means 67 for transmitting a communication command includes the response assisting device communication control unit 15 and the IP network communication I / F control unit 14 of the main processing unit 2.

  FIG. 21 is an application diagram of one embodiment of the response assisting apparatus according to the present invention. In FIG. 21, the response assisting device 5 of the conference device S includes an intention display input unit 23, an intention display output unit 24, and a conversion table 26, and an operation console 27 and a personal computer 28 for inputting input information are included in the intention display input unit 23. And an output device 29 (audible display device such as a speaker) as an output device for outputting output information as a voice message, a display device 30 (display device) as an output device for displaying output information as a character message, or output information Is connected to the intention display output unit 24 as an output device that moves the display board.

  When input information is input to the intention display input unit 23 by a button operation of the console 27 or a key input operation of the personal computer 28, the input information is converted into a communication command by the conversion table 26, and is transferred from the main processing unit 2 to the IP network NT. Sent to.

  On the other hand, when a communication command is received from the IP network NT via the main processing unit 2, the communication command is converted into corresponding information in the conversion table 26, and the converted information is incorporated in the intention display output unit 24. From a voice message storage unit (voice message storage unit), a character data storage unit (character message storage unit) or a power conversion unit (power generation unit), a voice message, a character message, or a driving force stored in advance corresponding to information It is converted and output as output information to drive the audio output device 29, the display device 30, or the mechanical device 31 (power device).

  For example, when a conference participant approves the agenda of the conference, a voice message “I Agree” from the audio output device 29, a text message “I Agree” from the display device 30, or “Agree” from the mechanical device 31. This can be done with a display board.

  FIG. 22 is an operation flowchart of the main part of one embodiment of the intention display method of the conference apparatus according to the present invention. In FIG. 22, when a conference participant inputs input information (step P31), the input information is converted into a standard command (step P32), and the command is transmitted (step P33).

  Subsequently, when a command is received (step P34), the received command is converted into output information (step P35), and output information is output (step P36).

  Thereby, participants of the conference can promptly express their intentions such as approval and disagreement about the content of the conference, and can proceed with an efficient conference with a sense of reality close to that of the actual conference.

  In addition, it is possible to input fairly complicated and advanced input information with a simple operation while performing an audio conference or a video conference, and a conference environment close to an actual conference can be formed.

  Furthermore, the intention of the conference participant can be communicated by a standard voice message or text message set in advance by a simple operation, and an improvement in convenience can be appealed.

  In addition, the intention of the conference participants can be communicated by the movement of the power unit (for example, raising or lowering the display board), and the movement of the power unit (for example, the display board of pictograms) can be made playful. Meetings and video conferencing can be provided.

  As described above, the conference system according to the present invention has the additional data and state management table, the unified clock (time stamp) management between conference devices and the communication delay time between conference devices, the visualization of audio data, and the stillness of video data. With the total configuration of the imaging and response assisting device, it is possible to form a simulated environment that is close to the actual conference environment and perform conference communication with a sense of reality.

  This conference system is close to the actual conference environment and forms a simulated environment that is close to the face-to-face. It can be applied to any conference system.

System configuration diagram of an embodiment of a conference system according to the present invention Block diagram of a main part of an embodiment of a conference apparatus according to the present invention One embodiment of the main processing unit according to the present invention related to input / output data (information) Audio / image data processing unit according to an embodiment of the present invention Input / output data (information) related diagram One embodiment of display data control unit according to the present invention related to input data (information) Block diagram of essential parts of an embodiment of an auxiliary response block according to the present invention Block diagram of essential parts of an embodiment of a conference apparatus for transmitting / receiving additional data according to the present invention The main part operation | movement flowchart of one Embodiment of the state management table preparation method of the conference apparatus based on this invention An image of an embodiment of a setting / operation status management table according to the present invention Block diagram of essential parts of an embodiment of a conference apparatus for transmitting and receiving communication delay time information according to the present invention Operation flow diagram of an embodiment of a communication delay time acquisition method for a conference system according to the present invention Operation flow diagram of an embodiment of a unified clock information acquisition method of a conference system according to the present invention An embodiment image diagram of a meeting time management table according to the present invention Block diagram of essential parts of an embodiment of a conference apparatus for transmitting and receiving voice visualization data according to the present invention Block diagram of essential parts of an embodiment of a conference apparatus for transmitting and receiving still image data according to the present invention Operation flow diagram of one embodiment of video and audio transmission method of conference system according to the present invention Communication delay time information generation flow diagram according to the present invention Display screen according to an embodiment of the display device according to the present invention Display screen according to an embodiment of the display device according to the present invention Block diagram of essential parts of an embodiment of a conference apparatus for transmitting and receiving information for intention display according to the present invention Application diagram of one embodiment of a response assisting apparatus according to the present invention Main part operation flow diagram of one embodiment of intention display method of conference device according to the present invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Conference system 2 Main processing part 3 Audio | voice / image data processing part 4 Display data control part 5 Response processing apparatus 6 Display apparatus 7 Video | voice / audio input apparatus 8 Audio output apparatus 9 Apparatus operation input apparatus 10 System control part 11 Transmission / reception multiplexing / decomposition Unit 12 Setting / Operation Status Management Table 13 Conference Time Management Table 14 IP Network Communication I / F Control Unit 15 Response Auxiliary Device Communication Control Unit 16 Video Data Processing Unit (Encoding / Decoding)
17 Audio data processing unit (encoding / decoding)
18 Voice Visualization Data Generation Unit 19 Still Image Data Extraction Unit 20 Audio Visualization Data Control Unit 21 Video / Still Image Data Control Unit 22 Display Data Control Unit 23 Intention Display Input Unit 24 Intention Display Output Unit 25 Response Auxiliary Block 26 Conversion Table 27 Operation Table 28 Personal computer 29 Audio output device 30 Display device 31 Mechanical device 41 Transmission means 42 Reception means 43 Multiplexing means 44 Decomposition means 45 Display means 46 Storage means 47 Signal generation means 48 Transmission means 49 Conference signal transmission means 50 Reply means 51 Reception Means 52 Delay time calculating means 53 Delay time notifying means 54 Management means 55 Visible means 56 Audio visual data transmitting means 57 Audio visible data receiving means 58 Audio display means 59 Imaging means 60 Multiplexing transmission means 61 Still image extracting means 62 Video reproducing means 63 minutes Decomposition reception means 64 Still image display means 65 Input means 66 Conversion table 67 Command transmission means 68 Command reception means 69 Output means S, A, B to M Conference device KSU Conference support means NT IP network MC Microphone SP Speaker CCD camera WH Unification Clock BT Conference start button TB1 Status management table TB2 Time management table

Claims (14)

A conference system configured to connect at least two or more conference terminals via an IP network and transmit / receive audio data of audio conference communication or audio data and video data of video conference communication between the conference terminals. And
Each conference terminal includes a transmission unit that multiplexes the operation state of the conference terminal on its own side as additional data with the audio data or the video data, and transmits the multiplexed data to the other conference terminal via the IP network. And a decomposing means for decomposing the additional data from the received multiplexed data when the multiplexed data transmitted via the IP network is received by the transmitting means of the counterpart conference terminal, and the decomposing means And a display means for displaying the operating state of the counterpart conference terminal based on the additional data decomposed by the above.   Each conference terminal includes a storage unit that stores the operation state of all the conference terminals on the other side connected to the IP network in a state management table, and based on the additional data decomposed by the decomposition unit, The conference system according to claim 1, wherein the state management table is sequentially updated.   3. The operation state as the additional data stored in the state management table is a level of the audio data, a level of the video data, or communication delay time information received from a counterpart conference terminal. Conference system.   The conference system according to claim 2, wherein setting information of each conference terminal is included as the additional data stored in the state management table. A conference system configured to connect at least two or more conference terminals via an IP network and transmit / receive audio data of audio conference communication or audio data and video data of video conference communication between the conference terminals. And
When each of the conference terminals receives a conference signal transmitting means for transmitting a conference start signal of the conference terminal on the own side and a conference start signal transmitted from the conference terminal on the other side, the reception corresponding to the received conference start signal A reply means for immediately returning a signal to the other party's conference terminal, and a delay time for calculating a communication delay time to the other party's conference terminal based on the received signal received when receiving from the other party's conference terminal Calculation means, delay time notifying means for notifying the communication delay time calculated by the delay time calculating means to the conference terminal on the other side, and unified based on the communication delay time notified from the delay time notifying means And a time stamp management means for managing the time stamp.   Each conference terminal has a visualization means for visualizing the audio data received from the other party's conference terminal, and voice visualization data from the other party's conference terminal visualized by the visualization means, The voice visualization data transmitting means for adding and transmitting the communication delay time information up to and receiving the voice visualization data of the conference terminal on the other side from the conference terminal on the other party side. 6. The conference system according to claim 5, further comprising: visualized voice display means for judging a delay situation based on the communication delay time information to the terminal and displaying the judgment result together with the voice visualized data.   7. The conference system according to claim 6, wherein the visualizing voice display means displays a determination result as an alarm display when the communication delay time exceeds a predetermined time.   Each conference terminal captures an image of its own conference terminal and outputs the captured video data; a video playback unit that plays back video data received from the other conference terminal; and the video playback A still image extraction means for extracting still image data from the video data reproduced by the means at a constant period; and the communication delay time information is added to the still image data extracted by the still image extraction means, and the imaging means Multiplexing transmission means for multiplexing and transmitting multiplexed data to the video data picked up in step (b), and receiving the multiplexed data from the partner conference device, the communication delay time added together with the still image data. 8. The conference system according to claim 5, 6 or 7, further comprising a still image display means for displaying.   9. The conference system according to claim 8, wherein the still image display means displays the still image data as a continuous slide.   Each of the conference terminals converts the input information input from the input means and input means for inputting the input information from an input device in order to display the intention display of the conference participant as a voice message, a text message, or a pictogram. 2. A command transmission means for converting to a communication command based on a table and transmitting the communication command; and an output means for outputting output information corresponding to the communication command when the communication command is received. The conference system according to claim 5 or 8.   The conference system according to claim 10, wherein the input unit includes an interface with a personal computer constituting the input device, and the input information is input from the personal computer.   The output means includes voice message storage means for storing a voice message in response to the communication command. When the communication command is received, the corresponding voice message is read out from the voice message storage means and used as an audible indicator. The conference system according to claim 10, wherein the audio is output to the audio output device and output via the audio output device.   The output means includes character message storage means for storing the character message corresponding to the communication command, and when the communication command is received, the corresponding character message is read from the character message storage means, and a visual display 11. The conference system according to claim 10, wherein the conference system outputs the data to a display device as a character and outputs characters via the display device. The output means includes power generation means for generating a driving force in response to the communication command. When the communication command is received, the power generation means converts the driving power device into a driving force and drives the connected power device. 11. The conference system according to claim 10, wherein various display boards are moved (for example, the display boards are set up or down) and displayed.

Images