JP6160658B2 - CONFERENCE SYSTEM AND CONTROL METHOD - Google Patents

Description

  The present invention relates to a terminal device and a conference system capable of preventing information leakage in a conference system in which conference information including images, sounds, etc. of attendees is transmitted and received between places that are separated from each other in a conference. About.

2. Description of the Related Art Conference systems that conduct conferences between places that are distant from each other via a communication network such as the Internet have become widespread.
In this conference system, in a conference room where one of the parties such as attendees holding the conference is present, the terminal device of the conference system is used to shoot and collect images of the conference room such as the parties of the conference and voices such as remarks, The data is converted into electronic data, transmitted to the other party's conference device, displayed on a display such as a television screen in the other party's conference room, and output as sound through a speaker, and the conference is held in a state close to an actual conference.

  Recently, small and light portable devices have become widespread as terminal devices for conference systems, and are often brought into a conference room or a normal room when a conference is performed.

  Recently, companies have introduced a telecommuting system. In this case, a conference system has also been used for meetings between employees' homes and companies. From this situation, the number of conferences using the conference system may be from a large number to a small number, or the other party may be one.

  As for the meeting room, it is not an independent meeting room that is separate from the office where the company works. Etc. are often used, and it has become an environment in which it is easy for others other than the parties of the conference to suddenly look into the room and call out or enter.

  As a result, a person other than the party of the conference may be able to see the image of the other party during the conference or be heard a conversation, resulting in an information security problem.

  On the other hand, conventionally, when another person comes in, the parties to the conference deal with it by turning off the image or turning off the sound by operating the terminal device.

  However, this kind of response depends on the actions of the person, and when the above situation occurs, it is not always possible to respond quickly. There is a risk that information leaks as a result.

  As a conventional general countermeasure against information leakage in information communication, for example, the remote access using VPN (Virtual Private Network) etc. is used in the network to access the data of the server connected to the corporate network. The method is used. In VPN, data is strictly protected and managed by authentication and encryption, so the risk of information leakage is reduced.

VPN uses a network to send and receive data, so when handling large volumes of data with a high network load, data is often stored in a terminal device to improve efficiency. Done.
In such a case, there is a possibility that information leakage of this data may occur for some reason unless a procedure for deleting data stored in the terminal device is performed after the work is completed.

  In order to deal with this problem, some companies place importance on the safety of information management, and some thin client systems have been introduced to employees who are outside the office, such as employees who are out or working from home. ing.

  A thin client is a system architecture in which a computer (client) used by an employee has a minimum function, and resources such as applications and files are managed not on the client side but on the server side.

  When this thin client system is used, data is not basically stored on the client side, so the risk of information leakage is reduced.

  In this way, general measures against information leakage have been taken, but these measures can be taken by other people other than the parties to the conference in the conference system, as described above, to the other party's image and voice of the conversation. Leakage is not intended and cannot be prevented.

  Further, as a conventional general technique, a technique for operating a device by using speaker identification has been proposed (for example, see Patent Document 1).

  In the technique described in Patent Document 1, a voice is input to the apparatus, and whether or not the voice is that of a specific user and identification of a word are performed from the input voice.

  Based on the identification result, if the input voice belongs to a specific user, the application program desired by the user is specified from the identified word, and the application program operation unique to the user is determined. Run the application program in the environment.

  With this technology, when someone else enters the conference room or working room, the user utters a specific word voice, which executes the specific application program from the input voice and You can turn it off or switch screens, so you can prevent you from seeing a screen you don't want others to see.

  However, this technology uses voice input instead of the user's key operation. Relying on the human action is the same as the one input by the key operator by the conference party as described above. The problem remains that there is a risk of occurrence.

  In addition, if the parties to the conference take action by turning off the image or turning off the sound by operating the terminal device, the conference cannot be continued and will be interrupted. May cause confusion.

  Such a situation is not limited to the case where a conference is held in the conference room or the normal room as described above, and a conversation between two parties or three or more parties is performed using a mobile phone terminal having a similar function. In some cases, similar problems exist.

  In this way, in a conference using the conference system, there is a risk that information other than the parties to the conference may see information on the other party in the conference or hear audio, resulting in information leakage. However, the present invention is intended to solve such problems.

  The features of the present invention as means for solving the above-described problems are as follows.

The terminal device of the present invention outputs the conference information of the other party transmitted from the other party's terminal device by the own terminal device, and transmits the conference information of the other party to the other party's terminal device. Identification means for performing acoustic analysis by collating with an acoustic model of noise and identifying whether the sound in the vicinity of the terminal device is a sound designated as a sound to which the conference information of the other party should be output; Limiting means for limiting the output of the other party's conference information when the means identifies that the other party's meeting information is not a sound designated as a sound to be output; a conversion means for converting the sound in the vicinity of the terminal device said self into a digital signal, the row stomach linear predictive analysis on the digital signals, and acoustic analysis processing means for obtaining the feature amount, versus the digital signal And section detection processing means for determining the interval for performing sound recognition processing Te performs matching between the acoustic model of the noise of the speaker and around the feature amount obtained by the acoustic analysis processing means or said specified sound Sound identification processing means for identifying whether or not the conference information output of the other party is a sound output by a speaker, and the restriction minimizes the output volume of the speaker before the restriction. It is characterized by being.

  According to the present invention, a restriction or restriction release is performed on the output of the other party's conference information, and a message indicating that restriction or restriction release has been performed is transmitted to the partner terminal apparatus. Since this can be known, the risk of information leakage in the conference is reduced, and the conference can be smoothly advanced with the other party of the conference.

The figure which shows the structural example of the remote conference system of embodiment of this invention. The hardware block diagram of the terminal device 100 of embodiment of this invention External view of terminal device 100 according to an embodiment of the present invention Functional block diagram of the terminal device 100 of the embodiment of the present invention Figure showing an example of message data The figure which shows the example of the display screen of the message receiving side when the meeting becomes impossible The figure which shows the example of the display screen of the message reception side when meeting becomes possible Diagram showing an example of the data structure of the speaker's acoustic model Figure showing an example of the data structure of an acoustic model of noise Figure showing another example of the data structure of the speaker's acoustic model Figure showing another example of data structure of acoustic model of noise Flowchart for restricting conference information output processing Flow chart for canceling restrictions on meeting information output processing Flowchart when a message for changing the conference information output process is received

  An embodiment of the present invention will be described.

  FIG. 1 is a diagram illustrating a configuration example of a remote conference system that performs a conference with a remote place as a conference system according to an embodiment of the present invention.

  In FIG. 1, reference numeral 100 denotes a remote conference system terminal device, which is connected to each other via a communication network 900 including a LAN 200, a dedicated line 300, and the Internet 400 to transmit / receive image data, audio data, and the like. Is done.

  A part of the terminal device 100 is used for telecommuting. In this case, the terminal device 100 is connected to the Internet 400 and connected to another terminal device 100 via the network 900.

  The LAN 200 is a network constructed in a local area such as a company office or a building, and each LAN 200 can communicate with other LANs 200 in the area A via the dedicated line 300 in the area A where the LAN 200 is located. It is connected to the. Each dedicated line 300 is communicably connected to the dedicated line 400 in another area A via the Internet 400.

  For example, the region A is set to a unit such as Japan or the United States, and the LAN 200 is set to a business unit of a company such as a Tokyo office or a New York office.

  In addition, the terminal device 100 for telecommuting is installed at the home of the telecommuter.

  Each LAN 200 and each dedicated line 300 is provided with a router 500, and the telecommuting terminal device 100 is connected to the Internet 400 via the router 500.

  The router 500 selects an optimal route for communication such as image data and voice data between the terminal devices 100 that conduct a remote conference connected to each LAN 200 or between them and the terminal device 100 working from home. Are connected between these terminal devices 100 or between them and the terminal device 100 working from home.

  A gateway device 550 is provided between each dedicated line 300 and the Internet 400.

  The gateway device 550 is a communication device for connecting to a network having a different protocol, and a gateway device 550 other than the one shown in this figure is provided between the networks as necessary.

  In FIG. 1, 600 is a relay device, 700 is a remote communication management system, and 800 is a program providing system.

The relay device 600 is provided for each LAN 200 and relays conference information such as image data and audio data when the terminal device 100 and another terminal device 100 are connected.
Also, the relay device, when there is a delay in receiving conference information such as image data with another terminal device 100 that is the counterpart of the terminal device 100, the image of the image data transmitted by the relay device 600. After the resolution is changed to reduce the amount of information, the image data is transmitted to the terminal device 100 of the other party.

  This eliminates the delay and ensures the synchronization of the conference and conference information communication.

  The remote communication management system 700 is connected to the Internet 400 and centrally manages the terminal device 100 and the relay device 600 based on a remote communication management program.

  For example, the remote communication management system 700 communicates with the terminal device 100 and the relay device 600, manages the registration of the terminal device 100, manages the ID and IP address of the terminal device 100, and manages the billing for use. Notification of other terminal devices 100 that can communicate, grasping the usage state of each terminal device 100, determination of the relay device 600 through which communication is performed, and the like are performed.

If necessary, the status of each terminal device 100 (not connected, logged in, during a meeting, etc.) is notified to the other terminal devices 100.
The program providing system 800 is connected to the Internet 400, stores a program for realizing various functions or various means used in each component of the remote conference system in a hard disk, and transmits the program to each component of the remote conference system. Is.

  Examples of programs stored and transmitted by the program providing system 800 include a conference device program used in each terminal device 100, a relay device program used in the relay device 600, and a remote communication management program used in the remote communication management system 700. .

  The terminal device 100 includes a device main body 110, a camera 130, and a display 140.

  When a telecommuter communicates with another terminal device 100 or the like from the terminal device 100 working at home via the communication network 900, a connection is established by VPN in order to protect and manage data to be communicated.

  The VPN can be constructed on the communication network 900 by encrypting and decrypting the communication data. In the embodiment shown in FIG. 1, the VPN is executed on one function of the router 500 or the terminal device 100 working from home. It can be realized as a program processing.

  In addition, since the Internet is used as the communication network 900, encryption and authentication are performed using protocols such as IPsec (Security Architecture for Internet Protocol) and PPTP (Point to Point Tunneling Protocol), which is equivalent to a dedicated line. A highly secure communication path can be secured.

  Similarly, for the terminal device 100 other than the telecommuting, the gateway device 550 can establish a VPN with the remote device 100 working from home.

  The gateway device may have a user authentication function and a device authentication function. By providing these functions, only the telecommuter is permitted to connect, and the correctness of the place where the device is installed is guaranteed. Security can be further enhanced.

  In addition, although the example of the conference between the places which are distant from each other is shown as a remote conference system, the conference is not only in the case of being away from the distance but also in the distance as in the conference room in the same building. Includes meetings between nearby conference rooms and places where the same room cannot hear the voice. In other words, the conference means a conference that is performed directly via a conference device, not a conference that is performed directly face to face. In addition, the conference includes not only a meeting between a plurality of people but also a meeting held by one person.

  The meeting information is shown for the image and sound during the meeting, but at least one of them may be used. For example, in the case of only an image, the attendee's voice may be displayed as subtitles in the image. In addition, the conference information includes things (for example, product samples) presented to the parties at the meeting, documents to be distributed, images that are not distributed but displayed on the screen by a display device such as a projector, and the like. As described above, the conference information means information that can be grasped by the attendees at the meeting place in the case where the conference information is held directly and face-to-face.

FIG. 2 is a hardware configuration diagram of the terminal device 100 according to the embodiment of this invention.
As shown in FIG. 2, the terminal device 100 includes a CPU (Central Processing Unit) 111 that controls and processes the entire operation of the terminal device 100, and a ROM (Read that stores various programs such as the terminal device 100. SSD, which stores various setting values of the terminal device 100 and RAM (Random Access Memory) 113 used as a work area of the CPU 111 and reads or writes data in the flash memory 115 according to the control of the CPU 111 (Solid State Drive) 114, a media drive 116 for reading or writing data on a recording medium 116a such as a flash memory, a cursor operated when selecting the destination of the terminal device 100 of the other party of the conference or performing various inputs Etc., and the operation button 117 and the terminal device 100 are turned on / off. It has a power switch 118 for FF and a network I / F 119 for data communication of conference information.

  The terminal device 100 also includes a camera 130 that captures an image of a meeting room of a party or the like and outputs image data, a camera IF 120 that connects to the camera 130, the voice of a person such as the party, ambient noise, and the like. The microphone 121 that collects and inputs sound including non-speech sounds, the speaker 122 that outputs the sound of the other party's conference room, the input of sound signals to the microphone 121 and conversion into sound data, and the speaker 122 A sound input / output I / F 123 that converts sound data into sound signals and outputs them, a display 140 that displays an image of the other party's conference room, and a display I / F 124 that outputs image data to the display 140.

  Among these, the display 140 is externally attached, and the camera 130, the microphone 121, and the speaker 122 are connected to the device main body 110 of the terminal device 100 and housed integrally.

  Reference numeral 125 denotes an external device connection I / F that connects these cameras, microphones, speakers, and the like when they are externally attached.

  Reference numeral 126 denotes a bus line such as an address bus or a data bus for electrically connecting the above components.

  The flash memory 115 is not limited to a flash memory as long as it is a nonvolatile memory, and may be, for example, an EEPROM (Electrically Erasable and Programmable ROM).

  The camera 130 uses CMOS (Complementary Metal Oxide Semiconductor) as an element that digitizes a subject image as a moving image or a still image, and outputs USB standard data. Note that not only the CMOS but also a CCD may be used.

  FIG. 3 is an external view of the terminal device 100 according to the embodiment of the present invention.

  As illustrated in FIG. 3, the terminal device 100 includes a housing 1021, an arm 1074, and a camera housing 1075.

  A mesh-like air intake hole 1021e is formed on substantially the entire front wall 1021a of the housing 1021, and a mesh-like exhaust hole (similar to the air intake hole 1021e) is formed on substantially the entire rear wall 1021b of the housing 1021. As a result, by driving a cooling fan built in the housing 1021, outside air in front of the terminal device 100 is taken in through the intake hole 1021e of the front side wall 1021a, and the rear side wall 1021b It is possible to exhaust the rear of the terminal device 100 through the exhaust hole.

  A sound collecting hole 1021f is formed in the central portion of the front side wall 1021a, and sounds including non-voice sounds such as voice and ambient noise can be collected by the built-in microphone 121.

  An operation panel 1022 is formed on the upper left side of the housing 1021. The operation panel 1022 is provided with an operation button 117 and a power switch 118, and a mesh-like sound output hole 1022f through which an output sound from the built-in speaker 122 is passed.

  Further, a concave portion for accommodating the arm 1074 and the camera housing 1075 is formed on the right side of the upper surface of the housing 1021 as the accommodating portion 1021p.

  The arm 1074 is attached to the housing 1021 via the torque hinge 1073. The arm 1074 has a pan angle θ1 of ± 180 degrees with respect to the housing 1021, and a range of a pan angle θ1 of ± 180 degrees and a tilt angle θ2 of 90 degrees (a feeling of clicking occurs when tilting approximately 45 degrees) It is configured to be rotatable in the vertical and horizontal directions.

  The camera housing 1075 is provided with a built-in camera 130, which can shoot a conference room or the like of parties to the conference. The camera housing 1075 is formed with a torque hinge 1075a.

  The camera housing 1075 is attached to the arm 1074 via a torque hinge 1075a. The camera housing 1075 is set to 0 degrees when the camera housing 1075 is linear with respect to the arm 1074, and is approximately 100 degrees on the front side of the terminal apparatus 100 and approximately 90 degrees on the rear side of the terminal apparatus 100. The tilt angle θ3 is configured to be rotatable.

  A connection port 1021g corresponding to the external device connection I / F 125 is formed in the lower portion of the side surface of the housing 1021, and cable terminals such as an external camera, a microphone, and a speaker are connected to the connection port 1021g.

  Reference numeral 140 denotes an external display 140.

FIG. 4 is a functional block diagram of the terminal device 100 in the present embodiment.
The terminal device 100 includes a transmission / reception unit 11, an operation input reception unit 12, a login request unit 13, an imaging unit 14a, an image display control unit 14b, a sound input unit 15a, a sound output unit 15b, a final narrowing unit 16, and a delay detection unit 17. A storage / reading processing unit 18 and a leakage prevention processing unit 19.

  Each of these units is a function realized by any one of the components shown in FIG. 2 operating according to a command from the CPU 111 according to a program stored in the ROM 112.

  Further, the terminal device 100 includes a storage unit 1000 whose reading or writing is controlled by the SSD 114 illustrated in FIG.

  Each functional block of the terminal device 100 will be described.

  The transmission / reception unit 11 of the terminal device 100 is realized by the network I / F 119 illustrated in FIG. 2, and transmits / receives various data to / from other terminal devices 100 and other devices or systems via the communication network 900.

  The operation input reception unit 12 is realized by the operation buttons 117 shown in FIG. 2 and receives various inputs by the user. For example, when the user selects the terminal device 100 of the other party of the conference, when the user selects the terminal device 100 on the screen of the operation display screen (not shown) of the terminal device 100 and presses the decision button, the selection is made. Information is entered.

  Further, the operation input accepting unit 12 accepts input of restriction instruction information for restricting the output of the conference information of the other party performed by the terminal device 100 and input of this restriction release instruction information. In either case, the parties to the conference operate the operation buttons 117.

  The login request unit 13 is realized by a command from the CPU 111 shown in FIG. 2, and login request information indicating that a login is requested from the transmission / reception unit 11 to the remote communication management system 700 via the communication network 900, and The current IP address of the terminal device 100 is transmitted.

  This transmission is set to be automatically performed when the power supply switch 118 shown in FIG. 2 is turned on to turn on the terminal device 100 and the terminal device 100 is started up.

  The imaging unit 14a is realized by the camera 130 and the camera I / F 120 shown in FIG. 2, and takes an image of a meeting room such as an attendee who is a subject and outputs the image data obtained by the shooting. To do.

  The image display control unit 14b is realized by the display I / F 124 shown in FIG. 2, and performs control for transmitting and displaying image data to the external display 140.

  The sound input unit 15a is realized by the microphone 121 and the sound input / output I / F 123 shown in FIG. 2, collects and inputs sound in the vicinity of the terminal device 100, and converts the sound signal of this sound into sound data. Output. Collected sounds include voice and non-voice sounds.

  The sound output unit 15b is realized by the speaker 122 and the sound input / output I / F 123 shown in FIG. 2, and converts a sound signal related to sound data into sound and outputs the sound.

  The final narrowing unit 16 is for performing a final narrowing process for narrowing down from a plurality of relay devices 600 to one relay device 600 to be finally used, and includes a measurement unit 16a, a calculation unit 16b, and a final selection unit 16c. This is realized by a command from the CPU 111 shown in FIG.

  Among these, the measurement unit 16a measures the reception date and time (in msec units) when the pre-transmission information is received by the transmission / reception unit 11, for each of the later-described pre-transmission information received by the transmission / reception unit 11.

  For each pre-transmission information whose reception date and time is measured by the measurement unit 16a, the calculation unit 16b determines the pre-transmission information based on the difference between the measured reception time and the transmission date and time included in the pre-transmission information. The required time (msec unit) from transmission to reception is calculated.

  The final selection unit 16c finally selects 1 as the relay device 600 that uses the relay device 600 to which the pre-transmission information that required the shortest required time from among the required times calculated by the calculation unit 16b is relayed. One relay device 600 is selected.

  The delay detection unit 17 is realized by a command from the CPU 111 shown in FIG. 2, and calculates the delay time (msec unit) of image data or audio data sent from another terminal device 100 via the relay device 600. To detect.

  The storage / read processing unit 18 is executed by the SSD 114 shown in FIG. 2, and stores various data in the storage unit 1000 and reads various data stored in the storage unit 1000.

  The storage unit 1000 stores a terminal ID (Identification) for identifying the terminal device 100, a password, image data, sound data, and the like.

  Note that the terminal device ID of this embodiment and the relay device ID, which will be described later, are a language, characters, symbols, various marks, or combinations thereof used to uniquely identify the terminal device 100 and the relay device 600, respectively. Indicates identification information.

  The information leakage prevention unit 19 analyzes the sound generated in the vicinity of its own terminal device 100, and when this sound is not a predesignated sound or when restriction instruction information is input by a party of the conference, While restricting the output processing of the conference information such as the image display of the other party of the conference being performed by the terminal device 100 and the sound output of the other party, that the other terminal device 100 has performed this restriction. When the message indicating that this restriction has been performed is received at the other terminal device 100, the message is displayed on the own terminal device 100.

  The information leakage prevention unit 19 includes a sound identification unit 19a, a determination unit 19b, a restriction unit 19c, and a state management unit 19d.

  The sound identification unit 19a performs an acoustic analysis of the sound input from the microphone 121 of the sound input unit 15a, compares the sound feature amount with an acoustic model expressing the acoustic feature provided in advance, Whether or not the voice of a speaker such as a party of the conference is identified.

  When it is determined that the input sound is not the sound specified by the sound identification unit 19a, the determination unit 19b determines that the input sound is not the sound for outputting the conference information of the other party, and the conference When the restriction instruction information is input by the party, the input is received, and it is determined that the restriction on the output of the conference information such as the image display and voice output performed by the terminal device 100 is determined.

  Further, when the restriction cancellation release instruction information is input by the party to the conference, the determination unit 19b receives the input and determines that the limitation should be released.

  When the determining unit 19b determines that the output of the conference information should be limited, the limiting unit 19c limits the output of the conference information such as image display and audio output performed by the terminal device 100. To change the output contents.

  When the determination unit 19b determines that the restriction should be released, the restriction unit 19c releases the restriction.

  The state management unit 19d performs management for notifying the terminal device 100 of the other party of the conference of the change status of the output processing of its own terminal device 100, and changes the output processing of the terminal device 100 from the terminal device 100 of the other party of the conference When there is a notification of the state, the terminal device 100 of its own manages to display that fact.

The storage unit 1000 stores data of the acoustic model corresponding to the sound used in the information leakage prevention unit 19 and various data.
The display of the output process change of the counterpart terminal device 100 by the state management unit 19d is performed on the display 140 under the control of the image display control unit 14b.

  Next, the determination unit 19b, the restriction unit 19c, and the state management unit 19c will be described specifically. The sound identification unit 19a will be described later.

  If the input sound is not a sound that should output the other party's conference information, the determining unit 19b determines that the other party's conference information output process performed by the terminal device 100 of itself should be restricted. .

  In response to this, the restricting unit 19c restricts the output process of the other party's conference information performed by its own terminal device 100. As this restriction, the restricting unit 19c applies to the image display control unit 14b. Instructing the display 140 to turn off the screen, etc., obtaining the output volume data before the restriction (current time) from the sound output unit 15b and storing it in the storage unit 1000, and then minimizing the output volume. Instruct.

When the terminal device 100 uses Windows (registered trademark of Microsoft Corporation) as the OS, the instruction processing for the image display control unit 14b is assigned to the keyboard and function keys. This can be implemented by substituting the target key signal (for example, F3 key) and interrupting the processing of the terminal device 100.
Also, the instruction processing for the sound output unit 15b can be executed using the audio mixer API if this processing is Windows (registered trademark of Microsoft Corporation). Note that the minimum value for minimizing the output volume may be 0.

  In addition, when the party of the conference determines that the conference information of the other party should be restricted without being identified by the sound identification unit 19a and the determination of the determination unit 19b, the party operates by operating the operation button 117. The input receiving unit 12 directly inputs restriction instruction information.

  When there is this input, the determination unit 19b determines that the output of the conference information should be restricted, and is the same as the case where it is determined that the input sound is not the sound to output the other party's conference information. Execute the process.

  When there is an operation input of release instruction information performed by a party to the conference when the restriction is no longer necessary, the determination unit 19b determines that the restriction should be released in response to the input. The operation input of the release instruction information is performed by the operation input receiving unit 12 by operating the operation button 117.

  The canceling operation input is performed by inputting a password. When this input is made, the determination unit 19b compares the input information with a preset password, and if they match, the restriction should be canceled. Is determined.

When the determination unit 19b determines that the restriction is released, the restriction unit 19c instructs the image display control unit 14b to shift to screen on.
Further, the restriction unit 19c refers to the output volume data before restriction of the speaker 122 stored in the storage unit 1000, and instructs the speaker 122 to restore the volume to the output volume before restriction by the volume data. .

  When the restriction unit 19c instructs the state management unit 19d to turn off the image and minimize the output volume, the state management unit 19d determines that the restriction has been performed on the terminal device 100 of the other party of the conference. A message to be shown is created and an instruction to notify the counterpart terminal device 100 is given to the transmission / reception unit 11.

  Specifically, the state management unit 19d creates an XML (Extensible Markup Language) based message according to XMPP IM (Extensible Messaging and Presence Protocol: Instant Messaging and Presence) of RFC3921 as shown in FIG.

  In this case, the state management unit 19d designates the identifier of the terminal device 100 and the like of the own terminal device 100 and the identifier of the transmission destination terminal device 100 in the attributes “from” and “to” of the <presence> tag.

  In the example of FIG. 5A, its own terminal device 100 or the like is designated as ID1, and the identifier of the counterpart terminal device 100 or the like that is the transmission destination is designated as ID4.

  For the <show> tag value, specify “CHAT” to indicate that a meeting is in progress.

  Specify the screen status in the <status> tag value. In FIG. 5A, “DISPLAY_OFF” indicating the state of “screen off” is designated.

  The state management unit 19d sends this message to the transmission / reception unit 11 to instruct the transmission of the message. As a result, the transmission / reception unit 11 transmits a message by XMPP.

  In the case where the restriction by the restriction unit 19c is released in the terminal device 100 of its own and there is an instruction to shift from the screen off to the screen on and an instruction to restore the volume, the state management unit 19d displays the state in FIG. A message is generated and the transmission / reception unit 11 is instructed to transmit a message in the same manner as when the screen is off.

  In the message shown in FIG. 5 (B), the attributes “from” and “to” of the <presence> tag include the identifier ID1 of the terminal device 100 of its own and the destination partner as in the case of FIG. 5 (A). The identifier ID4 of the terminal device 100 or the like is designated.

  As for the value of the <show> tag, “CHAT” indicating that the conference is in progress is designated as in the case of FIG.

  For the value of the <status> tag, specify “DISPLAY_ON” indicating the status of “screen on”.

  Note that the <status> tag value specifies “DISPLAY_OFF”, which indicates the status of “screen off”. This “screen off” indicates that there is a restriction on the output processing of conference information. This is representatively shown, and includes contents indicating a state of minimizing the output volume of the speaker 122.

  The notification message of the change status of the output process is received by the transmission / reception unit 11 of the counterpart terminal device 100 that is the transmission destination, but the status management unit 19c of the counterpart terminal device 100 receives the message received from the transmission / reception unit 11 as follows. Analyze as follows.

  First, the identifier (ID1 in this embodiment) of the device 100 of the conference partner that is the transmission source is acquired from the “from” attribute of the <presence> tag, and then the screen status is acquired from the value of the <status> tag. .

  If the acquired screen state is DISPLAY_OFF, the state management unit 19d recognizes that the output of the conference information is restricted in the partner conference device.

  Then, he / she instructs his / her image display control unit 14b to display that the conference is impossible.

  FIG. 6 shows an example of a display screen on the message receiving side when the conference becomes impossible. In this figure, together with the image of the other party of the conference as the conference information, specifically, a mark indicating a state in which the conference cannot be temporarily performed is superimposed on the image of the other party. The meaning of the mark may be additionally displayed as text as shown on the right side of the screen.

  If the acquired screen state is DISPLAY_ON, the state management unit 19d recognizes that the restriction on the output of conference information has been lifted in the other party's conference device, and displays a screen indicating that a conference is possible as shown in FIG. The display is instructed to its own image display control unit 14b. That is, as shown in FIG. 7, the mark in FIG. 6 is terminated and hidden, and when the meaning of the mark is additionally displayed, the display of the text portion to which the meaning of the mark is added is terminated. You want to hide.

  In this embodiment, a message regarding information regarding restrictions is directly exchanged between the terminal devices 100 having a conference, but the message may be routed through the remote communication management system 700. Further, the restriction status may be transmitted to the remote communication management system 700 separately from the exchange between the terminal devices 100.

  Next, the sound identification unit 19a will be specifically described.

The sound identification unit 19a has the following functions.
(1) Conversion process from analog sound signal to digital signal (2) Acoustic analysis process (3) Section detection process (4) Sound identification process Each function will be described.
(1) Conversion of sound analog signal into digital signal Input of a sound analog signal collected by the microphone 121 is received, and the analog signal is converted into, for example, a digital signal of 8 kHz, 16 bits by an AD converter. This conversion process is performed in the sound input / output I / F 123 of FIG.
The conversion process by the sound identification unit 19a is a process for sound identification and differs from the conversion process of the sound input unit 15a in terms of content, but the sound input unit is different in that it converts an analog sound signal into a digital signal. Since it is the same as the conversion processing of 15a, the processing content is set so that this conversion processing can be commonly used for both purposes.

  Therefore, these processes may be performed separately.

(2) Acoustic analysis processing The acoustic analysis processing is performed on the digital signal of the sound converted in (1).

  For this analysis processing, LPC (linear prediction) analysis or the like can be used. LPC analysis analyzes this digital signal by removing fine frequency components from the signal and predicting the strength and frequency outline (envelope) of the remaining part.

  Since digital signals change over time, this analysis is performed on a set of signals over a period of time, ie, a frame.

  This LPC analysis is performed, for example, under the condition that the high-frequency emphasis is a first order difference, a Hamming window is cut out to 256 points, the movement width is 16 ms, and the LPC analysis order is 20. A feature quantity such as a difference and logarithmic power is obtained for each frame.

  Specifically, in this LPC analysis, first, a high frequency enhancement filter is applied to the sound waveform data, and then a window function is applied. By applying this window function, it is possible to reduce the influence of discontinuous points that are generated when cut out in units of frames.

  A linear prediction coefficient is obtained so that the sum of squares of prediction errors of signals cut out in units of frames is minimized. The cepstrum coefficients from 1 to 10 are obtained by recursive calculation from the obtained prediction error and linear prediction coefficient.

  The mel cepstrum coefficient is obtained by converting the cepstrum into a mel cepstrum by Oppenheim's recursion method. Since the logarithmic power, which is an auxiliary feature amount, can be obtained in the process described above, the logarithmic power temporal primary difference is also calculated each time the logarithmic power is obtained.

  The sound analysis is not limited to the method described here, and a method such as frequency (FFT) analysis can also be used.

(3) Section Detection Processing Further, section detection processing is performed for determining a section in which sound identification processing is performed on the digital sound signal converted by the AD converter.

  In the section detection process, for example, the section is detected by observing and analyzing the sound logarithmic power and the primary autocorrelation. Autocorrelation is a measure of how well a signal matches a time shifted signal and is expressed as a function of the magnitude of the time shift.

  When a section is detected, feature quantities such as the mel cepstrum coefficient and logarithmic power of the section corresponding to the detected section are sent to the sound identification process. If not detected, the acoustic analysis process and the section detection process are repeated.

  By performing this section detection process, the accuracy of the identification process described later can be increased. If the accuracy of the model used in the identification process is sufficiently high, this section detection process can be omitted.

(4) Sound identification process In the sound identification process, the feature quantity of the detected section is collated with an acoustic model expressing the acoustic feature. For example, a typical HMM (Hidden Markov Model) can be used as the acoustic model. In addition, a GMM (Gaussian mixture distribution model) can also be used.

  As an acoustic model used in the sound identification unit 19a, an acoustic model of a speaker who is a party of the conference and an acoustic model of ambient noise are provided in advance.

  FIG. 8 is a diagram showing an example of the data structure of the speaker acoustic model used in the sound identification unit 19a. In this data structure, there are a total of n HMMs, and HMMs corresponding to different n speakers are stored in the storage unit 1000 in association with numbers. These HMMs are modeled corresponding to people who normally use the terminal device 100.

  FIG. 9 is a diagram showing a data structure of an acoustic model of noise used in the sound identification unit 19a. In this data structure, there are a total of m HMMs, and HMMs for different m types of noise are stored in the storage unit 1000 in association with numbers. Each of these HMMs has a characteristic noise such as a door opening / closing sound that indicates that a person has entered the room where the terminal device 100 is used, and other noise such as an operation sound of the terminal device 100. The types are distinguished and stored in the storage unit 1000.

  Of these acoustic models, the acoustic model of the speaker who is the party to the conference and the characteristic acoustic noise model that indicates that a person has entered, for example, the noise model other than the door opening / closing sound, outputs the conference information of the other party It is designated in advance as an acoustic model of the sound to be played.

  Each HMM expresses a speaker's voice or noise feature quantity based on a statistical appearance probability.

  These HMMs are preferably trained in advance using the Baum-Welch algorithm. For the training data, voices of each speaker and various noise feature values collected in advance can be used.

  The Baum-Welch algorithm is an algorithm for learning parameters so as to maximize the likelihood of training data, and is used for re-estimation of the HMM initial model.

  In the sound identification unit 19a, the feature quantity of the detected section is collated with the HMM shown in FIG. 8 or FIG. 9 by L. Rabiner et al. The score Si for each HMM of the acoustic model or noise model is calculated by the Viterbi algorithm as described in detail in Reference 1).

  Here, i is a number for the HMM, and i = S1,..., Sn, N1,. A forward algorithm can be used instead of the Viterbi algorithm. The Viterbi algorithm is an algorithm for obtaining an optimal state sequence and likelihood on this route in a certain model.

  Then, the calculated scores Si are compared, and the number I of the HMM having the highest score in the acoustic model or the noise model is obtained by the following equation.

番号IをもつHMMの得点SIが次式の判定式から、しきい値TIより大きくなる場合は、有効な得点とする。

If the score SI of the HMM having the number I is larger than the threshold value TI from the following judgment formula, the score is valid.

この実施形態では、モデルごとにしきい値が異なるようにしているが、これに限られるものではなく、すべてのモデルに対して同一のしきい値を用いたり、図５に示す話者HMMと図６に示す騒音HMM用の２つのしきい値を用いるなど、他のしきい値設定を用いてもよい。

In this embodiment, the threshold value is different for each model. However, the present invention is not limited to this, and the same threshold value is used for all models, or the speaker HMM shown in FIG. Other threshold settings may be used, such as using two threshold values for the noise HMM shown in FIG.

  In this way, the number of an HMM having a valid score is determined. Based on this, sound identification is performed as follows.

When the number of the HMM having a valid score starts with S, it can be considered that the party of the terminal device 100 being used uttered.
On the other hand, if the HMM number with a valid score starts with N and is a characteristic noise model indicating that a person has entered the room, it can be considered that someone else has entered the room. .

  In addition, when an HMM starting with N other than that is selected, it can be regarded as a mere noise.

  If none of the HMM scores shown in FIG. 8 and FIG. 9 are valid scores, it may be considered that someone other than the party of the terminal device 100 speaks in the room where the terminal device 100 is installed. it can.

  The HMMs shown in FIGS. 8 and 9 are stored for each speaker or for each type of noise, but two fully coupled (ergodic) HMMs of speakers and noise are prepared, and the state of each HMM It is also possible to collate by assigning types or noise types.

  In this case, examples of the structures of the speaker HMM and the noise HMM stored in the data storage unit 42 are shown in FIGS. 10 and 11, respectively.

  In FIG. 10, the HMM has a total of n states, and probability density functions for different n speakers are stored in association with the states. In FIG. 11, there are a total of m states, and probability density functions for m different types of noise are stored corresponding to the states.

The probability that the feature quantity extracted from this probability density function appears can be obtained. Although not shown, the transition probabilities aSji (i = 1,..., N) and aNji (i = 1,. Stored in the section.
In the identification processing using the HMM as described above, the Viterbi algorithm as described in detail in the above-mentioned reference 1 is used to collate the detected section with the HMM shown in FIGS. 8 and 9. The score Ss for each speaker HMM and the score Sb for the noise HMM are obtained. If the score is larger than the threshold value by the following equation, the score is effective.

ここで、Ts、Tbは、話者HMMの得点に対するしきい値と騒音HMMの得点に対するしきい値である。この実施形態では、HMM毎にしきい値が異なっているが、これに限ったものではなく、全てのモデルに対して同一のしきい値を用いることもできる。

Here, Ts and Tb are a threshold for the score of the speaker HMM and a threshold for the score of the noise HMM. In this embodiment, the threshold value is different for each HMM, but is not limited to this, and the same threshold value can be used for all models.

  In this way, the number of an HMM having a valid score is determined. Based on this, speaker identification is performed as follows.

If the HMM having a valid score is a speaker HMM, it can be considered that the person using this terminal apparatus 100 has made a speech.
On the other hand, when the HMM having an effective score is a noise HMM, the trace back processing by the Viterbi algorithm as described in detail in the above-mentioned reference 1 is performed, and the state of the noise HMM corresponding to the frame of the speech feature amount Find the series.

  Then, the state number that stays longest continuously from this series is obtained (for example, when the state series is 113333324, the state number 3 which is the largest number is the number to be obtained). When this state number corresponds to a characteristic state number indicating entry, it can be considered that there is entry.

When the state number is a number other than the number corresponding to the characteristic state number indicating entry, it can be regarded as a mere noise.
If neither of the HMM scores is valid, it can be considered that a person who does not use the terminal device 100 speaks in a room where the terminal device 100 is installed.

  The speaker identification unit 19a in the present embodiment uses an HMM as an acoustic model, but other models can also be used.

  For example, a standard pattern obtained by arithmetically averaging feature values consisting of several frames, artificial neural network, Muroi et al., "Word recognition using duration control state transition model", IEICE Transactions J72-D-II 11, pp. 1769-1777 (1989-11) (Reference 2), it is also possible to use a DST (Duration-based State Transition) model.

  Compared to the HMM, the DST model can express the temporal structure of the speech pattern uttered by the speaker, and is particularly effective for identifying a specific speaker. Therefore, in a system environment where only one telecommuter or a small number of speakers need to be identified with the system configuration as shown in FIG. 1, the DST model is used in place of the HMM to identify speakers more accurately. be able to.

Next, processing performed by the terminal device 100 will be described with reference to a flowchart.
FIG. 12 is a flowchart showing a flow in the case of limiting the conference information output process in its own terminal device 100, and the processing in its own terminal device 100 in this case will be described with reference to this.

The information leakage prevention unit 19 starts this processing from step 30 when a conference is being held by the remote conference system.
First, in step 31, it is determined whether or not there is an instruction operation input. If there is an input, in step 37, the determination unit 19 b confirms whether or not the instruction input is an instruction to perform a restriction. If it is the instruction information, it is determined that the restriction should be performed.

  If it is determined that the restriction should be performed, the process proceeds to the restriction process in step 38 and the restriction process is performed. If it is not determined that the restriction should be performed, the process returns to step 31.

  If it is determined in step 31 that no instruction has been input, the flow advances to step 32 to perform acoustic analysis.

  In step 32, the sound identification unit 19a accepts an input of an analog signal of sound collected by the microphone 121. The input sound is converted into a digital signal of, for example, 8 kHz and 16 bits by an AD converter.

  The conversion to the digital signal is continuously performed while the microphone 121 collects sound, and the converted digital signal is output to the bus line 126.

  In step 33, the sound identification unit 19a performs an acoustic analysis process on the digital signal of the converted sound. This acoustic analysis process is continuously performed while a digital sound signal is output for a block of digital signals for a certain period, that is, a frame.

  As an analysis result of the acoustic analysis process, feature quantities such as a mel cepstrum coefficient and logarithmic power are output to the bus line 126.

  In step 34, the sound identification unit 19a performs a section detection process on the converted digital signal.

  The section detection process detects a section to be identified, and is continuously performed while the sound digital signal is output.

  Next, in step 35, the sound identification unit 19a confirms whether or not a section is detected in step 34. If a section is detected, the process proceeds to a sound identification process in step 36.

  If no section is detected, the process returns to step 31.

  If a section is detected, in step 36, the sound identification unit 19a performs sound identification processing. In this sound identification process, the feature amount of the section of the acoustic analysis process detected in step 34 is collated with the acoustic model expressing the acoustic feature stored in the storage unit 1000, and the specified story It is identified whether or not the voice of the person.

  Specifically, it is discriminated whether the voice is uttered by a party such as an attendee of the meeting or the noise other than the voice of the person, such as the opening / closing sound of the door when the person enters the room.

  In step 37, based on the identification result of the sound identification process, the determination unit 19b determines whether or not the conference information output process should be restricted.

  Regarding audio, it is judged that it should be restricted except in the case of voices uttered by parties such as attendees of conferences. Judge that it should be.

If it is determined in step 37 that the restriction should be made, the process proceeds to a restriction process in step 38.
If it is not determined that it should be restricted, the process returns to step 31.

  In step 38, the restriction unit 19b instructs the screen display control unit 14b to turn off the screen, and turns off the screen of the display 140.

  In step 39, the restricting unit 19c instructs the audio output unit 15b to store the output volume data before the restriction (current time) and minimize the audio output volume. Thereby, the output volume of the speaker 122 is minimized.

  Next, in step 40, the state management unit 19d performs a message creation process as shown in FIG. 5A indicating that the restriction of the output process of the conference information has been performed in its own terminal device 100.

  Next, in step 41, the message created in step 40 is transmitted. This transmission process is performed when the state management unit 19d instructs the transmission / reception unit 11 to transmit a message, and the transmission / reception unit 11 performs the transmission process.

  In the flowchart of FIG. 12, the case where the restriction on the output process of the conference information has already been performed is not particularly shown, but the restriction on the output process is not continued. It is possible to prevent this process from being started when the restriction is already being continued as a start condition.

  FIG. 13 is a flowchart showing a flow when the restriction on the conference information output process performed in the terminal device 100 is cancelled, and the processing in the terminal device 100 in this case will be described with reference to this flowchart.

The information leakage prevention unit 19 starts this process from step 50.
First, in step 51, it is determined whether or not there is an instruction operation input. If there is an input, in step 52, the determination unit 19b confirms whether or not the instruction input is a valid release instruction input. It is determined whether the restriction should be removed.
If it is determined that it should be canceled, the process proceeds to step 53 and subsequent steps.
If it is not determined that the restriction should be made, in step 51, monitoring for the presence of a release operation input is continued.

  If it is determined in step 52 that it should be released, in step 53, the restriction unit 19c instructs the screen display control unit 14b to turn on the screen, and restores the screen of the display 140 to on.

  Next, at step 54, the restricting unit 19c instructs the sound output unit 15b to return the sound output volume of the speaker 122 to the output sound volume before the restriction, and restores the output sound volume of the speaker 122.

  Next, in step 55, the state management unit 19d performs a message creation process as shown in FIG. 5B indicating that the restriction on the output process of the conference information has been lifted in the terminal device 100 of itself.

  Next, in step 56, the message created in step 55 is transmitted. This transmission process is performed when the state management unit 19d instructs the transmission / reception unit 11 to transmit a message, and the transmission / reception unit 11 performs the transmission process.

  FIG. 14 is a flowchart showing a flow when a message indicating a change in the output process of the conference information is received from the terminal device 100 of the other party of the conference. With reference to this, the reception side that is the other party of the conference in this case is referred to Processing in the terminal device 100 will be described.

  This processing is started from step 60, and the receiving side terminal device 100 performs message reception processing in step 61.

  This message reception process is performed by the transmission / reception unit 11 of the terminal device 100 on the receiving side. The message shown in FIG. 5A created in step 40 of FIG. 12 and the message shown in FIG. 5 created in step 55 of FIG. The message shown in (B) is received.

  Next, in step 62, the state management unit 19d of the terminal device 100 on the receiving side performs processing for analyzing the message received from the transmission / reception unit 11.

  That is, when the message is as shown in FIG. 5A, since the screen state is “DISPLAY_OFF”, the state management unit 19d changes the display 140 to the screen off in the terminal device 100 of ID1 on the transmission side. And that the sound output volume of the speaker 122 is minimized.

  When the message is as shown in FIG. 5B, the screen state is “DISPLAY_ON”, so that the state management unit 19c restores the display 140 to the screen-on state in the terminal device 100 on the transmission side, and the speaker. It is recognized that the audio output volume 122 has been restored to the original volume.

  Next, in step 63, the result of the analysis process in step 62 is confirmed. If the result is that the transmission side is screen off and the audio output volume is minimized, the state management unit 19d is the transmission source as shown in FIG. It instructs the image display control unit 14b on the receiving side to display a mark indicating a situation where the conference cannot be temporarily performed over the image of the other party of the conference.

  In step 63, if the result of the analysis processing in step 62 is that the transmission side is returned to the screen-on state and the audio output volume is restored, the state management unit 19d ends the display of the mark in FIG. 6 as shown in FIG. And instructing the image display control unit 14b on the receiving side to hide it.

  In step 64 or step 65, the display instructed in step 63 is performed on the display 140, and the process ends in step 66.

  In the above-described embodiment, when a conference is being performed using the remote conference system, the sound collected at the conference location is identified by the terminal device 100 of the remote conference system, and the designated speaker is identified. Or whether the door is opened or closed due to a person entering the room.

  Then, based on this identification result, when a voice other than the voice of the designated speaker or a door opening / closing sound due to a person entering the room is identified. The terminal device 100 performs processing for restricting output processing of conference information such as a screen and audio output at the location of the conference itself. A similar restriction process is also performed when restriction instruction information is input by a party of the conference.

  While this restriction process is performed, a message indicating that the restriction process has been performed at the terminal device 100 of the own party is transmitted to the terminal device 100 of the other party of the conference, and the restriction process is performed by the partner device 100 of the other party. Is displayed.

  The restriction process is canceled when a person such as a party of the conference inputs a cancellation operation to the terminal device 100. When the cancellation input is made, the restriction is canceled and the terminal device of the other party of the conference A message indicating that the restriction process has been canceled in its own terminal device is transmitted to display that the other conference device 100 has canceled the restriction process.

  Therefore, it is possible to reduce information leakage when someone other than the party of the conference enters the conference room, and the other party of the conference can understand that the conference is temporarily suspended for this reason. The conference can proceed smoothly without occurring.

Note that the output processing when the partner terminal device 100 that is the transmission destination receives the notification message of the change status of the output processing is performed on the display 140, but may be performed by the speaker 122, or may be used in combination. Also good.
Further, another output device, for example, a lamp may be separately provided to perform lighting.

  Further, the limitation on output may be separate for audio and screen depending on the situation. For example, when the screen is in a position where it is difficult for others to see, it may be possible to limit only the sound.

  In addition, although said embodiment was performed about the case where a meeting is performed in a conference room or a normal room, it is not restricted to this, The conversation between two persons or three or more persons is carried out using the mobile phone terminal which has the same function. The conference system and the terminal device of the present invention include a system and a mobile phone terminal used in such a case.

100 Conference device 110 Device body 111 CPU
112 ROM
113 RAM
114 SSD
115 Flash memory 116 Media drive 116a Recording medium 117 Operation button 118 Power switch 119 Network I / F
120 Camera IF
121 Microphone 122 Speaker 123 Audio input / output I / F
124 Display I / F
125 External device connection I / F
126 Bus line 130 Camera 140 Display 1021 Case 1021e Air intake hole 1021f Sound collection hole 1021g Connection port 1022 Operation panel 1074 Arm 1075 Camera housing 200 LAN
300 leased line 400 internet A area 500 router 600 relay device 700 remote communication management system 800 program providing system 900 communication network 11 transmission / reception unit 12 operation input reception unit 13 login request unit 14a imaging unit 14b image display control unit 15a audio input unit 15b audio Output unit 16 Final selection unit 16a Measurement unit 16b Calculation unit 16c Final selection unit 17 Delay detection unit 18 Storage / reading processing unit 19 Leakage prevention processing unit 19a Speaker identification unit 19b Judgment unit 19c Restriction unit 19d Status management unit

JP 2000-99076 A

Claims (14)

In the conference system that outputs the conference information of the other party transmitted from the other party's terminal device in its own terminal device, and transmits its own conference information to the other party's terminal device.
Identification that identifies whether the sound in the vicinity of the terminal device is the sound that is designated as the sound to which the conference information of the other party should be output by collating with the acoustic model of the speaker and the surrounding noise Means,
Limiting means for limiting the output of the other party's conference information when the identification means identifies that the other party's conference information is not a sound designated as a sound to be output;
Have
The identification means includes
Conversion means for converting sound in the vicinity of the terminal device to a digital signal;
And acoustic analysis processing means for linear prediction analysis line stomach, calculates a characteristic amount with respect to said digital signal,
A sound identification processing means for performing a comparison between the feature amount obtained by the acoustic analysis processing means and an acoustic model of the speaker and surrounding noise, and identifying whether or not the sound is designated;
Have
The output of the conference information of the other party is an output of sound by a speaker,
The conference system according to claim 1, wherein the restriction minimizes the output volume of the speaker before the restriction. 2. The conference system according to claim 1, further comprising: a state management unit that transmits a message indicating that the restriction has been performed to the other terminal device when the restriction is performed by the restriction unit. . In the conference system that outputs the conference information of the other party transmitted from the other party's terminal device in its own terminal device, and transmits its own conference information to the other party's terminal device.
Identification that identifies whether the sound in the vicinity of the terminal device is the sound that is designated as the sound to which the conference information of the other party should be output by collating with the acoustic model of the speaker and the surrounding noise Means,
When the identification means identifies that the other party's conference information is not a sound designated as a sound to be output, the output of the other party's conference information is restricted and the restriction information is released. Restriction means for releasing the restriction when it is determined that the release should be performed based on the instruction information input by the input means for performing input;
Have
The identification means includes
Conversion means for converting sound in the vicinity of the terminal device to a digital signal;
And acoustic analysis processing means for linear prediction analysis line stomach, calculates a characteristic amount with respect to said digital signal,
A sound identification processing means for performing a comparison between the feature amount obtained by the acoustic analysis processing means and an acoustic model of the speaker and surrounding noise, and identifying whether or not the sound is designated;
Have
The output of the conference information of the other party is an output of sound by a speaker,
The limit is a shall be minimized output volume of the speaker of the before limitation time,
Conference system during release of the restriction, characterized in that to return the volume level of the speaker output volume of the pre-limitation at point. When the restriction by the restriction means is performed, a message indicating that the restriction has been performed is transmitted to the other terminal device, and when the release is performed, a message indicating that the release has been performed. The conference system according to claim 3, further comprising state management means for transmitting to the other party's terminal device. The conference system according to claim 2 or 4,
When the state management unit receives a message indicating that the restriction has been performed in the counterpart terminal device from the counterpart terminal device, the state management means stores information indicating that the restriction has been performed in the counterpart terminal device. A conference system characterized by outputting in a terminal device. The conference system according to claim 4,
The state management means, when receiving a message indicating that the restriction has been lifted in the partner terminal device from the partner terminal device, the output indicating that the restriction has been performed in the partner terminal device. A conference system characterized in that the system is terminated. The counterpart terminal device is a telecommuter terminal device,
The conference system according to claim 1, wherein the conference information is conference information of the telecommuter. In the control method in the conference system that outputs the conference information of the other party transmitted from the other party's terminal device in its own terminal device, and transmits its own conference information to the other party's terminal device.
The terminal device
Identification that identifies whether the sound in the vicinity of the terminal device is the sound that is designated as the sound to which the conference information of the other party should be output by collating with the acoustic model of the speaker and the surrounding noise And
When it is identified that the other party's meeting information is not a sound designated as a sound to be output, the other party's meeting information is limited in output,
When identifying whether or not it is the specified sound,
Conversion processing for converting sound in the vicinity of the terminal device to a digital signal;
And acoustic analysis processing linear prediction analysis line stomach, calculates a characteristic amount with respect to said digital signal,
Performing a sound identification process for comparing the feature amount obtained by the acoustic analysis process with the acoustic model of the speaker and surrounding noise, and identifying whether or not the sound is designated,
The output of the conference information of the other party is an output of sound by a speaker,
The control method according to claim 1, wherein the restriction minimizes the output volume of the speaker before the restriction. The terminal device
9. The control method according to claim 8, wherein, when the restriction is performed, a message indicating that the restriction is performed is transmitted to a partner terminal device. In the control method in the conference system that outputs the conference information of the other party transmitted from the other party's terminal device in its own terminal device, and transmits its own conference information to the other party's terminal device.
The terminal device
The acoustic analysis is performed by comparing with the acoustic model of the speaker and the surrounding noise, and it is identified whether the sound in the vicinity of the terminal device is the sound designated as the sound that should output the conference information of the other party,
Input when said identified as not a sound is specified as the sound to be output meeting information of the other party performs restriction on the output of the other party of the conference information, to input the instruction information to release the said restriction When it is determined that the release should be performed based on the instruction information input by the means, the restriction is released,
When identifying whether or not it is the specified sound,
Conversion processing for converting sound in the vicinity of the terminal device to a digital signal;
And acoustic analysis processing linear prediction analysis line stomach, calculates a characteristic amount with respect to said digital signal,
Performing a sound identification process that performs a comparison between the feature amount obtained by the acoustic analysis process and an acoustic model of the speaker and the surrounding noise, and performs identification of whether or not the specified sound,
The output of the conference information of the other party is an output of sound by a speaker,
Control method characterized by returning the volume level of the speaker at the time of cancellation of the restriction is shall be minimized output volume of the speaker of the before limitation time the limit to the output volume of the pre-limitation at point. The terminal device
When the restriction is performed, a message indicating that the restriction is performed is transmitted to the counterpart terminal apparatus, and when the cancellation is performed, a message indicating that the cancellation is performed is transmitted to the counterpart terminal apparatus. The control method according to claim 10, further comprising: The control method according to claim 9 or 11,
The terminal device
When a message indicating that the restriction on the output of conference information has been performed at the other party's terminal device is received from the other party's terminal device, the information indicating that the restriction has been performed at the other party's terminal device The control method characterized by outputting in the terminal device. The control method according to claim 11, wherein
The terminal device
When the other terminal device receives a message indicating that the restriction has been released from the other terminal device, the output indicating that the restriction has been performed at the other terminal device is terminated. A control method characterized by that. The counterpart terminal device is a telecommuter terminal device,
The control method according to claim 8, wherein the conference information is conference information of the telecommuter.

Images