JP2009060220A - Communication system and communication program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform smooth communication in which it is easy to hear voice of persons having a dialog between different points. <P>SOLUTION: Positions of the persons performing communication between the different points X and Y are determined, and on the basis of the determination results, at least either an operation for changing sensitivity of microphones 104C, 204A to the voice in the positions, or an operation for increasing the magnitude of voice outputted from speakers 105C, 205A to the positions is performed. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a communication system and a communication program that enable communication between different points via a network.

A communication system that enables communication between different points via a network is progressing day by day due to an increase in communication line capacity represented by broadband, an increase in performance of computers in the system, and the like. Some communication systems communicate high-quality color images bidirectionally in real time with audio. For example, in a video conference system in the business field, there is one that can talk with a plurality of life-size displayed persons with realism. is there. As a video conference system capable of bidirectional communication, there is a technique described in Patent Document 1. In the video conference system described in Patent Document 1, participants who gather at each conference place can interact through a display or a microphone.
JP-T-2001-517395

  By the way, in a video conference system, since a plurality of participants have a conversation on a common agenda, the voices of other participants do not become obstacles to communication.

  However, in a communication system in which remote offices are always connected using a large screen display or the like, a plurality of persons in the office may have a conversation with different persons at the same time. At this time, the voice emitted from a person different from the conversation partner becomes an obstacle, and the voice of the conversation partner may not be heard or difficult to hear, and smooth communication cannot be achieved.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a communication system and a communication program that can easily hear voices of persons who interact with each other at different points.

In order to achieve the above object, a communication system according to the present invention provides:
A communication system that enables communication between different points via a network,
A camera to shoot the speaker,
A display for displaying images taken by the camera;
A microphone that converts the voice of the speaker into a voice signal;
A speaker for outputting the audio signal converted by the microphone to the outside;
A control unit for controlling the operation in the communication system,
The control unit determines the position of a conversation person who is communicating between different points, and based on the determination result, the operation of changing the sensitivity of the microphone with respect to the sound at the position, or the position with respect to the position It is characterized in that at least one of the operations for increasing the sound output from the speaker is executed.

In addition, the communication program according to the present invention includes:
A camera to shoot the speaker,
A display for displaying images taken by the camera;
A microphone that converts the voice of the speaker into a voice signal;
A speaker for outputting the audio signal converted by the microphone to the outside;
Is a communication program that enables communication between different points using a computer.
A determination step of determining the location of the interlocutor communicating between the different points;
Based on the determination result determined in the determination step, at least one of an operation of changing the sensitivity of the microphone with respect to the sound at the position or an operation of increasing the sound output from the speaker with respect to the position. An operational process to perform;
Is executed by a computer.

  According to the communication system and the communication program according to the present invention, it is easy to hear the voices of those who have a conversation between different points, and smooth communication can be achieved.

  FIG. 1 is a schematic diagram of a communication system according to the present invention.

  The room X and the room Y at different points are connected via the network 3 so that two-way communication is possible.

  A display 102 is installed in the living room X, and a display 202 is installed in the living room Y. The image of the room Y is displayed on the display 102, and the image of the room X is displayed on the display 202. For example, the person “A” who is enrolled in the room X can interact with the person “B” who is enrolled in the room Y using the communication system shown in FIG.

  FIG. 2 is a block diagram of a communication system according to the present invention, showing a typical control configuration.

  A dialogue system 1 is installed in the living room X, and a dialogue system 2 is installed in the living room Y. The dialogue system 1 and the dialogue system 2 are connected via a network 3 to constitute a communication system as a whole. Since the dialogue system 1 and the dialogue system 2 have the same configuration, each configuration will be described based on the dialogue system 1.

  The dialogue system 1 includes a PC 101, a display 102, a camera 103, a microphone 104, and a speaker 105. The PC (computer) 101 transmits a signal to the dialog system 2 and receives a signal from the dialog system 2. The display 102, the speaker 103, and the like are connected to the PC 101, and the PC 101 controls operations of the display 102, the speaker 103, and the like according to a predetermined program.

  A CPU (Central Processing Unit) 101A controls the operation of the entire PC (computer) 101, and is connected to a ROM (Read Only Memory) 101B, a RAM (Random Access Memory) 101C, and the like. The CPU 101A reads various control programs stored in the ROM 101B, develops them in the RAM 101C, and controls the operation of each unit. Further, the CPU 101A executes various processes in accordance with the program expanded in the RAM 101C, and stores the processing results in the RAM 101C. Then, the processing result stored in the RAM 101C is stored in a predetermined storage destination. In this embodiment, the CPU 101A constitutes a control unit by cooperating with the ROM 101B and the RAM 101C.

  The ROM 101B stores programs, data, and the like in advance, and this recording medium is composed of a magnetic or optical recording medium or a semiconductor memory.

  The RAM 101C forms a work area for temporarily storing data processed by various control programs executed by the CPU 101A.

  An HDD (Hard Disk Drive) 101D has a function of storing predetermined data. It has a structure in which a number of metal disks coated or vapor-deposited with a magnetic material are stacked at regular intervals, and this is rotated at high speed by a motor to read and write data by bringing the magnetic head closer. The communication program according to the present invention is stored in the HDD 101D (the communication program is also stored in the HDD of the PC 201).

  The display 102 displays a video of the room Y and a video of a person belonging to the room Y, and the video displayed is taken by the camera 203 in the dialogue system 2. The display 102 is preferably a large screen with high resolution in order to enhance the presence.

  The camera 103 captures a room X or a person belonging to the room X and distributes the captured video to the dialogue system 2 via the network 3.

  The microphone 104 collects sound generated in the room X, and the sound collected by the microphone 104 is distributed to the speaker 205 in the dialogue system 2 via the network 3. The microphone 104 requires at least two channels, and is preferably a stereo microphone in order to enhance the sense of reality.

  The speaker 105 provides the sound emitted from the room Y in the room X, and requires at least two channels. The speaker 105 may be built in the display 102 or may be separate from the display 102.

  Next, a description will be given of an operation for making communication by using the communication system described with reference to FIG. 2 to determine the position of the conversation person and adjusting the sound output from the speaker. For example, assuming that the person “A” who is enrolled in the room X and the person “B” who is enrolled in the room Y are interacting in a positional relationship with respect to the display as shown in FIG. explain.

  As shown in FIG. 3, the camera 103 in the living room X is installed at the center and above the display 102, and the camera 203 in the living room Y is installed at the center and above the display 202. The display 102 is divided into three regions α, β, and γ, and a microphone and a speaker are installed in each region (for example, a microphone 104A and a speaker 105A are installed in the α region on the display). ) Similarly to the display 102, the display 202 in the living room Y is also divided into three regions α, β, and γ, and a microphone and a speaker are installed in each region. Note that the three areas of the display 102 and the display 202 are merely examples, and may be divided into a plurality of areas.

  The video shot by the camera 203 in the room Y is displayed on the display 102 in the room X, and the person B in the room Y is displayed in the γ region of the display 102. Accordingly, since it is easier to interact with the person B displayed on the display 102 in front of the person B, the person A interacting with the person B is located in front of the γ region of the display 102. Similarly, since it is easier for the person A shown on the display 202 to interact with the person A, the person B interacting with the person A is located in front of the α area of the display 202. .

  A method in which the person A enrolled in the room X in this state and the person B enrolled in the room Y interact smoothly without being disturbed by the voices of other people will be described with reference to FIG.

  FIG. 4 is a flowchart for explaining the operation of adjusting the voice by determining the position of the conversation person based on the direction of the speaker's face.

  First, the dialogue system 1 or the dialogue system 2 determines whether or not a human voice is detected by the microphone (step S1). Since it can be determined that the conversation has started by detecting the human voice with the microphone, the determination operation is first performed. Note that this determination operation is not limited to a method of detecting a human voice because it is only necessary to be able to determine that the dialog has started. For example, when a dialog is performed, a predetermined button in the dialog system 1 or the dialog system 2 is pressed by the dialog person. It may be determined whether or not it has been done.

  When a human voice is detected in step S1 (step S1; Yes), in order to adjust the volume of a speaker or the like, a person in front of the display is first identified (step S2). In the example of FIG. 3, since the person A is in front of the display 102 and the person B is in front of the display 202, the persons A and B are based on images taken by the camera 103 and the camera 203. Identified.

  Next, based on the specified face direction of the person, a dialogue destination area on the display is detected (step S3). The face direction is detected from the face of a person in the images taken by the cameras 103 and 203. A specific detection method is, for example, a method of calculating image width parameters of a person's left eye, right eye, mouth, and skin region and detecting the parameter by comparing with a pre-stored lookup table (for example, Japanese Patent Laid-Open No. 2000-97676). (Technique described in the publication). When the face direction is detected, it is detected which area on the display is being viewed. In the example shown in FIG. 3, in the living room X, the face of the person A faces the γ area on the display 102 (the area where the person B is projected). Considered the destination area. On the other hand, in the living room Y, the face of the person B faces the α area on the display 202 (the area where the person A is projected), so the α area on the display 202 is regarded as the conversation destination area.

  Then, in step S3, the dialogue destination area on the display 102 and the dialogue destination area on the display 202 are collated, and the position of the conversation person having a conversation between the room X and the room Y is determined (as a determination step). Step S4). This determination is performed in either the PC 101 in the room X or the PC 201 in the room Y.

  In the example shown in FIG. 3, in step S3, the γ region on the display 102 and the α region on the display 202 are detected as the dialogue destination region, and the video displayed on each display and the video are viewed. Since the face direction of the person matches, it can be determined that the dialogue between the room X and the room Y has been established. Similarly, when the β area of the display 102 and the β area of the display 202 are set as the dialogue destination area, the dialogue between the room X and the room Y is set when the α area of the display 102 and the γ area of the display 202 are set as the dialogue destination area. It can be determined that

  In FIG. 3, the interrogators are person A and person B, and the positions of the interrogators are the γ region (position where person A is present) on display 102, and for person B the α region (position where person B is present) on display 202. It is determined that

  When the position of the talker is determined in step S4, the voice sensitivity is changed to be high in the microphone in the area where the talker is present (step S5 as an operation process), and the conversation is performed in the speaker in the area where the talker is located. The other party's voice is greatly output (step S6 as an operation process).

  In the example shown in FIG. 3, the voice sensitivity of the microphone 104C in the γ region (position where the person A is present) in the living room X is changed so as to sufficiently pick up the voice of the person A, and the α region (person B in the living room Y). The sound sensitivity of the microphone 204A at the position where the person B is located is changed so that the sound of the person B is sufficiently picked up. The loudspeaker 105 C in the room X outputs a loud voice of the person B, and the loudspeaker 104 A in the alpha room Y in the living room Y outputs a loud voice of the person A. Note that at least one of the operations may be performed instead of changing the sound sensitivity of the microphone and increasing the output value of the speaker. Also, instead of installing a plurality of microphones and speakers as shown in FIG. 3, one microphone and one speaker are installed to change the sensitivity of the microphone to the voice of the talker position, The sound output from the speaker may be increased.

  As described above with reference to FIGS. 3 and 4, if the position of the conversation person is determined and the sound is adjusted by the microphone or the speaker based on the determination result, even if a person other than the conversation person speaks, It is easy to hear the voices of those who have a conversation without disturbing the voice, and it is possible to communicate smoothly.

  As another method, it is conceivable to determine the position of the conversation person based on the direction of the line of sight of the speaker. This point will be described with reference to FIGS. Unlike FIG. 3, it is assumed that the person A and person C who are enrolled in the room X and the person B who is enrolled in the room Y are having a two-to-one conversation with respect to the display as shown in FIG. Will be described in detail.

  Steps S11 to S12 and S14 to S16 in FIG. 6 are the same as steps S1 to S2 and S4 to S6 in FIG. In step S13 of FIG. 6, a dialog destination area on the display is detected based on the identified line of sight of the person. The direction of the line of sight of the person is detected from the movement of the eyes of the person in the images taken by the cameras 103 and 203 (for example, the technique described in Japanese Patent Application Laid-Open No. 2000-146553).

  In the example shown in FIG. 5, in the living room X, the lines of sight of the person A and the person C are directed to the γ area (area where the person B is projected) on the display 102. It is considered as a dialogue destination area. On the other hand, in the living room Y, the line of sight of the person B faces the α area (the area where the person A is projected) and the β area (the area where the person C is projected) on the display 202. The α region and the β region are regarded as dialogue destination regions.

  When the detection in step S13 is completed, the dialogue destination area on the display 102 and the dialogue destination area on the display 202 are collated, and the position of the conversation person who has a conversation between the room X and the room Y is determined (step). S14). Based on the result of the determination, the voice sensitivity of the microphone in the area where the talker is present is changed to be high (step S15), and the voice of the conversation partner is greatly output from the speaker in the area where the talker is present ( Step S16).

  In the example shown in FIG. 6, the voice sensitivities of the microphone 104B in the β region (position where the person C is present) and the microphone 104C in the γ region (position where the person A is present) in the room X are changed to change the person A and the person C. The sound is sufficiently picked up, and the sound sensitivity of the microphone 204A in the α region (position where the person B is present) in the room Y is changed so that the sound of the person B is sufficiently picked up. In addition, the sound of the person B is greatly output from the speaker 105B in the β region and the speaker 105C in the γ region in the living room X, and the sound of the person A is output greatly from the speaker 104A in the α region in the living room Y.

  In this way, if the position of the conversation person is determined based on the direction of the person's line of sight, and the sound is adjusted by a microphone or a speaker based on the determination result, the voice of the person who is not the conversation person is spoken. This makes it easier to hear the voices of those who are interacting with each other, and allows smooth communication.

  Further, as shown in step S23 in the flowchart of FIG. 7, the conversation destination area on the display may be detected based on the direction of the voice emitted from the speaker. A specific detection method is a method (for example, a technique described in Japanese Patent Application Laid-Open No. 9-251299) in which detection is performed by frequency analysis of sound input from a microphone. Note that steps S21 to S22 and S24 to S26 in FIG. 7 are the same as steps S1 to S2 and S4 to S6 in FIG.

  Further, as shown in the flowchart of FIG. 8, the dialogue area on the display may be determined based on the location selected by the speaker on the display.

  When a conversation is performed in the room X and the room Y, the user selects a partner to interact on the display. For example, on the display 102 or 202, a message “Please select a partner to interact with on the display” is displayed to prompt the user to select.

  In step S31 in FIG. 8, it is determined whether or not the other party to interact with on the display has been selected by the user. When there is a selection on the display, it is determined that the selected location of the display is an area where there is a conversation partner, and the dialogue destination area is determined based on the selected location (step S32). By grasping the coordinates of the selected location on the display, it is possible to determine which region is the selected location. When the position of the interlocutor is determined in step S32, it is changed to increase the voice sensitivity in the microphone in the area where the interlocutor is present in the same manner as the operation described in FIG. 4 or the like (step S34). Loudly output the voice of the conversation partner (step S35).

  Further, as shown in the flowchart of FIG. 9, the conversation partner's face information is extracted from the voice information related to the conversation partner's name input to the microphone, and the extracted face information is collated with the image taken by the camera. The position of the person may be determined.

  First, it is detected whether there is a call for the name of the conversation partner in the microphone (step S41). When there is a name call, the called name is recognized by speech (step S42), and face information is extracted from the name information that has been recognized (step S43).

  Name speech recognition recognizes input speech by analyzing the wavelength and the like of a speech signal input by a microphone. The face information is extracted by referring to a database in which the relationship between the name and the face information is defined.

  When the face information is extracted, an area where the corresponding person is present is specified based on the extracted face information (step S44). A person's face is extracted from the image photographed by the camera, and the position information of the eyes and mouth is compared with the face information extracted in step S43, and the region where the corresponding person exists is specified.

  And after specifying the area | region where the applicable person exists, it determines that the specified area | region is a position where a dialog person exists (step S45), and it changes so that audio | voice sensitivity may be made high in the microphone of the area | region where a dialog person exists ( In step S46), the voice of the conversation partner is greatly output from the speaker in the area where the conversation person is present (step S47).

  As described above with reference to FIGS. 6 to 9, not only the direction of the speaker's face, but also the direction of the speaker's line of sight, the direction of the sound emitted from the speaker, the selection location on the display selected by the speaker, etc. Based on the determination result, the microphone's voice sensitivity is changed or the speaker's volume is adjusted, so that the voices of other people interacting with each other without any obstacles. The voice becomes easier to hear.

  Although the embodiments of the present invention have been described with reference to the drawings, the present invention is not limited to the embodiments, and the present invention can be modified or added without departing from the scope of the present invention. included.

  3 and 5, for example, when there is a person who is difficult to determine as a conversation person, it is regarded as a conversation person, and the conversation person is changed to increase the voice sensitivity with the microphone, or the voice of the conversation partner is heard with the speaker. You may make it output large.

It is the schematic of a communication system. It is a block diagram of a communication system. It is explanatory drawing which shows the state in which the person A in the living room X and the person B in the living room Y are talking. It is a flowchart explaining the operation | movement which determines the position of a conversation person based on the direction of a speaker's face, and adjusts a sound. It is explanatory drawing which shows the state in which the person A and the person C in the living room X and the person B in the living room Y are talking. It is a flowchart explaining the operation | movement which determines the position of a conversation person based on the direction of a speaker's eyes | visual_axis, and adjusts a sound. It is a flowchart figure explaining the operation | movement which determines the position of a conversation person based on the direction of the audio | voice emitted from a speaker, and adjusts an audio | voice. It is a flowchart figure explaining the operation | movement which determines the position of a conversation person based on the selection location on the display which the speaker selected, and adjusts an audio | voice. It is a flowchart explaining the operation | movement which determines the position of a conversation person based on the audio | voice information and face information regarding the name of a conversation partner, and adjusts a sound.

Explanation of symbols

1, 2, Dialog system 101, 201 PC
102, 202 Display 103, 203 Camera 104, 204 Microphone 105, 205 Speaker 101A, 201A CPU
101B, 201B ROM
101C, 201C RAM
101D, 201D HDD

Claims (12)

A communication system that enables communication between different points via a network,
A camera to shoot the speaker,
A display for displaying images taken by the camera;
A microphone that converts the voice of the speaker into a voice signal;
A speaker for outputting the audio signal converted by the microphone to the outside;
A control unit for controlling the operation in the communication system,
The control unit determines the position of a conversation person who is communicating between different points, and based on the determination result, the operation of changing the sensitivity of the microphone with respect to the sound at the position, or the position with respect to the position A communication system that performs at least one of operations for increasing sound output from a speaker. The said control part detects the dialog destination area | region on the said display based on the direction of a speaker's face, and determines the said position by collating the said dialog destination area | region detected in each said point. 1. The communication system according to 1. The control unit detects the dialogue destination area on the display based on the direction of the line of sight of a speaker, and determines the position by collating the dialogue destination area detected at each of the points. 1. The communication system according to 1. The control unit detects a dialogue destination area on the display based on a direction of a voice emitted from a speaker, and determines the position by comparing the dialogue destination area detected at each of the points. The communication system according to claim 1. The communication system according to claim 1, wherein the control unit determines the position based on a selection location on the display selected by a speaker. The control unit extracts face information of the conversation partner from voice information related to the name of the conversation partner input to the microphone, and determines the position by comparing the extracted face information with an image photographed by the camera. The communication system according to claim 1. A camera to shoot the speaker,
A display for displaying images taken by the camera;
A microphone that converts the voice of the speaker into a voice signal;
A speaker for outputting the audio signal converted by the microphone to the outside;
Is a communication program that enables communication between different points using a computer.
A determination step of determining the location of the interlocutor communicating between the different points;
Based on the determination result determined in the determination step, at least one of an operation of changing the sensitivity of the microphone with respect to the sound at the position or an operation of increasing the sound output from the speaker with respect to the position. An operational process to perform;
A communication program characterized by causing a computer to execute. The determination step detects the dialog destination area on the display based on the direction of a speaker's face and determines the position by comparing the dialog destination area detected at each of the points. The communication program according to 7. The determination step detects the dialog destination area on the display based on the direction of the line of sight of a speaker, and determines the position by collating the dialog destination area detected at each of the points. The communication program according to 7. In the determination step, the dialogue destination area on the display is detected based on the direction of a voice emitted from a speaker, and the dialogue destination area detected at each point is collated to determine the position. The communication program according to claim 7. The communication program according to claim 7, wherein the determination step determines the position based on a selection location on the display selected by a speaker. In the determination step, face information of the conversation partner is extracted from voice information related to the name of the conversation partner input to the microphone, and the position is determined by collating the extracted face information with an image photographed by the camera. The communication program according to claim 7.

Images