JP2017092576A - Communication device, control method, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately control a direction of a camera.SOLUTION: A communication device comprises a detection part, a directivity control part and an imaging control part. The detection part detects a generation source of a sound different from utterance at a position where the communication device is installed. The directivity control part performs control for reducing a sensitivity of a microphone that is directed towards the detected generation source, among a plurality of microphones mounted in the communication device. The imaging control part controls a direction of imaging by the camera in accordance with the control upon the microphone.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a communication device, a control method, and a control program.

  2. Description of the Related Art Conventionally, a communication system for a video conference that realizes a conversation between users at each site using a network such as the Internet is known. In a communication system for video conferencing, video and sound collected by cameras and microphones installed at each site are transmitted and received between communication devices at each site connected to the network, and display devices at other sites are installed. Realize video conferencing between sites by outputting from speakers. For example, projectors and liquid crystal panels are used as display devices. In such a video conference, for the purpose of realizing a smoother conversation, a device equipped with a speaker tracking function that detects sound with a microphone array function etc. and controls the direction of the camera in the direction of the sound source of the detected sound May be used.

  However, the conventional technique has a problem that the direction of the camera may be controlled in an unnecessary direction. Specifically, in the prior art, the camera direction is controlled in the direction of the sound detected by the microphone array function or the like, but the detected sound is not necessarily the voice of the speaker in the conference. Therefore, there is a possibility that the direction of the camera is controlled in a direction that is not a speaker. For example, when a projector is used as a display device, there is a possibility that the direction of the camera is controlled in the direction of the projector by detecting the sound generated by the rotation of a fan mounted on the projector. In general, a sound other than the voice of a speaker is subjected to processing for removal by an acoustic processing function such as a noise reduction function or a noise cancellation function. However, when applying the acoustic processing function, it is difficult to avoid the deterioration of the sound quality of the speaker's voice.

  The present invention has been made in view of the above, and an object thereof is to suitably control the orientation of a camera.

  In order to solve the above-described problems and achieve the object, the communication device according to the present invention is a communication device, and a detection unit that detects a sound source different from speech at a base where the communication device is installed A directivity control unit that performs control to reduce the sensitivity of the microphone directed toward the detected generation source among the plurality of microphones mounted on the communication device, and a camera according to the control on the microphone A shooting control unit that controls the shooting direction.

  According to one aspect of the present invention, there is an effect that the direction of the camera can be suitably controlled.

FIG. 1 is a diagram illustrating a system configuration example of a communication system according to the first embodiment. FIG. 2 is a block diagram illustrating a hardware configuration example of the communication apparatus according to the first embodiment. FIG. 3 is a block diagram illustrating a functional configuration example of the communication apparatus according to the first embodiment. FIG. 4 is a flowchart illustrating an example of a flow of control processing by the communication apparatus according to the first embodiment.

  Embodiments of a communication device, a control method, and a control program according to the present invention will be described below with reference to the accompanying drawings. Hereinafter, as an example of a communication system to which the communication device according to the present invention is applied, a video conference system that enables a conference between geographically distant bases will be exemplified. However, the present invention can be widely applied to various communication systems that transmit and receive video and sound between a plurality of communication devices, and various communication devices used in the communication systems. It is not limited by.

(Embodiment 1)
[System Configuration According to Embodiment 1]
The configuration of the communication system according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating a system configuration example of a communication system according to the first embodiment.

  As shown in FIG. 1, the communication system 1 includes a communication device 100 installed at a plurality of bases and a relay device 200. The communication device 100 and the relay device 200 installed at a plurality of bases are connected to a network 2 such as the Internet or a LAN (Local Area Network) and can communicate with each other. Among these, the number of communication devices 100 included in the communication system 1 corresponds to the number of bases participating in a video conference or the like. In FIG. 1, the case where the communication apparatus 100 is installed in each of the base A, the base B, and the base C is taken as an example.

  The communication device 100 transmits and receives various types of information to and from the communication device 100 at another site via the relay device 200. Further, the communication device 100 controls the output of the received information. For example, the output information is video of each base taken by a camera, sound of each base collected by a microphone (mainly the voice of a speaker), and the like. The communication device 100 may be a dedicated terminal for a video conference, or may be a general-purpose terminal such as a PC (Personal Computer), a smartphone, or a tablet terminal. The general-purpose terminal implements each function of the communication apparatus 100 as one of the applications by installing the control program according to the present embodiment. The relay device 200 is a server device or the like that relays transmission of various types of information such as video and sound between a plurality of communication devices 100 installed at each base.

  In the configuration described above, the communication device 100 detects a sound source that is different from the speaker's utterance at the site where the communication device 100 is installed. For example, the sound different from the speaker's utterance is a sound generated by the rotation of a fan mounted on a projector or the like installed at the base. It is not preferable to transmit the sound generated by the rotation of the fan to the communication device 100 at another site. In addition, when applying a speaker tracking function that detects sound with a microphone array function, etc., and controls the camera direction in the direction of the detected sound source, the camera direction is in the direction of the projector that is generating sound by the rotation of the fan. It is not preferable to control this. From these things, the communication apparatus 100 has detected the generation sources of sound other than speech.

  Then, communication device 100 performs control to reduce the sensitivity of the microphone directed toward the detected generation source among the plurality of microphones mounted on communication device 100. For example, the communication apparatus 100 is equipped with a plurality of microphones, and a microphone array function is realized by the plurality of microphones. The communication device 100 having the microphone array function reduces the sensitivity of the microphone directed toward the generation source such as a projector that emits sound by the rotation of the fan. Thereby, in the collection of sound by controlling the directivity of the microphone array function, it is possible to suppress the collection of sounds other than speech.

  Subsequently, the communication apparatus 100 controls the direction of shooting by the camera according to the control with respect to the microphone. For example, the direction of shooting by the camera is controlled by the speaker tracking function according to the directivity control of the microphone array function that reduces the sensitivity of the microphone directed toward the projector, which is the source of sound other than speech. To do. Thereby, since the sensitivity of the microphone directed to the projector or the like is lowered, it is possible to suppress the direction of photographing by the camera from being controlled to the direction of the projector or the like. Thereafter, the communication device 100 transmits output information including video acquired by camera shooting and sound collected by control of the microphone to the communication device 100 installed at another base.

  That is, the communication device 100 detects a sound generation source different from an utterance in a video conference or the like, and according to directivity control of the microphone array function that reduces the sensitivity of the microphone directed to the detected generation source, Since the orientation of the camera is controlled, the orientation of the camera can be suitably controlled. In other words, the communication device 100 can suppress the control of the direction of shooting by the camera to the direction of the sound source other than the utterance by suppressing the collection of sound other than the utterance in the conference or the like.

[Hardware Configuration of Communication Device According to Embodiment 1]
Next, the hardware configuration of communication apparatus 100 according to Embodiment 1 will be described using FIG. FIG. 2 is a block diagram illustrating a hardware configuration example of the communication apparatus 100 according to the first embodiment.

  As shown in FIG. 2, the communication device 100 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an SSD (Solid State Drive) 15, and a media drive. 17, an operation button 18, and a power switch 19. In addition, the communication device 100 includes a network I / F 21, a camera 22, an image sensor I / F 23, a microphone array 24, a speaker 25, a voice input / output I / F 26, a display I / F 27, and an external device. Connection I / F28.

  The CPU 11 controls the overall operation of the communication device 100. The CPU 11 controls the overall operation of the communication apparatus 100 by executing a program stored in the ROM 12 or the like using the RAM 13 or the like as a work area. The ROM 12 stores a program for realizing processing by the communication device 100. The RAM 13 is a work area when executing a program stored in the ROM 12 or the like. The SSD 15 controls reading and writing of data with respect to the flash memory 14 capable of storing various programs and various data. The media drive 17 controls reading and writing (recording) of data with respect to the recording medium 16.

  The operation button 18 is operated to select the communication device 100 at another site that is the communication destination of the communication device 100 and other various settings. The power switch 19 is for switching between turning on and off the power of the communication apparatus 100. The network I / F 21 is an interface for controlling connection to the network 2 and transmission / reception of various information. The camera 22 photographs the inside of the base. The image sensor I / F 23 is an interface for controlling driving of the camera 22 in accordance with control by the CPU 11. For example, the camera 22 is configured to be able to switch the shooting direction following the direction of sound detected by the microphone array function. As one aspect, the camera 22 may be an omnidirectional camera, but is not limited thereto. Note that the switching of the shooting direction may be either digital or analog.

  The microphone array 24 collects sounds in the base by a microphone array function including a plurality of microphones, and inputs the collected sounds to the communication device 100. The speaker 25 outputs sound. The voice input / output I / F 26 controls the directivity with respect to the microphone array 24 according to control by the CPU 11 to process input of signals (mainly voice signals), and controls the volume of the speaker 25 and the like to output signals. It is an interface for processing. The display I / F 27 is an interface for transmitting video data to be displayed on the display device 50 under the control of the CPU 11. For example, the display device 50 is a projector or a liquid crystal panel that is externally attached to the communication device 100. The external device connection I / F 28 is an interface for connecting various external devices to the communication device 100. The communication device 100 also includes a bus 20 such as an address bus or a data bus for electrically connecting the above-described units. Note that the hardware configuration shown in FIG. 2 is an example, and hardware other than the above may be added.

[Functional Configuration of Communication Device According to Embodiment 1]
Next, the functional configuration of the communication apparatus 100 according to Embodiment 1 will be described with reference to FIG. FIG. 3 is a block diagram illustrating a functional configuration example of the communication apparatus 100 according to the first embodiment.

  As illustrated in FIG. 3, the communication device 100 includes an operation input receiving unit 101, a display control unit 102, an audio output control unit 103, a detection unit 104, a determination unit 105, a directivity control unit 106, and an imaging. A control unit 107, a voice input control unit 108, and a transmission / reception control unit 109 are included. Each of the above units may be realized by software (program) or a hardware circuit. Moreover, each said part is a function implement | achieved when CPU11 runs the control program expand | deployed on RAM13 from the flash memory 14 grade | etc., For example.

  The operation input receiving unit 101 receives various operation inputs by a user who uses the communication device 100. For example, the operation input accepting unit 101 accepts input of information regarding various settings and information for power control in response to a user operation on the operation button 18, the power switch 19, and the like.

  The display control unit 102 controls display processing for the display device 50. For example, the display control unit 102 performs drawing processing or the like on the video at the other site received from the communication device 100 at the other site, and outputs the processed data to the display device 50. As a result, the display device 50 displays and outputs an image including a video at another site.

  The sound output control unit 103 controls sound output processing for the speaker 25. For example, the audio output control unit 103 decodes sound data at another site received from the communication device 100 at another site, and outputs the decoded data (mainly audio data) to the speaker 25. As a result, the speaker 25 reproduces and outputs the audio data at the other site.

  The detection unit 104 detects a sound generation source different from the utterance. More specifically, when a predetermined image is included in the video captured by the camera 22, the detection unit 104 has a sound source different from the speaker's utterance in the direction including the predetermined image. Detect what to do. For example, the sound source different from the utterance refers to the display device 50 such as a projector. For example, the predetermined image refers to an image including predetermined characters, numbers, symbols, and the like. The communication device 100 holds these characters, numbers, symbols, and the like in advance. That is, the detection unit 104 holds a manufacturer's logo type, product model number, and the like attached to the display device 50 such as a projector for detection of a predetermined image, and the video captured by the camera 22 includes the predetermined image. If so, the presence of the display device 50 in that direction is detected.

  A projector or the like is equipped with a fan, and the sound generated by the rotation of the fan is unnecessary in a meeting. The detecting unit 104 detects the presence or absence of a projector or the like that is a source that generates such unnecessary sound, and the direction in which it exists. The predetermined image is not limited to the manufacturer's logo type or product model number. Specifically, the predetermined image may be anything as long as a sound source different from the utterance can be detected. For example, when a two-dimensional code or the like is employed for a predetermined image, the detection unit 104 generates a source in the direction of the video including the two-dimensional code if a character read from the two-dimensional code matches a character stored in advance. Detect that exists. Note that characters, symbols, and the like to be held in the communication apparatus 100 may be specified in advance by the user, or may be arbitrarily added / changed.

  The determination unit 105 determines whether or not the size of the predetermined image is greater than or equal to a predetermined threshold value. More specifically, when the detection unit 104 detects a projector or the like that is a sound source different from the utterance, the determination unit 105 determines that the size of a predetermined image such as a character, number, or symbol is a predetermined threshold value. It is determined whether it is above. Then, when the size of the predetermined image is equal to or greater than the predetermined threshold, the determination unit 105 notifies the directivity control unit 106 that the size of the predetermined image is equal to or greater than the predetermined threshold. On the other hand, when the determination unit 105 determines that the size of the predetermined image is less than the predetermined threshold, the communication device 100 ends the process. The threshold for determining the size of characters, numbers, symbols, etc. may be held in the communication device 100 in advance, or may be specified in advance by the user, or may be arbitrarily changed. Also good.

  In other words, the determination unit 105 determines whether the size of characters or numbers pasted on the projector or the like is equal to or greater than a predetermined threshold, and thus the positional relationship between the projector and the communication device 100 is close. It is determined whether or not it is installed. For example, if the size of letters, numbers, and the like is equal to or greater than a predetermined threshold, they are installed at positions close to each other, so that the speaker tracking function may be affected by the rotation of a fan mounted on a projector or the like. On the other hand, if the size of letters, numbers, etc. is less than the predetermined threshold, it is assumed that the distance from the communication device 100 to the projector etc. is sufficiently far away, and the sound due to the rotation of the fan mounted on the projector etc. is unnecessary for the conference It shall not be a sound.

  The directivity control unit 106 performs control to reduce the sensitivity of the microphones directed to the direction of the sound generation source different from the utterance among the plurality of microphones of the microphone array 24. More specifically, when the determining unit 105 determines that the size of the predetermined image is equal to or greater than a predetermined threshold, the directivity control unit 106 moves in the direction of a projector or the like that is a sound generation source different from the utterance. It is determined whether or not the volume of sound collected by the directed microphone is equal to or greater than a predetermined threshold. Then, the directivity control unit 106 has a microphone array function so as to reduce the sensitivity of the microphones directed toward the projector or the like among the plurality of microphones of the microphone array 24 when the volume is equal to or higher than a predetermined threshold. Control directivity. The directivity control unit 106 ends the process when the volume of the sound collected by the microphone directed in the direction of the projector or the like is less than a predetermined threshold. Note that the threshold for determining the volume may be stored in advance in the communication apparatus 100, may be specified in advance by the user, or may be arbitrarily changed.

  The shooting control unit 107 controls the direction of shooting by the camera 22 in accordance with control of the microphone array 24. More specifically, the shooting control unit 107 restricts shooting in the direction of the microphone with reduced sensitivity according to the directivity of the microphone array function controlled by the directivity control unit 106, and Control the orientation. For example, it may be set to prohibit shooting in the direction of the microphone with reduced sensitivity, and when the speaker tracking function is applied, shooting in the direction of the microphone with reduced sensitivity is suppressed as a result. . As a result, the sound collection by the microphone array 24 and the photographing by the camera 22 are restricted with respect to the direction of the display device 50 that can be a sound generation source different from the speech. The shooting control unit 107 outputs the video shot by the camera 22 to the transmission / reception control unit 109.

  The sound input control unit 108 controls sound input processing from the microphone array 24. For example, the voice input control unit 108 encodes the sound in the base collected by the microphone array 24 with an arbitrary encoding format such as PCM (Pulse Code Modulation) and the encoded data (mainly the voice data). ) To the transmission / reception control unit 109.

  The transmission / reception control unit 109 controls transmission / reception of various kinds of information to / from the communication device 100 at another site via the relay device 200 via the network 2. The transmission / reception control unit 109 corresponds to a “transmission control unit” as one aspect. For example, the transmission / reception control unit 109 transmits output information including the video output by the imaging control unit 107 and the audio data output by the audio input control unit 108 to the communication device 100 installed at another base. To do. Also, the transmission / reception control unit 109 receives output information from the communication device 100 installed at another base. The transmission / reception control unit 109 outputs data related to video included in the received output information to the display control unit 102 and outputs data related to audio to the audio output control unit 103. As a result, the video and audio of another base are output under the control of the display control unit 102 and the audio output control unit 103. In addition, the communication apparatus 100 repeatedly executes the processes by the detection unit 104, the determination unit 105, the directivity control unit 106, and the imaging control unit 107 during a certain period of time.

[Control processing flow according to Embodiment 1]
Next, the flow of control processing by the communication apparatus 100 according to Embodiment 1 will be described with reference to FIG. FIG. 4 is a flowchart illustrating an example of a flow of control processing by the communication device 100 according to the first embodiment.

  As illustrated in FIG. 4, when the communication device 100 receives an operation for starting control (step S101: Yes), the communication device 100 inputs an image captured by the camera 22 (step S102). On the other hand, when the communication device 100 has not received an operation for starting control (step S101: No), the communication device 100 is in a state of waiting for the operation. Then, the communication device 100 determines whether or not a predetermined character is included in the captured video (step S103). At this time, when a predetermined character is included in the captured video (step S103: Yes), the communication device 100 has a sound source different from the utterance in the direction of the video including the predetermined character. It is determined that the character size is equal to or greater than a predetermined threshold value (step S104). On the other hand, the communication apparatus 100 performs the process of step S108 when the predetermined | prescribed character is not contained in the image | photographed image | video (step S103: No).

  Subsequently, when the size of the character is equal to or greater than the predetermined threshold (step S104: Yes), the communication device 100 inputs the sound collected by the microphone directed in the direction of the sound source different from the utterance. (Step S105). On the other hand, when the character size is less than the predetermined threshold (step S104: No), the communication device 100 executes the process of step S108. Thereafter, the communication apparatus 100 determines whether or not the volume of the sound from the sound source different from the utterance is equal to or higher than a predetermined threshold (step S106). At this time, when the volume is equal to or higher than the predetermined threshold (step S106: Yes), the communication apparatus 100 reduces the sensitivity of the microphone directed to the direction of the sound source different from the utterance, and the direction by the camera 22 Is restricted (step S107). On the other hand, when the volume is less than the predetermined threshold (step S106: No), the communication device 100 executes the process of step S108.

  The communication device 100 determines whether an operation for ending control is accepted (step S108). At this time, when the operation for ending the control is not received (step S108: No), the communication device 100 determines whether or not a predetermined time has elapsed since the control of the microphone and the camera 22 (step S109). On the other hand, the communication apparatus 100 complete | finishes a process, when operation for control completion | finish is received (step S108: Yes). In addition, when a predetermined time has elapsed from the control of the microphone and the camera 22 (step S109: Yes), the communication device 100 executes the process of step S102 again. On the other hand, when the predetermined time has not elapsed since the control of the microphone and the camera 22 (step S109: No), the communication device 100 executes the process of step S108.

[Effects of Embodiment 1]
As described above, the communication apparatus 100 detects the sound source different from the utterance, and controls the directivity of the microphone array function for reducing the sensitivity of the microphone directed toward the detected source. Since the direction of shooting by the camera 22 is controlled according to the directivity, the direction of the camera 22 can be suitably controlled.

  Further, the communication apparatus 100 assumes that when a video obtained by the camera 22 includes a predetermined character or the like, a sound source different from the utterance exists in a direction including the predetermined character. When the size of the predetermined character is detected and equal to or larger than a predetermined threshold value, the directivity of the microphone array function for reducing the sensitivity of the microphone directed toward the detected generation source is controlled. As a result, when the communication device 100 is close to the generation source, the communication device 100 reduces the sensitivity of the microphone directed toward the generation source, so that unnecessary sound is mixed in the conference from the generation source. This can be suppressed.

  Further, the communication device 100 increases the sensitivity of the microphone directed in the direction of the generation source when the volume of the sound collected by the microphone directed in the direction of the sound generation source different from the speech is equal to or greater than a predetermined threshold. Controls the directivity of the microphone array function to be lowered. As a result, the communication apparatus 100 can suppress the mixing of unnecessary sound in the conference from the generation source with higher accuracy.

(Embodiment 2)
Although the embodiments of the communication device 100 according to the present invention have been described so far, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, different embodiments of (1) configuration and (2) program will be described.

(1) Configuration Information including processing procedures, control procedures, specific names, various data, parameters, and the like shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. Each component of the illustrated apparatus is functionally conceptual and does not necessarily need to be physically configured as illustrated. That is, the specific form of the distribution or integration of the devices is not limited to the illustrated one, and all or a part of the distribution or integration is functionally or physically distributed or arbitrarily in any unit according to various burdens or usage conditions. Can be integrated. For example, the camera 22 does not have to be mounted on the communication device 100 and may be connected to the communication device 100 as an external device.

  Further, in the above embodiment, whether or not there is a display device 50 such as a projector that can be a sound source different from the utterance in the vicinity of the communication device 100 such as letters, numbers, symbols, etc. Although it is determined based on the size, any means may be used as long as it can recognize whether or not the display device 50 such as a projector exists in the vicinity of the communication device 100. For example, it may be determined whether or not the display device 50 such as a projector is present in the vicinity of the communication device 100 based on the width and interval between characters, numbers, and symbols included in the video.

(2) Program The control program executed by the communication apparatus 100 is, as one form, a file in an installable format or an executable format as a CD-ROM, flexible disk (FD), CD-R, DVD ( And recorded on a computer-readable recording medium such as a digital versatile disk. The control program executed by the communication apparatus 100 may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The control program executed by the communication apparatus 100 may be provided or distributed via a network such as the Internet. The control program executed by the communication device 100 may be provided by being incorporated in advance in a ROM or the like.

  The control program executed by the communication device 100 has a module configuration including the above-described units (the detection unit 104, the determination unit 105, the directivity control unit 106, the imaging control unit 107, and the transmission / reception control unit 109). As the hardware, the CPU reads the program from the storage medium and executes it, so that the above-described units are loaded on the main storage device, and the detection unit 104, the determination unit 105, the directivity control unit 106, the imaging control unit 107, the transmission / reception control. The unit 109 is generated on the main storage device.

DESCRIPTION OF SYMBOLS 1 Communication system 100 Communication apparatus 101 Operation input reception part 102 Display control part 103 Audio | voice output control part 104 Detection part 105 Judgment part 106 Directivity control part 107 Shooting control part 108 Voice input control part 109 Transmission / reception control part 200 Relay apparatus

Japanese Patent No. 5028944 Japanese Patent No. 3140866 Japanese Patent No. 5776313

Claims (8)

A communication device,
At a base where the communication device is installed, a detection unit that detects a sound generation source different from speech,
A directivity control unit that performs control to reduce the sensitivity of the microphone directed toward the detected generation source among the plurality of microphones mounted on the communication device;
And a shooting control unit that controls a shooting direction of the camera according to control of the microphone.   The said detection part detects that the said generation source exists in the direction in which the said predetermined image is included, when the predetermined image is contained in the image | video acquired by imaging | photography of the said camera. The communication apparatus according to 1. A determination unit for determining whether the size of the predetermined image is equal to or greater than a predetermined threshold;
The directivity control unit performs control to reduce sensitivity of a microphone directed toward the generation source when it is determined that the size of the predetermined image is equal to or greater than a predetermined threshold. Item 3. The communication device according to Item 2.   The directivity control unit performs control to reduce the sensitivity of the microphone directed toward the source when the volume of sound collected by the microphone directed toward the source is equal to or greater than a predetermined threshold. The communication device according to claim 1, wherein the communication device is performed.   The communication apparatus according to claim 1, wherein the photographing control unit restricts photographing in the direction of the microphone with reduced sensitivity.   It further includes a transmission control unit that controls transmission of output information including video acquired by photographing with the camera and sound collected by controlling the microphone to a communication device installed at another base. The communication device according to any one of claims 1 to 5. A control method in a communication device,
Detecting a sound source different from the utterance at the base where the communication device is installed;
Performing a control to reduce the sensitivity of the microphone directed toward the detected source among the plurality of microphones mounted on the communication device;
And a step of controlling a direction of photographing by the camera in accordance with control of the microphone. In communication equipment,
Detecting a sound source different from the utterance at the base where the communication device is installed;
Performing a control to reduce the sensitivity of the microphone directed toward the detected source among the plurality of microphones mounted on the communication device;
A control program for executing the step of controlling the direction of shooting by the camera in accordance with the control of the microphone.

Images