JP4411959B2 - Audio collection / video imaging equipment - Google Patents

Description

The present invention relates to a sound collection / video imaging device and an imaging condition determination method suitable for use when, for example, a plurality of conference attendees in two conference rooms conduct a video conference.
In particular, the present invention accurately selects a microphone used by a speaker, selects one imaging means suitable for imaging a speaker using the selected microphone from among a plurality of imaging means, The present invention relates to a sound collection / video imaging apparatus capable of appropriately capturing a selected speaker.
The present invention further relates to a voice collection / video imaging apparatus and method that perform voiceprint authentication, and select a microphone installation area that is selected based on the voiceprint authentication and that can be picked up by a selected imaging means.

  A video conference system is used in order for conference attendees in two conference rooms located at distant locations to hold a conference. The video conference system captures the attendees in each conference room with the image capturing means, collects the sound with the microphone, and uses the communication path to collect the image captured with the image capturing means and the sound collected with the microphone. The captured image is displayed on the display unit of the television receiver in the other party's conference room, and the sound collected from the speaker is output.

  In such a video conference system, in each conference room, a problem has been encountered that it is difficult for voices of speakers who are far away from the imaging means and the microphone to be collected. A microphone may be provided for each. There is also a problem that the audio output from the speaker of the television receiver is difficult to hear for conference attendees located away from the speaker.

  In JP 2003-87887 A and JP 2003-87890 A, in addition to a normal video conference system that provides video and audio when a video conference is performed between conference rooms located at a distance from each other, Voice input / output device with a built-in microphone and speaker that can clearly hear the voices of meeting attendees in the conference room from the speaker and is less susceptible to the noise in the conference room on this side or less burden on the echo canceller Is disclosed.

For example, a voice input / output device disclosed in Japanese Patent Laid-Open No. 2003-87887 is described with reference to FIGS. Supported from the bottom to the top by a speaker box 5 with a built-in speaker 6, a conical reflector 4 that diffuses a sound that opens radially upward, a sound shielding plate 3, and a column 8. A plurality of unidirectional microphones (four in FIGS. 6 and 7 and six in FIG. 23) are arranged radially at equal angles on a horizontal plane. The sound shielding plate 3 is for shielding the sound from the lower speaker 5 from entering a plurality of microphones.
The audio input / output devices disclosed in Japanese Patent Laid-Open Nos. 2003-87887 and 2003-87890 are used as means for complementing a video conference system that provides video and audio.
Japanese Patent Laid-Open No. 2003-87887 JP 2003-87890 A

Conventionally, selection of a microphone for recording a speaker's voice has been manually performed by a speaker or a third party such as a presenter.
In addition, one television camera is fixed on a wall or the like, and the direction of the television camera is adjusted based on the selected data, the shooting direction is changed, and the video of the speaker is captured. However, in such a method, the following problems are encountered.
(1) If you forget to switch microphones, the video of the speaker will not be captured.
(2) Since it takes time to move the camera, if the speaker changes frequently, the shooting of the speaker will not be in time.
(3) Images in a plurality of directions cannot be transmitted simultaneously.

  An object of the present invention is to accurately select a speaker's microphone, further select an imaging means corresponding to the selected microphone, and select the voice and video of the speaker who is speaking using the selected microphone. An object of the present invention is to provide a sound collection / video imaging device for output.

According to the present invention, a plurality of microphones arranged in an annular shape and each in a radial manner are provided corresponding to each of the plurality of microphones, and the corresponding microphones can be imaged so that the sound collection range of the corresponding microphones can be imaged. The sound collection range of the microphone is picked up at a predetermined position between a set of a plurality of first small image pickup means, the microphones, and the corresponding first small image pickup means, which are arranged in proximity to each other. A first relationship between at least one second small-sized image pickup means, each of the microphones and each corresponding first small-size image pickup means, and the first located in the vicinity of the microphone. If the two small image pickup means are located, the storage means storing the second relationship with the second small image pickup means, and the sound collection signals of the plurality of microphones are detected, Based on the first relationship stored in the storage means, a microphone selection means for selecting a microphone that has detected a valid sound collection signal among the detected sound collection signals, and a first one that is close to the selected microphone. An imaging unit that selects one small imaging unit and selects the second small imaging unit when the second small imaging unit corresponding to the second relationship stored in the storage unit exists. If there is a selection means, a first imaging signal picked up by the first small-size imaging means selected by the imaging means selection means, and the corresponding second small-size imaging means, the second small-size imaging means Imaging signal selection means for selectively outputting the second imaging signal picked up by the camera, and combining the first imaging signal and the second imaging signal selected by the imaging signal selection means into one. Ma Includes an image synthesizing means for dividing one screen, the sound pickup-image pickup device is provided.

According to the present invention, it is possible to automatically and quickly switch and output the video and audio of a speaker who uses the selected microphone.
That is, according to the present invention, the use of one of a plurality of microphones is selected, the imaging means for imaging the speaker portion of the selected microphone is selected, and the selected imaging means uses the microphone. The person can be effectively imaged. According to the present invention, even if the speaker changes during the conference, the imaging means for displaying the speaker is quickly and appropriately selected together with the speed of switching the microphone.
In the present invention, since the imaging means is automatically switched according to the selection of the microphone, there is no need to manually change the setting as in the prior art, and a clear image of the speaker using the selected microphone is obtained. Can continue to be projected.

  According to the present invention, a plurality of imaging means are provided, and if necessary, images of a speaker and directions other than the speaker or a person can be simultaneously captured and transmitted simultaneously. Further, according to the present invention, it is possible to project only a specific direction regardless of the speaker, or to prevent a video in a specific direction from being displayed.

  Furthermore, according to the present invention, it is possible to prevent the selection of a camera that can shoot the direction one by one by providing a time during which the processing is not performed for a certain period of time for a very short time pronunciation such as a simple match. .

  In addition, according to the present invention, it is possible to automatically switch between the microphone on this side and the imaging means in accordance with an instruction from a conference partner. That is, it is possible to output a person's voice and its video according to the desire of the person in the other party's conference room.

  Furthermore, according to the present invention, the microphone can be accurately selected by using the speaker direction detection technique and the image recognition technique. Based on the result, an imaging unit corresponding to the selected microphone can be selected.

The following describes the sound collection / video imaging apparatus according to the embodiment of the present invention.
FIGS. 1A to 1C are configuration diagrams showing an example to which the sound collection / video imaging apparatus according to the embodiment of the present invention is applied.
As illustrated in FIG. 1A, the first and second sound collection / video imaging devices 1A and 1B are installed in the two conference rooms 901 and 902, respectively. The imaging devices 1A and 1B are connected by a communication line 920, for example, a telephone line.

[Outline of the sound collection and imaging device]
FIG. 2 is a plan layout view of the sound collection / video imaging apparatus 1A according to the embodiment of the present invention. The first and second audio collecting / imaging devices 1A and 1B have the same configuration.
As a representative example of the first sound collection / video imaging device 1A, the first sound collection / video imaging device 1A includes the first call device 10A corresponding to the sound collection means of the present invention, It has two first television camera (television camera) devices 40A1 and 40A2 corresponding to the imaging means of the invention. The communication device detects the speech of the conference participant, determines the speaker, and reports the speech of the speaker to other conference participants in the conference room and the conference attendee in the other conference room. Furthermore, the communication device provides the imaging conditions of the television camera devices 40A1 and 40A2 based on the identification of the speaker.
The TV camera devices 40A1 and 40A2 automatically capture an optimal image based on the provided imaging conditions.

The first sound collection / video imaging device 1A may include the television receiver 50A and / or the first projector device 60A.
The projector device 60A is, for example, a projector device using liquid crystal as modulation means. When various materials used for the conference are provided from a personal computer, the projector device 60A projects the image on the screen S and is visible to the conference attendees A1 to A8. And
The television receiver 50A projects the images captured by the TV camera devices 40A1 and 40A2 or the images captured by the TV camera devices 40B1 and 40B2 in the other party's conference room onto the screen S and displays them on the conference attendees A1 to A8. . It is to be noted that the television receiver 50A is deleted and the video captured by the television camera devices 40A1 and 40A2 or the video captured by the television camera devices 40B1 and 40B2 in the other party's conference room is provided as the video provided from the personal computer. It can also be switched and projected onto the screen S via the projector device 60A and displayed on the conference attendees A1 to A8. Hereinafter, a case will be described in which an image captured by the television camera devices 40A1 and 40A2 is displayed by the projector device 60A without using the television receiver 50A.

Preferably, communication device 10A and projector device 60A are placed on table 911. FIG. 1B shows the communication device 10 </ b> A placed on the table 911.
As illustrated in FIGS. 1C and 2, a plurality of conference attendants A1 to A6 (A1 to A8) around the communication device 10A (six people in FIG. 1C and eight people in FIG. 2). Is located.

The second sound collection / video imaging device 1B (not shown) also includes a second call device 10B and second two television camera (television camera) devices 40B1 and 40B2.
The sound collection / video imaging device 1B may include the second projector device 60B and the television receiver 50B.
Preferably, call device 10B and projector device 60B are placed on table 912 in conference room 902.

[Calling equipment]
A voice response is made via the communication line 920 between the first call device 10A and the second call device 10B.
Normally, a conversation via the communication line 920 is performed by one speaker and one speaker, that is, one-to-one, but the communication device according to the embodiment of the present invention uses one communication line 920. By using this, a plurality of conference attendees in the conference rooms 901 and 902 can talk with each other. However, in this embodiment, in order to avoid voice congestion and to enable imaging of a speaker with a television camera device, the number of speakers at the same time (same time zone) is limited to one person.
Details of the communication device will be described later.

[Television camera device and television receiver]
For example, the television camera devices 40A1 and 40A2 in the first sound collection / video imaging device 1A image the caller specified by the first call device 10A. Therefore, the television camera devices 40A1 and 40A2 have pan, tilt, zoom functions, and the like.
Images captured by the television camera devices 40A1 and 40A2 are displayed on the projector device 60B (or the television receiver 50B) in the conference room on the other side via the communication line 920.
If necessary, video captured by the television camera devices 40A1 and 40A2 can be displayed on the projector device 60A (or the television receiver 50A) in the conference room on the own side.

[Identification method of imaging target]
The identification method of the imaging target imaged by the TV camera devices 40A1 and 40A2 uses the direction of the speaker in the first call device 10A and the voiceprint recognition result of the speaker registered in advance. The details are performed in the imaging adjustment unit 36, which will be described later.

The second sound collection / video imaging apparatus 1B performs the same processing as the first sound collection / video imaging apparatus 1A.
As described above, the sound collecting / imaging devices 1A and 1B select (specify) the caller and collect the sound of the selected caller in the call devices 10A and 10B. Furthermore, the TV camera devices 40A1 and 40A2 capture the video of the selected (specified) caller based on the command of the imaging adjustment unit 36.
The collected audio and the captured video are transferred to the conference room on the other side, the sound is played back by the call device in the other party's voice collection and video imaging device, and the video is displayed on the projector device (or television receiver). To do.

Details of the Call Device The configuration of the call device in the sound collection / video imaging device according to the embodiment of the present invention will be described with reference to FIGS. The same applies to the communication device 10A and the second communication device 10B.
FIG. 3 is a perspective view of a communication device as an embodiment of the present invention.
FIG. 4 is a cross-sectional view of the communication device illustrated in FIG.
FIG. 5 is a plan view of the microphone / electronic circuit housing portion of the communication device illustrated in FIGS. 3 and 4, and is a plan view taken along line XX in FIG.

As illustrated in FIG. 3, the communication device includes an upper cover 11, a sound reflection plate 12, a connecting member 13, a speaker housing unit 14, and an operation unit 15.
As illustrated in FIG. 4, the speaker housing 14 includes a sound reflecting surface 14 a, a bottom surface 14 b, and an upper sound output opening 14 c. The reception / reproduction speaker 16 is accommodated in a lumen 14d which is a space surrounded by the sound reflection surface 14a and the bottom surface 14b. The sound reflecting plate 12 is positioned above the speaker housing portion 14, and the speaker housing portion 14 and the sound reflecting plate 12 are connected by a connecting member 13.

  A constraining member 17 passes through the connecting member 13, and the constraining member 17 is between the constraining member lower fixing portion 14 e on the bottom surface 14 b of the speaker housing portion 14 and the constraining member fixing portion 12 b of the sound reflecting plate 12. Restrained. However, the restraining member 17 is only penetrated by the restraining member penetration portion 14 f of the speaker housing portion 14. The reason why the restraining member 17 penetrates the restraining member through portion 14f and is not restrained here is that the speaker housing portion 14 vibrates due to the operation of the speaker 16, but the vibration is not restrained around the upper sound output opening 14c. Because.

The voice spoken by the speaker in the other party's conference room is extracted from the upper sound output opening 14c through the reception / reproduction speaker 16, and is defined by the sound reflecting surface 12a of the sound reflecting plate 12 and the sound reflecting surface 14a of the speaker accommodating portion 14. And spread in all directions of 360 degrees around the axis CC along the space.
As illustrated, the cross section of the sound reflecting surface 12a of the sound reflecting plate 12 depicts a gentle trumpet arc. The cross section of the sound reflecting surface 12a has a cross-sectional shape illustrated over 360 degrees (over all directions) about the axis CC.
Similarly, as illustrated in the cross section of the sound reflection surface 14a of the speaker housing portion 14, a gentle convex surface is drawn. The cross section of the sound reflecting surface 14a also has the illustrated cross sectional shape over 360 degrees (omnidirectional) about the axis CC.

The sound S emitted from the reception / reproduction speaker 16 passes through the upper sound output opening 14c, passes through a sound output space having a trumpet-shaped cross section defined by the sound reflection surface 12a and the sound reflection surface 14a, and the communication device is placed. Along the surface of the table 911, the sound is diffused 360 degrees in all directions around the axis C-C, and is heard at a volume equal to all the attendees A1 to A6. In the present embodiment, the surface of the table 911 is also used as part of the sound propagation means.
The diffusion state of the sound S output from the receiving / reproducing speaker 16 is shown by arrows.

The sound reflecting plate 12 supports the printed circuit board 21.
On the printed circuit board 21, as illustrated in a plan view in FIG. 5, the microphones MC1 to MC6, the light emitting diodes LED1 to 6 of the microphone / electronic circuit housing unit 2, the microprocessor 23, the codec (CODEC) 24, the first digital signal. Various electronic circuits such as a processor (DSP 1) DSP 25, a second digital signal processor (DSP 2) DSP 26, an A / D converter block 27, a D / A converter block 28, and an amplifier block 29 are mounted on the sound reflector. Reference numeral 12 also functions as a member that supports the microphone / electronic circuit housing portion 2.

  The printed circuit board 21 has a damper 18 that absorbs vibration from the reception / reproduction speaker 16 so that vibration from the reception / reproduction speaker 16 is transmitted to the sound reflector 12 and does not enter the microphones MC1 to MC6. It is attached. The damper 18 includes a screw and a cushioning material such as an anti-vibration rubber inserted between the screw and the printed board 21, and the cushioning material is screwed to the printed board 21 with a screw. That is, the vibration transmitted from the reception / reproduction speaker 16 to the printed circuit board 21 is absorbed by the buffer material. Thereby, the microphones MC1 to MC6 are not affected by the sound from the speaker 16.

Microphone Arrangement As illustrated in FIG. 5, six microphones MC1 to MC6 are located radially from the central axis C of the printed circuit board 21 at an equal angle and at equal intervals (equal angle of 60 degrees in the present embodiment). ing. Each microphone is a unidirectional microphone. Its characteristics will be described later.
Each of the microphones MC1 to MC6 is swingably supported by a first microphone support member 22a and a second microphone support member 22b, both of which are flexible or elastic (in order to simplify the illustration, the microphones Only the first microphone support member 22a and the second microphone support member 22b in the MC1 portion are illustrated), and is not affected by the vibration from the reception / reproduction speaker 16 by the damper 18 using the above-described cushioning material. In addition to the countermeasures, the first microphone support member 22a and the second microphone support member 22b having flexibility or elasticity absorb the vibration of the printed circuit board 21 that is vibrated by the vibration from the reception / reproduction speaker 16, and reproduce the reception. The noise of the receiving / reproducing speaker 16 is avoided so as not to be affected by the vibration of the speaker 16.

As illustrated in FIG. 4, the reception / reproduction speaker 16 is oriented perpendicularly to the central axis CC of the plane on which the microphones MC1 to MC6 are located (in the present embodiment, it is directed upward) With the arrangement of the reception / reproduction speaker 16 and the six microphones MC1 to MC6, the distance between the reception / reproduction speaker 16 and each of the microphones MC1 to MC6 is equal. The sound reaches the microphones MC1 to MC6 with almost the same volume and phase. However, due to the configuration of the sound reflection surface 12a of the sound reflection plate 12 and the sound reflection surface 14a of the speaker housing portion 14, the sound of the reception and reproduction speaker 16 is not directly input to the microphones MC1 to MC6. In addition, as described above, by using the damper 18 using the buffer material, the first microphone support member 22a and the second microphone support member 22b having flexibility or elasticity, the reception / reproduction speaker 16 is provided. The influence of vibration is reduced.
As shown in FIG. 1C, for example, conference attendees A1 to A6 are usually substantially equal to the vicinity of microphones MC1 to MC6 arranged at intervals of 60 degrees in the direction of 360 degrees around the communication device. Located at intervals. In the example illustrated in FIG. 2, eight conference attendees are located around the call device.

Light emitting diodes LED1 to 6 are arranged in the vicinity of the microphones MC1 to MC6 as means for notifying that the speaker has been determined (microphone selection result display means).
The light emitting diodes LED1 to 6 are provided so as to be visible from all the conference attendants A1 to A6 even when the upper cover 11 is attached. Therefore, the upper cover 11 is provided with a transparent window so that the light emitting states of the light emitting diodes LED1 to LED6 can be visually recognized. Of course, the upper cover 11 may be provided with openings in the portions of the light emitting diodes LEDs 1 to 6, but a light-transmitting window is preferable from the viewpoint of dust prevention to the microphone / electronic circuit housing portion 2.

The printed circuit board 21 includes a first digital signal processor (DSP 1) 25, a second digital signal processor (DSP 2) 26, and various electronic circuits 27 to 29 for performing various signal processing described later. It is arranged in a space other than the part where the MC 6 is located.
In the present embodiment, the DSP 25 is used as signal processing means for performing processing such as filter processing and microphone selection processing together with various electronic circuits 27 to 29, and the DSP 26 is used as an echo canceller.

FIG. 6 is a schematic configuration diagram of the microprocessor 23, the codec 24, the DSP 25, the DSP 26, the A / D converter block 27, the D / A converter block 28, the amplifier block 29, and other various electronic circuits.
The microprocessor 23 performs overall control processing of the microphone / electronic circuit housing unit 2. The codec 24 compresses and encodes audio to be transmitted to the other party conference room.
The DSP 25 performs various signal processing described below, such as filter processing and microphone selection processing.
The DSP 26 functions as an echo canceller.
In FIG. 6, four A / D converters 271 to 274 are illustrated as an example of the A / D converter block 27, and two D / A converters are illustrated as an example of the D / A converter block 28. The converters 281 to 282 are illustrated, and two amplifiers 291 to 292 are illustrated as an example of the amplifier block 29.
In addition, as the microphone / electronic circuit housing portion 2, various circuits such as a power supply circuit are mounted on the printed circuit board 21.

In FIG. 5, a pair of microphones MC1-MC4: MC2-MC5: MC3-M6 arranged in a straight line at symmetrical (or opposite) positions with respect to the central axis C of the printed circuit board 21 are respectively two channels. The analog signals are inputted to A / D converters 271 to 273 for converting them into digital signals. In this embodiment, one A / D converter converts a 2-channel analog input signal into a digital signal. Therefore, the detection signals of two (one pair) microphones, for example, microphones MC1 and MC4, which are positioned on a straight line across the central axis C, are input to one A / D converter and converted into digital signals. Yes. Further, in this embodiment, in order to identify the speaker of the voice to be sent to the other party's conference room, the difference between the two microphones positioned on a straight line, the volume of the voice, etc. are referred to. When the signals of two microphones located on the line are input to the same A / D converter, the conversion timing is also substantially the same, and there is little timing error when taking the difference between the audio outputs of the two microphones. There are advantages such as being easy.
The A / D converters 271 to 274 can also be configured as A / D converters 271 to 274 with a variable gain amplification function.
The collected sound signals of the microphones MC1 to MC6 converted by the A / D converters 271 to 273 are input to the DSP 25, and various signal processing described later is performed.
As one of the processing results of the DSP 25, the result of selecting one of the microphones MC1 to MC6 is output to the light emitting diodes LED1 to 6 which are an example of the microphone selection result display means.

The processing result of the DSP 25 is output to the DSP 26 and an echo cancellation process is performed. The DSP 26 includes, for example, an echo cancellation transmission processing unit and an echo cancellation reception unit.
The processing result of the DSP 26 is converted into an analog signal by the D / A converters 281 to 282. The output from the D / A converter 281 is encoded by the codec 24 as necessary, and is output to the line-out of the communication line 920 (FIG. 1A) via the amplifier 291 to the partner conference room. It is output as sound through the receiving / reproducing speaker 16 of the installed communication device.
Voice from a communication device installed in the other party's conference room is input via the line-in of the communication line 920 (FIG. 1A), converted into a digital signal by the A / D converter 274, and input to the DSP 26. And used for echo cancellation processing. In addition, the sound from the communication device installed in the other party's conference room is applied to the speaker 16 through a route (not shown) and output as sound.
An output from the D / A converter 282 is output as a sound from the reception reproduction speaker 16 of the communication device via the amplifier 292. That is, the conference attendees A1 to A6 use the reception / reproduction speaker 16 for the voice of the speaker in the conference room in addition to the voice of the speaker selected in the conference room from the reception / reproduction speaker 16 described above. Can be heard through.

Microphones MC1 to MC6
FIG. 7 is a graph showing the directivity of each of the microphones MC1 to MC6.
Each unidirectional characteristic microphone changes its frequency characteristic and level characteristic as illustrated in FIG. 7 depending on the arrival angle of the sound from the speaker to the microphone. The plurality of curves indicate directivity when the frequency of the sound collection signal is 100 Hz, 150 Hz, 200 Hz, 300 Hz, 400 Hz, 500 Hz, 700 Hz, 1000 Hz, 1500 Hz, 2000 Hz, 3000 Hz, 4000 Hz, 5000 Hz, and 7000 Hz. However, for simplicity of illustration, FIG. 7 typically illustrates the directivity for 150 Hz, 500 Hz, 1500 Hz, 3000 Hz, and 7000 Hz.

8A to 8D are graphs showing the analysis results of the position of the sound source and the sound collection level of the microphone, and each microphone is placed with a speaker placed at a predetermined distance, for example, a distance of 1.5 meters. The result of fast Fourier transform (FFT) of the collected sound at regular time intervals is shown. The X axis represents frequency, the Y axis represents signal level, and the Z axis represents time.
When the microphone having directivity shown in FIG. 7 is used, strong directivity is shown in front of the microphone. In the present embodiment, using such characteristics, the DSP 25 performs a microphone selection process.

When an omnidirectional microphone is used instead of a directional microphone as in the embodiment of the present invention, all sounds around the microphone are collected (sound collection). S / N is confused and cannot collect very good sound. In order to avoid this, in the present invention, S / N with surrounding noise is improved by collecting sound with one directional microphone.
Furthermore, a microphone array using a plurality of omnidirectional microphones can be used as a method for obtaining the directivity of the microphone. However, in such a method, the time axis (phase) of a plurality of signals is complicated, which is complicated. Since processing is required, it takes time and response is low, and the apparatus configuration is complicated. That is, the DSP signal processing system also requires complicated signal processing. The present invention solves such a problem by using the directional microphone illustrated in FIG.
Further, in order to synthesize a microphone array signal and use it as a directional sound collecting (sound collecting) microphone, there is a disadvantage that the outer shape is restricted by the pass frequency characteristic and the outer shape becomes large. The present invention also solves this problem.

The communication device configured as described above has the following advantages.
(1) Since the positional relationship between the even number of microphones MC1 to MC6 radially arranged at equal angles and at equal intervals and the reception / reproduction speaker 16 is constant and the distance is very close, the reception / reproduction speaker 16 The level at which the output sound returns directly to the microphones MC1 to MC6 via the conference room (room) environment is overwhelmingly dominant. For this reason, the characteristics (signal level (intensity) and frequency characteristics (f characteristics, phase) in which sound reaches the microphones MC1 to MC6 from the speaker 16 are always the same. That is, in the communication device according to the embodiment of the present invention. There is an advantage that the transfer function is always the same.
(2) Therefore, there is no change in the transfer function when the output of the microphone sent to the other party's conference room is switched when the speakers are different, and there is no need to adjust the gain of the microphone system each time the microphone is switched. Have In other words, there is an advantage that once the adjustment is made at the time of manufacturing the telephone device, there is no need to redo the adjustment.
(3) Even if the microphones are switched when the speakers are different for the same reason as described above, only one echo canceller (DSP 26) is required. The DSP is expensive, and it is not necessary to arrange a plurality of DSPs on the printed circuit board 21 on which various members are mounted and the space is small. As a result, the printed circuit board 21, and thus the communication device of the present invention can be reduced in size.
(4) Since the transfer function between the reception / reproduction speaker 16 and the microphones MC1 to MC6 is constant as described above, for example, the advantage that the sensitivity difference of the microphone itself having ± 3 dB can be adjusted by the microphone unit alone of the communication device. There is. Details of the sensitivity difference adjustment will be described later.
(5) The table on which the communication device is mounted is usually a round table or a polygonal table, so that sound of equal quality is centered on the axis C by one receiving / reproducing speaker 16 in the communication device. A speaker system capable of evenly dispersing (diffusing) in all directions at 360 degrees has become possible.
(6) The sound emitted from the receiving / reproducing speaker 16 is transmitted to the table surface of the round table (boundary effect), effectively and efficiently delivering high-quality sound to the meeting attendees, and facing the ceiling direction of the conference room There is an advantage that the sound and the phase on the side are canceled to become a small sound, the reflected sound from the ceiling direction is less for the conference attendee, and as a result, a clear sound is distributed to the participants.
(7) Since the sound emitted from the reception / reproduction speaker 16 reaches all the microphones MC1 to MC6 arranged radially and at equal intervals at the same angle at the same volume at the same time, it is determined whether the sound is the voice of the speaker or the received voice. It becomes easy. As a result, erroneous determination of microphone selection processing is reduced. Details thereof will be described later.
(8) Even number, for example, six microphones are arranged at equal angles radially and at equal intervals, and a pair of opposing microphones are arranged in a straight line, so that level comparison for direction detection can be easily performed.
(9) By the damper 18, the microphone support member 22, and the like, it is possible to reduce the influence of the vibration due to the sound of the reception and reproduction speaker 16 on the sound collection of the microphones MC1 to MC6.
(10) As illustrated in FIG. 4, structurally, the sound of the reception / reproduction speaker 16 does not propagate directly to the microphones MC1 to MC6. Therefore, in this communication device, the influence of noise from the reception / reproduction speaker 16 is small.

Modified Example The communication device described with reference to FIGS. 3 to 4 has the reception reproduction speaker 16 disposed in the lower portion and the microphones MC1 to MC6 (and related electronic circuits) disposed in the upper portion. And the positions of the microphones MC1 to MC6 (and related electronic circuits) can also be turned upside down as illustrated in FIG. Even in such a case, the above-described effects are exhibited.

  The number of microphones is not limited to six, and any number of microphones, such as four, eight, etc., may be arranged in a straight line (in the same direction) with a plurality of pairs radially centered on axis C at equal angles and at equal intervals. For example, the microphones MC1 and MC4 are arranged in a straight line. The reason why the two microphones MC1 and MC4 are arranged in a straight line as a preferred form is to select a microphone and identify a speaker.

Signal Processing Contents Hereinafter, processing contents mainly performed by the first digital signal processor (DSP) 25 will be described.
FIG. 10 is a diagram illustrating an outline of processing in the communication device performed by the DSP 25. The outline is described below.

(1) Measurement of ambient noise As an initial operation, preferably, ambient noise where the communication device 10A is installed is measured.
The call device can be used in various environments (conference rooms). In the present invention, in order to increase the accuracy of the selection of the microphone and to improve the performance of the communication device, in the initial stage, noise in the surrounding environment where the communication device is installed is measured, and the influence of the noise is collected by the microphone. It is possible to exclude it from the processed signal.
Of course, when the communication device is repeatedly used in the same conference room, noise measurement is performed in advance, and this processing can be omitted when the noise state does not change.
Note that noise measurement can also be performed in a normal state.

(2) Selection of Chairperson For example, when a telephone device is used for a two-way conference, it is beneficial to have a chairperson who manages the proceedings in each conference room. Therefore, as one aspect of the present invention, the chairperson is set from the operation unit 15 of the call device in the initial stage of using the call device. As a chairperson setting method, for example, the first microphone MC1 located in the vicinity of the operation unit 15 is used as a chairperson microphone. Of course, the chairman's microphone can be arbitrary.
Note that this process can be omitted when the chairperson who uses the telephone device repeatedly is the same. Or you may decide the microphone of the position where a chairperson sits beforehand. In that case, there is no need to select a chairman each time.
Of course, the selection of the chair is not limited to the initial state, and can be performed at any timing.

(3) Microphone sensitivity difference adjustment As an initial operation, preferably, the gain or attenuation unit of the amplification unit that amplifies the signals of the microphones MC1 to MC6 so that the acoustic coupling between the reception reproduction speaker 16 and the microphones MC1 to MC6 is equal. Automatically adjust the attenuation value.

Various processes exemplified below are performed as normal processes.
(1) Microphone selection and switching processing When a plurality of conference attendees talk at the same time in one conference room, voices are mixed and difficult for the conference attendees A1 to A6 in the other conference room. Therefore, in the present invention, in principle, one person is allowed to talk at a time. For this reason, the DSP 25 performs microphone selection / switching processing.
As a result, only the call from the selected microphone is transmitted to the call device in the other party's conference room via the communication line 920 and output from the speaker. Of course, as described with reference to FIG. 6, the LED in the vicinity of the selected speaker's microphone is turned on, and the selected speaker's voice can be heard from the speaker of the communication device in the room. And recognize who is the authorized speaker.
The purpose of this processing is to select a signal from a unidirectional microphone facing the speaker and send a signal having a good S / N to the other party as a transmission signal.
(2) Display of selected microphone A microphone is selected so that all the conference participants A1 to A6 can easily recognize which conference participant's microphone is selected and allowed to speak. Selection result display means, for example, corresponding ones of the light emitting diodes LED1 to LED6 are turned on.
(3) Determination of imaging conditions (third embodiment)
In the imaging adjustment unit 36 described as the third embodiment, the imaging conditions of the television camera devices 40A1 and 40A2 can be determined using the above-described microphone selection (specification) result by the communication device.
(4) As a background technique of the above-described microphone selection process, or in order to accurately perform the microphone selection process, various signal processes exemplified below are performed.
(A) Band separation and level conversion processing of microphone collected signal (b) Start / end determination processing of speech
To be used as a trigger to start selecting the microphone signal that faces the speaker direction.
(C) Speaker direction microphone detection processing
To analyze the collected sound signal of each microphone and determine the microphone used by the speaker.
(D) Speaker direction microphone switching timing determination processing, and microphone signal selection switching processing facing the detected speaker
An instruction to switch to the microphone selected from the above processing result is given. (E) Measurement of floor noise during normal operation

Measurement of floor (environment) noise This process is divided into an initial process and a normal process immediately after the communication device is turned on.
This process is performed under the following exemplary preconditions.

[Table 1]
(1) Conditions: Measurement time and threshold provisional value:
1. Test tone sound pressure: -40dB at microphone signal level
2. 2. Noise measurement unit time: 10 seconds Noise measurement in a normal state: The average value is calculated from the measurement result for 10 seconds, and this is repeated 10 times to obtain the average value to obtain the noise level.

[Table 2]
(2) Estimated effective distance and threshold based on the difference between floor noise and speech start reference level 1.26 dB or more: 3 meters or more
Detection level threshold for starting speech: Floor noise level +9 dB
Talk level detection level threshold: floor noise level + 6 dB
2.20 to 26 dB: within 3 meters
Detection level threshold for starting speech: Floor noise level +9 dB
Talk level detection level threshold: floor noise level + 6 dB
3.14 to 20 dB: within 1.5 meters
Detection level threshold for starting speech: Floor noise level +9 dB
Talk level detection level threshold: floor noise level + 6 dB
4.9-14dB: within 1 meter
Detection level threshold for starting speech:
Difference between floor noise level and speech start reference level ÷ 2 + 2 dB
Talk end threshold: Talk start threshold-3 dB
5.9 dB or less: tens of centimeters
Detection level threshold for speech start: -3 dB
6). Difference between floor noise level and speech start reference level ÷ 2
Talk end detection level threshold: -3 dB
7). Same or negative: Cannot be judged and cannot be selected

[Table 3]
(3) The noise measurement start threshold value of the normal process starts when the level becomes lower than the floor noise at the time of power-on + 3 dB.

Generation of Various Frequency Component Signals by Filter Processing FIG. 11 is a configuration diagram showing filtering processing performed by the DSP 25 using sound signals collected by a microphone as preprocessing. FIG. 11 shows processing for one microphone (channel (one sound collection signal)).
The collected sound signal of each microphone is processed by an analog low cut filter 101 having a cutoff frequency of 100 Hz, for example, and a filtered audio signal from which a frequency of 100 Hz or less has been removed is output to the A / D converter 102. , Digital high-cut filters 103a to 103e (collectively referred to as “collection signals”) having cut-off frequencies of 7.5 KHz, 4 KHz, 1.5 KHz, 600 Hz, and 250 Hz, respectively. 103), high frequency components are removed (high cut processing). The results of the digital high cut filters 103a to 103e are further subtracted for each filter signal of the adjacent digital high cut filters 103a to 103e in subtractors 104a to 104d (collectively 104).
In the embodiment of the present invention, the digital high cut filters 103a to 103e and the subtractors 104a to 104d are actually processed in the DSP 25. The A / D converter 102 can be realized as one of the A / D converter blocks 27.

  FIG. 12 is a frequency characteristic diagram showing the filter processing result described with reference to FIG. Thus, a plurality of signals having various frequency components are generated from a signal collected by a microphone having one directivity.

The start / end of speech is determined as one of the triggers for starting the band-pass filter processing and microphone signal level conversion processing microphone selection processing. A signal used for this purpose is obtained by the bandpass filter processing and level conversion processing illustrated in FIG. FIG. 13 shows only 1CH during processing of 6-channel (CH) input signals collected by the microphones MC1 to MC6.
The band-pass filter processing and level conversion processing unit in the DSP 25 respectively collects the collected sound signals of the microphones of each channel at 100 to 600 Hz, 200 to 250 Hz, 250 to 600 Hz, 600 to 1500 Hz, 1500 to 4000 Hz, 4000 to 7500 Hz. Band-pass filters 201a to 201f having band-pass characteristics (collectively, band-pass filter block 201), original microphone sound collection signals, and level converters 202a to 202g (for converting the levels of the band-pass sound collection signals). Collectively, it has a level conversion block 202).

  Each of the level converters 202a to 202g includes a signal absolute value processing unit 203 and a peak hold processing unit 204. Therefore, as illustrated in the waveform diagram, the signal absolute value processing unit 203 inverts the sign and converts it to a positive signal when a negative signal indicated by a broken line is input. The peak hold processing unit 204 holds the maximum value of the output signal of the signal absolute value processing unit 203. However, in the present embodiment, the held maximum value is somewhat lowered with the passage of time. Of course, the peak hold processing unit 204 can be improved so that the maximum value can be held for a long time by reducing the decrease.

A bandpass filter will be described. The bandpass filter used for the speech device of the sound collecting / imaging device is, for example, a bandpass filter made up of only a secondary IIR high cut filter and a low cut filter at the microphone signal input stage.
In the present embodiment, it is utilized that if the signal that has passed through the high-cut filter is subtracted from the signal having a flat frequency characteristic, the rest is substantially equivalent to the signal that has passed through the low-cut filter.
In order to match the frequency-level characteristics, an extra band-pass bandpass filter is required for one band, but the band required by the number of filter stages equal to the number of bands of the required bandpass filter + 1 and the filter coefficient A pass is obtained. The band frequency of the handpass filter required this time is a 6-band bandpass filter shown in Table 4 below per one channel (CH) of the microphone signal.

[Table 4]
BP characteristic band pass filter
BPF1 = [100Hz-250Hz] ・ ・ 201b
BPF2 = [250Hz-600Hz] ・ ・ 201c
BPF3 = [600Hz-1.5KHz] ・ ・ 201d
BPF4 = [1.5KHz-4KHz] ・ ・ 201e
BPF5 = [4KHz-7.5KHz] ・ ・ 201f
BPF6 = [100Hz-600Hz] ・ ・ 201a

In this method, the calculation program of the above IIR filter in the DSP 25 is only 6CH (channel) × 5 (IIR filter) = 30.
Contrast with the conventional bandpass filter configuration. Assuming that the band-pass filter uses a second-order IIR filter and a 6-band band-pass filter is prepared for each of six microphone signals as in the present invention, in the conventional method, 6 × 6 × 2 = 72. Circuit IIR / filtering is required. This processing requires considerable program processing even with the latest excellent DSP, and affects other processing.
In the embodiment of the present invention, the 100 Hz low cut filter is processed by an analog filter in the input stage. There are five types of cutoff frequencies of the prepared second-order IIR high cut filters: 250 Hz, 600 Hz, 1.5 KHz, 4 KHz, and 7.5 KHz. Of these, a high-cut filter with a cutoff frequency of 7.5 KHz is not necessary because the sampling frequency is actually 16 KHz. Deliberately rotate the phase of the attenuator to reduce the phenomenon.

  FIG. 14 is a flowchart when processing by the DSP 25 is performed according to the configuration illustrated in FIG.

  The filter processing in the DSP 25 illustrated in FIG. 14 performs high-pass filter processing as the first-stage processing and subtraction processing from the result of the first-stage high-pass filter processing as the second-stage processing. FIG. 12 is an image frequency characteristic diagram of the signal processing result. [X] below shows each processing case in FIG.

First stage [1] The input signal is passed through a 7.5 kHz high cut filter for the whole band pass filter. This filter output signal becomes a bandpass filter output of [100Hz-7.5KHz] by matching the analog low cut of the input.

  [2] Pass the input signal through a 4KHz high cut filter. This filter output signal becomes a bandpass filter output of [100Hz-4KHz] by combining with the input analog low cut filter.

  [3] Pass the input signal through a 1.5 kHz high cut filter. This filter output signal becomes a bandpass filter output of [100Hz-1.5KHz] by combining with the input analog low cut filter.

  [4] Pass the input signal through a 600Hz high-cut filter. This filter output signal becomes a bandpass filter output of [100Hz-600Hz] by combining with the input analog low cut filter.

  [5] Pass the input signal through a 250Hz high cut filter. This filter output signal becomes a bandpass filter output of [100Hz-250Hz] by combining with the input analog low cut filter.

The second stage [1] band pass filter (BPF5 = [4KHz ~ 7.5KHz]) executes the process of filter output [1]-[2] ([100Hz ~ 7.5KHz]-[100Hz ~ 4KHz]) The signal output is [4KHz to 7.5KHz].
[2] The bandpass filter (BPF4 = [1.5KHz to 4KHz]) will perform the above processing when the filter output [2]-[3] ([100Hz to 4KHz]-[100Hz to 1.5KHz]) is executed. Output [1.5KHz ~ 4KHz].
[3] The bandpass filter (BPF3 = [600Hz to 1.5KHz]) performs the above processing when the filter output [3]-[4] ([100Hz to 1.5KHz]-[100Hz to 600Hz]) is executed. Output [600Hz ~ 1.5KHz].
[4] The bandpass filter (BPF2 = [250Hz to 600Hz]) performs the process of filter output [4]-[5] ([100Hz to 600Hz]-[100Hz to 250Hz]). ~ 600Hz]. [5] The bandpass filter (BPF1 = [100 Hz to 250 Hz]) uses the signal [5] as it is as the output signal [5].
[6] The bandpass filter (BPF6 = [100 Hz to 600 Hz]) uses the signal [4] as it is as the output signal [4].
The bandpass filter output required by the above processing in the DSP 25 is obtained.

  The input microphone sound collection signals MIC1 to MIC6 are constantly updated in the DSP 25 as the sound pressure level of the entire band and the sound pressure level of the six bands that have passed through the bandpass filter as shown in Table 5.

In Table 5, for example, L1-1 indicates a peak level when the collected sound signal of the microphone MC1 passes through the first bandpass filter 201a.
The start and end of speech is determined by using a microphone sound collection signal that has passed through the 100 Hz to 600 Hz bandpass filter 201a shown in FIG. 13 and whose sound pressure level has been converted by the level converter 202b.

  The conventional band-pass filter is configured by combining a high-pass filter and a low-pass filter for each stage of the band-pass filter. Therefore, a 36-band band-pass filter of the specification used in this embodiment is used. When constructed, 72 circuits of filter processing are required. In contrast, the filter configuration of the embodiment of the present invention is simplified as described above.

Sentence start / end determination processing The first digital signal processor (DSP1) 25, based on the value output from the sound pressure level detector, as shown in FIG. If the voice start level exceeds the threshold of the voice start level, it is determined that the voice starts.If the level continues to be higher than the threshold of the voice start level after that, the voice is terminated and the sound collection signal level falls below the threshold. When it is determined as noise and the speech end determination time, for example, floor noise continues for 0.5 seconds, it is determined that the speech ends.
The start of the speech is as follows. Sound pressure level data (microphone signal level (1)) that has passed through a band pass filter of 100 Hz to 600 Hz that has been subjected to sound pressure level conversion by the microphone signal conversion processing unit 202b illustrated in FIG. It is determined that the speech starts when the threshold level is exceeded.
In order to avoid malfunction due to frequent microphone switching, the DSP 25 does not detect the start of the next speech until the speech termination determination time, for example, 0.5 seconds elapses after the speech start is detected. Yes.

The microphone selection DSP 25 performs speaker direction detection and automatic selection of a microphone signal facing the speaker in the mutual communication system based on a so-called “star chart method”.
FIG. 16 is a graph illustrating the operation mode of the communication device of the sound collecting / imaging device.
FIG. 17 is a flowchart showing normal processing of the telephone device.

As shown in FIG. 16, the communication device performs voice signal monitoring processing according to the collected sound signals from the microphones MC1 to MC6, performs speech start / end determination, performs speech direction determination, performs microphone selection, The result is displayed on the microphone selection result display means, for example, the light emitting diodes LED1 to LED6.
The operation will be described below with the DSP 25 in the call device 1 as a main component with reference to the flowchart of FIG. The overall control of the microphone / electronic circuit housing unit 2 is performed by the microprocessor 23, and the processing of the DSP 25 will be mainly described.

Step S1: Level Conversion Signal Monitoring Signals collected by the microphones MC1 to MC6 are respectively obtained in the band-pass filter block 201 and the level conversion block 202 described with reference to FIGS. Therefore, the DSP 25 constantly monitors seven types of signals for each microphone sound collection signal.
Based on the monitoring result, the DSP 25 proceeds to any one of the speaker direction detection processing, the speaker direction detection processing, and the speech start / end determination processing.

Step S2: Speech Start / End Determination Processing The DSP 25 determines the start and end of speech according to the method described in detail below with reference to FIG. When the processing of the DSP 25 detects the start of speech, the speech start detection is notified to the speaker direction determination processing in step 4.
In the speech start / end determination process in step 2, when the speech level becomes lower than the speech end level, a speech end determination time (for example, 0.5 second) timer is started and the speech end determination time and the speech level are set. When it is smaller than the speech end level, it is determined that the speech has ended.
If it becomes larger than the speech end level within the speech end determination time, it waits until it becomes smaller than the speech end level again.

Step S3: Speaker Direction Detection Processing The speaker direction detection processing in the DSP 25 is always performed by continuously searching for the speaker direction. Thereafter, the data is supplied to the speaker direction determination processing in step 4.

Step S4: Speaker direction microphone switching processing The timing determination processing in the speaker direction microphone switching processing in the DSP 25 has been selected from the result of the processing in step 2 and step 3 and the current speaker detection direction. If the speaker direction is different, the microphone selection in step 4 is instructed to select a microphone in a new speaker direction.
However, if the chairman's microphone is set from the operation unit 15 and the chairman's microphone and other meeting attendees speak at the same time, the chairman's comment is given priority.
At this time, the selected microphone information is displayed on the microphone selection result display means, for example, the light emitting diodes LED1 to LED6.

Step S5: Microphone sound collection signal transmission microphone signal switching processing uses only the microphone signal selected by the step 4 processing from among the six microphone signals as the transmission signal, for example, the first sound collection / video imaging device. In order to transmit from the first call device 10A of 1A to the second call device 10B of the second voice collecting / imaging device 1B on the other side via the communication line 920, the communication line 920 illustrated in FIG. Output to line-out.

Step S6: Determination of Imaging Conditions When the speaker can be determined by the above method, the imaging conditions by the TV camera devices 40A1 and 40A2 can be determined from the arrangement conditions of the plurality of microphones and the positions of the meeting attendees.
In addition, Preferably, the voiceprint recognition result of the meeting attendant described in 2nd Embodiment is used.
Details of this processing will be described as a third embodiment.

Processing for setting a speech start level threshold and a speech end threshold 1: Immediately after the power is turned on, the floor noise for a predetermined time, for example, 1 second is measured for each microphone.
The DSP 25 reads the peak-held level value of the sound pressure level detection unit at regular time intervals, for example, 10 mSec intervals in the present embodiment, and calculates an average value of values for a predetermined time, for example, 1 minute, to calculate floor noise. And
The DSP 25 determines a speech start detection level (floor noise +9 dB) and a speech end detection level threshold (floor noise +6 dB) based on the measured floor noise level. After that, the DSP 25 reads the peak-held level value of the sound pressure level detector at regular time intervals.
When it is determined that the speech has ended, the DSP 25 functions as a floor noise measurement, detects the start of speech, and updates the threshold for the detection level of speech end.

  According to this method, since the floor noise level at the position where the microphone is placed is different in this threshold setting, a threshold can be set for each microphone, and erroneous determination in selection of a microphone by a noise source can be prevented.

Process 2: Response to room with ambient noise (large floor noise) In Process 2, if the floor level is large and the threshold level is automatically updated in Process 1, the following measures are taken when it is difficult to detect the start and end of speech. .
The DSP 25 determines a threshold for the detection level of the speech start and the detection level of the speech end based on the predicted floor noise level.
The DSP 25 sets the speech start threshold level to be greater than the speech end threshold level (for example, a difference of 3 dB or more).
The DSP 25 reads the level value peak-held by the sound pressure level detector at regular time intervals.

  According to this method, since the threshold value is the same value for all microphones, it is possible to recognize the start of speech even if the person who is behind the noise source and the person who is not the same have the same loudness.

Talk start judgment
Process 1 The output level of the sound pressure level detector corresponding to the six microphones is compared with the threshold value of the speech start level.
When the output level of the sound pressure level detector corresponding to all the microphones exceeds the threshold of the speech start level, the DSP 25 determines that the signal is from the reception / reproduction speaker 16 and does not determine that the speech is started. This is because the distance between the reception / reproduction speaker 16 and all the microphones MC1 to MC6 is the same, so that the sound from the reception / reproduction speaker 16 reaches almost all the microphones MC1 to MC6.

Process 2 , two unidirectional microphones (with microphones MC1 and MC1) with the directional axes shifted by 180 degrees in the opposite direction at an equal angle of 60 degrees with respect to the six microphones illustrated in FIG. Three sets of MC4, microphones MC2 and MC5, and microphones MC3 and MC6) are used to make use of the level difference of the microphone signal. That is, the following calculation is performed.

[Table 6]
Absolute value of (the signal level of microphone 1−the signal level of microphone 4) [1]
Absolute value of (signal level of microphone 2−signal level of microphone 5) [2]
Absolute value of (signal level of microphone 3−signal level of microphone 6) [3]

The DSP 25 compares the absolute values [1], [2], and [3] with the threshold value of the speech start level, and determines that the speech is started when the threshold value of the speech start level is exceeded.
In the case of this process, since all the absolute values do not become larger than the threshold value of the speech start level as in process 1 (because the sound from the reception / reproduction speaker 16 reaches all the microphones equally), the reception / reproduction speaker 16 It is not necessary to determine whether the sound is from the speaker or from the speaker.

Speaker Direction Detection Processing For detecting the speaker direction, the characteristics of the unidirectional microphone illustrated in FIG. 7 are used. As illustrated in FIG. 7, the frequency characteristics and level characteristics of the unidirectional microphone change depending on the sound arrival angle from the speaker to the microphone. The results are illustrated in FIGS. 8 (A) to (C). 8A to 8C show a case where a speaker is placed at a predetermined distance from the communication device 10A, for example, a distance of 1.5 meters, and the sound collected by each microphone is subjected to fast Fourier transform (FFT) at regular time intervals. Results are shown. The X axis represents frequency, the Y axis represents signal level, and the Z axis represents time. The horizontal line represents the cut-off frequency of the band-pass filter, and the level of the frequency band sandwiched between the lines is the 5-band band pass from the microphone signal level conversion processing described with reference to FIGS. -It becomes the data converted into the sound pressure level that passed through the filter.

A determination method applied as an actual process for detecting the direction of the speaker in the communication device in the sound collection / video imaging device according to the embodiment of the present invention will be described.
Appropriate weighting processing is performed on the output level of each band-pass filter (0 for 1 dB full span (1 dBFs) step, 0 for 0 dBFs, 3 for -3 dBFs, or vice versa). This weighting step determines the processing resolution.
The above weighting process is executed for each sample clock, and the weighted score of each microphone is added and averaged with a fixed number of samples, and the microphone signal having a small (large) total score is determined as a microphone facing the speaker. To do. Table 7 below is an image of this result.

In this example illustrated in Table 7, the smallest total point is the first microphone MC1, so the DSP 25 determines that there is a sound source in the direction of the first microphone MC1 (there is a speaker). The DSP 25 holds the result in the form of a sound source direction microphone number.
As described above, the DSP 25 performs weighting on the output level of the band-pass filter of the frequency band for each microphone, and ranks the microphone signals with the lowest score (or higher) of the output of each band-pass filter. A microphone signal having the first rank in three or more bands is determined as a microphone facing the speaker. Then, the DSP 25 creates a score table as shown in Table 8 below, assuming that there is a sound source in the direction of the first microphone MC1 (there is a speaker).

  Actually, the performance of the first microphone MC1 is not necessarily the best in the output of all bandpass filters due to the reflection of sound and the influence of standing waves depending on the characteristics of the room, but the majority in the 5 bands If it is 1st place, it can be determined that there is a sound source in the direction of the first microphone MC1 (there is a speaker). The DSP 25 holds the result in the form of a sound source direction microphone number.

  The DSP 25 sums the output level data of each band band pass filter of each microphone in the form shown in Table 9 below, determines that the microphone signal having a high level is the microphone facing the speaker, and determines the result as the sound source direction microphone number. Hold in the form of.

[Table 9]
MIC1 Level = L1-1 + L1-2 + L1-3 + L1-4 + L1-5
MIC2 Level = L2-1 + L2-2 + L2-3 + L2-4 + L2-5
MIC3 Level = L3-1 + L3-2 + L3-3 + L3-4 + L3-5
MIC4 Level = L4-1 + L4-2 + L4-3 + L4-4 + L4-5
MIC5 Level = L5-1 + L5-2 + L5-3 + L5-4 + L5-5
MIC6 Level = L6-1 + L6-2 + L6-3 + L6-4 + L6-5

Talker direction microphone switching timing determination processing When activated by the speech start determination result in step 2 of FIG. 17, when a new speaker microphone is detected from the speaker direction detection processing result in step 3 and past selection information, The DSP 25 issues a microphone signal switching command to the microphone signal selection switching process in step 5, and notifies the microphone selection result display means (light emitting diodes LED1 to LED6) that the speaker microphone has been switched to the speaker. Informs that the speech device of the sound collecting / imaging device has responded to his speech.

In order to eliminate the influence of reflected sound and standing waves in a room with high reverberation, the DSP 25 prohibits the activation of a new microphone selection command unless the speech end determination time (for example, 0.5 seconds) elapses after the microphone is switched.
In the present embodiment, two microphone selection switching timings are prepared from the result of the microphone signal level conversion process in step 1 in FIG. 17 and the detection process result in the speaker direction in step 3.

First method : When it is possible to clearly determine the start of speech When speech from the direction of the selected microphone has ended and there is a new speech from another direction.
In this case, the DSP 25 starts speaking after all the microphone signal level (1) and the microphone signal level (2) are equal to or lower than the speech end threshold level and more than the speech end determination time (for example, 0.5 seconds). When any microphone signal level (1) is equal to or higher than the speech start threshold level, it is determined that speech has started, and a microphone facing the speaker direction is properly collected based on the information of the microphone number in the sound source direction. The microphone is determined, and the microphone signal selection switching process in step 5 is started.

Second method : When a new louder voice is spoken from another direction while speaking is in progress In this case, the DSP 25 stops speaking from the start of speaking (when the microphone signal level (1) exceeds the threshold level). The determination process starts after the determination time (for example, 0.5 seconds) has elapsed.
If it is determined that the sound source direction microphone number from the process 3 is changed and is stable before the end of the speech is detected, the DSP 25 is more than the speaker currently selected for the microphone corresponding to the sound source direction microphone number. It is determined that there is a speaker who is speaking loudly, the sound source direction microphone is determined as a valid sound collecting microphone, and the microphone signal selection switching process in step 5 is started.

The microphone signal selection switching process DSP 25 facing the detected speaker is activated by the command selected and determined by the command from the speaker direction microphone switching timing determination process in step 4 of FIG.
As shown in FIG. 18, the DSP 25 microphone signal selection switching process is composed of a 6-circuit multiplier and a 6-input adder. In order to select the microphone signal, the DSP 25 sets the channel gain (channel gain: CH Gain) of the multiplier to which the microphone signal to be selected is connected to [1] and the CH gains of the other multipliers to [0]. By doing so, the selected signal of (microphone signal × [1]) and the processing result of (microphone signal × [0]) are added to the adder, and a desired microphone selection signal is obtained at the output.

  When the channel gain is switched between [1] and [0] as described above, there is a possibility that a click sound is generated due to the level difference of the microphone signal to be switched. Therefore, in the communication device 10A, as illustrated in FIG. 19, the switching transition time, for example, 10 ms, is used to change the change in CH Gain from [1] to [0] and from [0] to [1]. In order to avoid the click sound caused by the difference in the level of the microphone signal, the signal is continuously changed over time.

  Further, by setting the maximum channel gain to other than [1], for example, [0.5], the echo cancellation processing operation in the DSP 25 at the subsequent stage can be adjusted.

  As described above, the communication device in the sound collection / video imaging device according to the first embodiment of the present invention is not affected by noise and can be effectively applied to a communication device such as a conference.

The communication device in the sound collection / video imaging device of the first embodiment of the present invention has the following advantages in terms of structure.
(1) The positional relationship between a plurality of microphones having a single directivity and a reception / reproduction speaker is constant, and furthermore, since the distance is very close, the sound emitted from the reception / reproduction speaker passes through the conference room (room) environment. The level that returns directly to the multiple microphones is overwhelmingly dominant. Therefore, the characteristics (signal level (intensity)) and frequency characteristics (f characteristics, phase) for sound to reach a plurality of microphones from the receiving / reproducing speaker are always the same. That is, there is an advantage that the transfer function is always the same in the communication device.

  (2) Therefore, there is no change in the transfer function when the microphone is switched, and there is an advantage that it is not necessary to adjust the gain of the microphone system every time the microphone is switched. In other words, there is an advantage that it is not necessary to redo once the adjustment is made at the time of manufacturing the communication device.

  (3) Even if the microphone is switched for the same reason as described above, only one echo canceller configured by a digital signal processor (DSP) may be used. The DSP is expensive, and the space for placing the DSP on a printed circuit board on which various members are mounted and there is little space may be small.

  (4) Since the transfer function between the receiving / reproducing speaker and the plurality of microphones is constant, there is an advantage that the sensitivity difference of the microphone itself having ± 3 dB can be adjusted by the unit alone.

  (5) The table on which the voice collecting / video imaging device communication device is mounted can be a speaker system that evenly distributes (spreads) sound of equal quality in all directions with a single receiving / reproducing speaker in the communication device. Became.

  (6) The sound emitted from the receiving / reproducing speaker is transmitted to the table surface (boundary effect), and the sound is effectively and evenly delivered to the conference attendees. The phase is canceled to produce a small sound, and there is an advantage that there is little reflected sound from the ceiling direction to the conference attendees, and as a result, a clear sound is distributed to the participants.

  (7) Since the sound emitted from the reception / reproduction speaker reaches all of the plurality of microphones at the same volume at the same time, it is easy to determine whether the sound is the speaker's voice or the reception voice. As a result, erroneous determination of microphone selection processing is reduced.

  (8) By arranging even number of microphones at equal intervals, level comparison for direction detection can be easily performed.

  (9) Due to a damper using a cushioning material, a microphone support member having flexibility or elasticity, vibration due to the sound of the reception and reproduction speaker that can be transmitted through the printed circuit board on which the microphone is mounted is collected by the microphone. The influence on can be reduced.

  (10) The sound of the receiving / reproducing speaker does not directly enter the microphone. Therefore, in this call device, there is little influence of noise from the receiving / reproducing speaker.

The communication device in the sound collection / video imaging device according to the first embodiment of the present invention has the following advantages from the viewpoint of signal processing.
(A) A plurality of unidirectional microphones are arranged radially at equal intervals so that the direction of the sound source can be detected, and the microphone signal is switched to collect (collect) sound with good S / N and clear sound. Can be sent to the other party.
(B) Sound from surrounding speakers can be collected with good S / N, and a microphone facing the speaker can be automatically selected.
(C) In the present invention, signal analysis is simplified by dividing a passing voice frequency band as a method of microphone selection processing and comparing levels for each divided frequency band.
(D) The microphone signal switching process according to the present invention is realized as a DSP signal process, and a plurality of signals are all cross-fade processed so as not to generate a clicking sound at the time of switching.
(E) The microphone selection result can be notified to a microphone selection result display means such as a light emitting diode or to the outside. Therefore, for example, it can be utilized as speaker position information for the conference system using the TV camera devices 40A1 and 40A2 illustrated in FIG.

Second Embodiment Referring to FIGS. 20 to 25, a second embodiment of the speech device of the sound collecting and video imaging apparatus of the present invention will be described.
Conventionally, there have been telephones, interphones, videophones and the like for transmitting conferences and individual voices to remote parties. However, in this case, the voices of the people around and the sound from the television set are noisy, and the voice of the speaker is often not transmitted well to the other party. For this reason, it is troublesome for the speaker to go close to the microphone, raise a loud voice, or lower the output sound of the television device each time.
If the communication device in the sound collecting and video imaging device of the first embodiment is used, noise around the communication device can be eliminated and the speaker can be identified accurately, but further improvement is desired.
In the second embodiment of the present invention, in order to further improve the communication device of the first embodiment, only voices of speakers who have previously registered voiceprints by performing voiceprint identification are clearly selected, and other noise and The sound becomes better communication by lowering the level.

FIG. 20 shows a device configuration of the communication device according to the second embodiment of the present invention.
The telephone apparatus illustrated in FIG. 20 has a configuration similar to that of the telephone apparatus illustrated in FIG. 6, and the components in the telephone apparatus illustrated in FIG. However, the following parts are different.
In the communication device according to the second embodiment, variable gain amplifiers 301 to 306 are arranged between microphones MC1 to MC6 and A / D converters 271 to 273, a voiceprint authentication unit 32 is added, and amplifier gain adjustment is performed. In addition to the output from the amplifier 291 to the LINE OUT terminal, an output signal is applied from the amplifier 291 to the voiceprint authentication unit 32. As described in the first embodiment, the variable gain amplifiers 301 to 306 may be configured such that the A / D converters 271 to 273 are the amplification function type A / D converters 271 to 273 with gain adjustment. In this case, the functions of the variable gain amplifiers 301 to 306 can be included in the A / D converters 271 to 273. In this embodiment, a case where variable gain amplifiers 301 to 306 are provided separately from A / D converters 271 to 273 will be described.
In the second embodiment, a third amplifier 293 is added so that an input signal from LINE IN or a signal from the amplifier 293 can be output to the recording output terminal REC OUT.

The six microphones MC1 to MC6 have the directivity illustrated in FIG. 7 and are arranged at equal angles and at equal intervals as described with reference to FIGS.
The A / D converters 271 to 273 are also two-channel A / D converters in the second embodiment, and two input signals (two-channel input signals) can be captured by one A / D converter.
The DSP 25 performs various processes listed in FIG. 10 described in the first embodiment, such as a microphone selection / switching process.
As described in the first embodiment, the second digital signal processor (DSP) 26 performs echo cancellation processing.

  The voiceprint authentication unit 32 includes a voiceprint authentication processor P that performs voiceprint authentication processing, a dictionary memory M1 for voiceprint processing, and a voiceprint registration memory M2 that registers voiceprints. In the voiceprint registration memory M2, a voiceprint of a person who performs speaker authentication in advance by the voiceprint registration device 32A is registered. The target person for speaker authentication is a meeting attendee who uses the communication device of the present embodiment. Details of the processing of the voiceprint authentication unit 32 will be described later.

As in the first embodiment, the DSP 25 selects one of the microphones MC1 to MC6 and outputs a microphone selection signal S251 indicating the number of the selected microphone to the microprocessor 23. The microprocessor 23 outputs the microphone selection signal S251 to the amplifier gain adjustment unit 34.
The microphone signal selected by the DSP 25 is applied to the DSP 26, echo canceled at the DSP 26, output to the D / A converter 282, amplified by the amplifier 292, and output from the reception / reproduction speaker 16. The conference attendee in use can listen to the voice of the speaker using the selected microphone from the reception reproduction speaker 16.

The selected voice signal S26 output from the DSP 26 to the D / A converter 282 is output to the LINE OUT terminal via the amplifier 291 and can be sent to the partner telephone apparatus.
Since the selected audio signal S26 output from the DSP 26 to the D / A converter 282 is output to the REC OUT terminal via the amplifier 293, it can also be recorded.
Furthermore, since the selected voice signal S26 output from the DSP 26 to the D / A converter 282 is output to the voiceprint authentication unit 32 via the amplifier 291, the voiceprint authentication unit 32 performs voiceprint authentication on the selected voice signal S26. Although the details of the voiceprint authentication will be described later, when the voiceprint authentication unit 32 is registered in the voiceprint registration memory M2 as a result of voiceprint authentication of the selected voice signal S26, an authentication pass signal S32 (“1” is passed when the authentication is passed). “0”) is output to the amplifier gain adjustment unit 34 when the authentication fails.

  A microphone selection signal S251 is input from the DSP 25 to the amplifier gain adjustment unit 34 via the microprocessor 23. In this state, when an authentication pass signal S32 indicating authentication pass is input from the voiceprint authentication unit 32 to the amplifier gain adjustment unit 34, the amplifier gain adjustment unit 34 receives the output signal of the microphone indicated by the microphone selection signal S251. Increase the gain of the corresponding variable gain amplifier (maintain that value if it is already set to a large value, or set it to a large value), and decrease the gain of the other variable gain amplifier ( If it is already set low, keep it at that value, or set it to a low value).

  Specifically, the amplifier gain adjustment unit 34 has a built-in microcomputer, and the microcomputer in the amplifier gain adjustment unit 34 has a corresponding gain to which the output signal of the microphone indicated by the microphone selection signal S251 is input. The gain setting values of the variable amplifiers are set to large values and output to the variable gain amplifiers, and the gain setting values of the other variable gain amplifiers are set to low values and output to these variable gain amplifiers. As a result, the variable gain amplifiers 301 to 306 are changed to the set gain.

For example, when the first microphone MC1 collects only the sound from the television device, if the sound is loud, it is selected by the DSP 25. As a result, the DSP 25 outputs a microphone selection signal S251 indicating that the first microphone MC1 has been selected to the amplifier gain adjustment unit 34 via the microprocessor 23.
A sound signal from the television device selected by the DSP 25 is input from the DSP 26 to the voiceprint authentication unit 32 via the amplifier 291 as the selected sound signal S26. Since the sound of the television apparatus is not registered in the voiceprint registration memory M2 of the voiceprint authentication unit 32, the selected voice signal S26 is rejected for authentication, and the authentication pass signal S32 of “0” is sent to the amplifier gain adjustment unit 34. Is output.
The amplifier gain adjustment unit 34 has already received the microphone selection signal S251 indicating that the first microphone MC1 has been selected. However, since the authentication pass signal S32 of “0” is input, the amplifier gain adjustment unit 34 The gain of the variable gain amplifier 301 to which the output signal of the first microphone MC1 indicated by the microphone selection signal S251 is connected is set low and output to the variable gain amplifier 301. The gain of the variable gain amplifier 301 is Lower. As a result, the collected sound signal of the first microphone MC1 is lowered by the variable gain amplifier 301 and is input to the A / D converter 271, so that there is a high possibility that it will be excluded from the microphone selection target.

On the other hand, when a voiceprint of a speaker who uses the third microphone MC3 is registered in advance in the voiceprint registration memory M2 of the voiceprint authentication unit 32 and the third microphone MC3 is selected by the DSP 25, the DSP 25 passes through the microprocessor 23. Then, the microphone selection signal S251 indicating that the third microphone MC3 has been selected is output to the amplifier gain adjustment unit 34, and the voice of the third microphone MC3 is input to the voiceprint authentication unit 32 as the selected voice signal S26 for voiceprint authentication. The In this case, since the voiceprint is registered in the voiceprint registration memory M2, the authentication is passed and an authentication pass signal S32 of “1” is output.
When the authentication pass signal S32 of “1” is input, the amplifier gain adjustment unit 34 is connected to the output signal of the third microphone MC3 with reference to the microphone selection signal S251 indicating that the third microphone MC3 has been selected. The gain of the variable gain amplifier 305 is set high and output to the variable gain amplifier 305, and the gain of the variable gain amplifier 305 is set to a certain high value. As a result, the collected sound signal of the third microphone MC3 is increased by the variable gain amplifier 305 and input to the A / D converter 273, and a high sound output is output from the DSP 26 as the selected sound signal S26. The selected audio signal S26 is, of course, converted to an analog signal by the D / A converter 282, amplified by the amplifier 292, output to the reception / reproduction speaker 16, amplified by the amplifier 291 and via the LINE OUT. Are input to the voiceprint authentication unit 32 again and are subject to voiceprint authentication.

When the sound from the television apparatus collected by the first microphone MC1 and the sound from the third microphone MC3 exist at the same time, the DSP 25 first selects the higher sound and authenticates the voice print as the selected sound signal S26. Input to the unit 32.
For example, when the sound of the television apparatus collected by the first microphone MC1 is higher than the sound from the third microphone MC3, the sound of the television apparatus from the first microphone MC1 is selected by the DSP 25 and is selected from the DSP 26 as the selected sound signal S26. If it is output, the voiceprint authentication unit 32 does not authenticate as described above. Therefore, as described above, the gain of the variable gain amplifier 301 to which the output signal of the first microphone MC1 is connected is lowered. As a result, in the next microphone selection process in the DSP 25, the sound collection signal of the first microphone MC1 is not selected, and the sound collection signal of the third microphone MC3 is selected. When the collected sound signal of the third microphone MC3 is output from the DSP 26 to the voiceprint authentication unit 32 as the selected voice signal S26, the voiceprint authentication process is passed. As a result, the gain of the variable gain amplifier 305 to which the third microphone MC3 is connected is set to a high value by the amplifier gain adjusting unit 34, and the collected sound signal of the third microphone MC3 becomes high, so that the voice is received as clear sound. It is output from the reproduction speaker 16, output from LINE OUT, and input to the voiceprint authentication unit 32 again.

In this way, the voice spoken by the voiceprint speaker registered in the voiceprint registration memory M2 of the voiceprint authentication unit 32 is finally selected, and as a clear signal, the voiceprint authentication unit 32 is sent from the reception reproduction speaker 16 to LINE OUT. Is output.
Therefore, if the communication device of the second embodiment is used, as illustrated in FIG. 1, a clear voice conversation can be easily performed with a person at a distance.
Further, even when using a communication device in a noise environment such as the sound of a television device as ambient noise, it is not necessary to move the speaking position of the speaker or to make a loud voice.
Furthermore, it is possible to talk with the other party without bothering to lower the sound level of the television device as noise each time. In particular, since the sound of the television apparatus as noise is kept low, the other party can only hear a clear conversation sound, and the conversation is performed smoothly. In that sense, the communication device of the second embodiment also has a function as a device for removing unnecessary noise.
Of course, even if a person who is not registered in the voiceprint registration memory M2 of the voiceprint authentication unit 32 speaks around the telephone device, such voice is not finally selected, and the voice of the speaker who is registered as a voiceprint. Only clearly and selectively output.

The end of the selected microphone is determined by the DSP 25 when the level of the microphone output signal decreases and continues for a predetermined time, as illustrated in FIG.
At this time, the amplifier gain adjustment unit 34 preferably resets the gain of the variable gain amplifier corresponding to the microphone whose speech has ended to a normal gain. Of course, it is possible to notify the amplifier gain adjustment unit 34 that the selection is completed from the DSP 25 via the microprocessor 23 by including it in the microphone selection signal S251.
In this way, by setting the gain of the variable gain amplifier corresponding to the selected microphone to the same gain as that of the other variable gain amplifiers, the next microphone selection becomes an equal condition.

  In the above embodiment, the case where the variable gain amplifiers 301 to 306 are used as the variable gain amplification means of the present invention has been described. However, as described above, the variable gain can be used as the A / D converters 271 to 273. Type A / D converters 271 to 273 can also be used. In this case, the variable gain amplifiers 301 to 306 are replaced with fixed gain amplifiers, and the amplifier gain adjustment unit 34 includes variable gain A / D converters 271 to 273. Can be adjusted (set).

  As a preferred example of the communication device of the present invention, the case where the microphones MC1 to MC6 described as the first embodiment are radially arranged at an equal angle has been described. However, as the second embodiment, the microphones MC1 to MC1 are described. The MC 6 is not limited to the case where each pair of microphones, for example, MC 1 and MC 4 are arranged on a straight line as in the first embodiment, but may be a predetermined arrangement. In this case, the DSP 25 selects, for example, a microphone that outputs a sound collection signal having the maximum amplitude as the microphone selection signal S251. Thereafter, the voiceprint authentication unit 32 performs the above-described voiceprint authentication.

A detailed example of processing contents of the voiceprint authentication unit 32 will be described with reference to FIGS.
In the present embodiment, the attendees of each conference input voices sequentially from the microphones MC1 to MC6 to the voiceprint registration device 32A, and output the voiceprint registration unit 32A together with the microphone number to the voiceprint authentication unit 32. In this example, the voice of each conference attendant is assumed to be a command with a voice of about 2 to 3 seconds, such as “Open File” and “Next”, as illustrated in FIG.
The voiceprint authentication processor P in the voiceprint authentication unit 32 converts the voice signal input from the voiceprint registration device 32A into a digital signal, and then performs a voice recognition process with reference to the dictionary recorded in the dictionary memory M1, thereby Converted into column data and recorded in the voiceprint registration memory M2 together with the microphone number. That is, the voiceprint authentication processor P collates the character string data corresponding to the voice command in the dictionary memory M1 in which the character string data corresponding to the voice command input in advance is stored, and selects a matching one.

FIGS. 21A to 21D are timing charts illustrating the control operation performed by the voiceprint authentication unit 32.
FIG. 21A is a timing chart of the microphone switching signal MC_SEL. When, for example, # 4 is described, it indicates that the fourth microphone MC4 is currently selected.
FIG. 21B is a timing chart of the microphone output signal. The microphone output signal is an audio signal corresponding to the microphone number indicated by the microphone switching signal MC_SEL in FIG. 21A, and is converted into a digital signal by an A / D converter in the voiceprint authentication processor P and input. In this example, it is a voice signal of a command such as a microphone output signal “OpenFile” or “Next”.
FIG. 21C is a timing chart showing a process performed by the voiceprint authentication processor P based on the information obtained in FIGS. It consists of buffering of each voice data and voice recognition processing after buffering.
FIG. 21D is a timing chart of character string data sequentially output as a result of the voice recognition process shown in FIG.

  As illustrated in FIG. 21A, the number of the first selected microphone is # 4, and the microphone output signal “Open File” is input to the voiceprint authentication processor P from the fourth microphone. The voiceprint authentication processor P inputs the microphone output signal digitally converted via the A / D converter, starts buffering as illustrated in FIG. 21C, and the voice data is the microphone number # 4 of the buffer. Is held in a buffer according to

Thereafter, when the microphone number changes from # 4 to # 1, the microphone switching signal MC_SEL = 1. As shown in FIG. 21B, the voice data of the microphone number # 1 is the voice data corresponding to “Next”, and the voiceprint authentication processing processor P ends the buffering of the microphone number # 4 and starts a new microphone number #. 1 buffering is started, and voice recognition processing processor P performs voice recognition processing in parallel based on the voice data of microphone number # 4 held in the buffer.
In the speech recognition process, the speech data of the microphone number # 4 is subjected to speech recognition processing, collated with the command group of the character string data stored in the dictionary memory M1, and the matching data is selected, and “Open” as the character string data File "is output as shown in FIG.
Thereafter, the same applies even if the microphone number changes from # 1 to # 2.
The control operation outlined above will be further described with reference to the flowchart.

FIG. 22 is a diagram showing a main flow of control performed by the voiceprint authentication processor P.
For example, a T1 timer of 2 kHz starts and shifts to a T1 timer interrupt shown in FIG. 23 every 50 μs. If there is a voice input of a certain level or higher (step ST11), the process proceeds to step ST12. It goes without saying that this constant level threshold value can be appropriately set according to the application.
Since the voice print authentication processor P is supplied with the microphone switching signal MC_SEL, if there is a voice input at a certain level or higher in step ST11, the voice print authentication processor P knows the voice microphone number (1 to 6). Accordingly, in step ST12, sampling of the input voice data is started, and the voice data is held in a buffer corresponding to the microphone number (1 to 6) of the voice.
If there is no voice input above a certain level, nothing is done in step ST12.

FIG. 25 is a diagram showing an interrupt flow when the microphone selection information is changed in the control of the main flow shown in FIG. That is, in the main flow that is a normal control operation, the interrupt number is generated when the microphone number selected by the call device changes and the information is notified through the microphone switching signal MC_SEL. For example, when the voice data of microphone number 4 (microphone switching signal MC_SEL = 4) is sampled and stored in the buffer of microphone number 4 before this interruption, the microphone switching signal MC_SEL changes from 4 to 1. is there.
In step ST40 of FIG. 25, when the voiceprint authentication processor P is performing audio sampling, no more audio data is stored in the buffer.
In this case, it is considered that the speech input from the currently performed microphone number 4 is completed, and the sampling is terminated (step ST41).
Furthermore, the voice data of microphone number 4 for which sampling has been completed is subjected to voice recognition processing in the voiceprint authentication processor P (step ST42). In the example of FIG. 21, in the voiceprint authentication processor P, the voice data of the microphone number 4 is recognized as “Open File”, and the character string data is output to the outside of the call device 1A.

In step ST10 of FIG. 22, the T1 timer is started, and for example, the T1 timer interrupt flow shown in FIG. 22 is started every 50 μs (20 kHz). In the T1 timer interruption, it is monitored whether there is an audio input every 5 μs and whether there is an audio input of a certain level or more, and appropriate measures are taken. First, it is checked in step ST20 whether audio sampling has been performed.
If voice sampling has been performed, the voiceprint authentication processor P further checks whether there is a certain level of voice input (step ST21), and if there is a certain level of voice input, the T2 timer described later stops. . The T2 timer is used to monitor a state where there is no utterance, and when there is no utterance for a certain period of time, the T2 timer automatically shifts to the next phase, speech recognition.
If the utterance, that is, the voice input is above a certain level, it is considered that the utterance is continuing, and the T2 timer is reset in step ST22.
Further, although voice sampling is performed in step ST20, if there is no voice input of a certain level or more, the current utterance may be terminated. Therefore, in order to monitor the duration time when there is no utterance, T2 A timer is started (step ST23).
Even if there is no voice input of a certain level or higher in step ST21, speech sampling is continued because there is a possibility that speech is resumed (step ST24).

If voice sampling is not performed in step ST20, the voiceprint authentication processor P checks in step ST25 whether there is a voice input of a certain level or higher. Thereby, it is checked whether or not the utterance has started, and if there is an audio input of a certain level or more, the voiceprint authentication processor P assumes that the utterance has started, and the audio is stored in the buffer corresponding to the newly selected microphone. Sampling is started (step ST26).
If there is no voice input exceeding a certain level in step ST25, the voiceprint authentication processor P does nothing and waits for the next valid utterance.

In step ST23 of FIG. 23, for example, when a T2 timer of 2 Hz is started and a certain period of time has elapsed, that is, the voiceprint authentication processor P is performing voice sampling (step ST20), If no case continues for a certain period of time, it is useless to continue the audio sampling, and the process proceeds to the T2 timer interrupt flow shown in FIG.
That is, the voice sampling performed at that time is terminated (step ST30), and the process proceeds to the voice recognition process (step ST31).
After shifting to the voice recognition process, in step ST32, the T2 timer is reset for the next utterance process.

According to the voiceprint authentication unit 32, even when a plurality of people overlap each other and issue a voice command to the call device through the microphones used by each of the plurality of conference attendees, the sound pressure level for each voice band To identify the main speaker and deliver the speech signal. Therefore, the voiceprint authentication unit 32 can avoid the possibility of erroneous recognition processing even when a plurality of voice commands are input at the same time, and appropriately determines and processes a voice command mainly spoken. It is possible.
The voiceprint authentication processor P of the voiceprint authentication unit 32 buffers the delivered voice command signal, performs voice recognition processing on the buffered voice signal, compares it with the command character string data stored in the dictionary memory M1, and matches. The character string data to be selected is selected and processed.
The voiceprint authentication processor P of the voiceprint authentication unit 32 is sequentially notified of the selected microphone number from the voiceprint registration device 32A. Therefore, when the selected microphone number is switched, the buffering is stopped, the voice signal that has been buffered up to that point is subjected to voice recognition processing, and the voice command signal from the updated microphone number is buffered. Since it starts, the accuracy of voice recognition is improved.

Third Embodiment With reference to FIGS. 2 and 26 to 31, a third embodiment of the sound collection / video imaging apparatus of the present invention will be described.
In the third embodiment of the present invention, a case will be described in which a television conference (TV conference) system is configured by using the above-described communication devices and adding imaging means to them.
FIG. 2 shows an initial state of the television camera devices 40A1 and 40A2 of the sound collection / video imaging device, and FIG. 31 shows a state in which the television camera devices 40A1 and 40A2 take an image based on determination of imaging conditions by the communication device and the imaging adjustment unit 36. FIG.

In the conventional conference system with a camera, the direction of the camera is controlled by the microphone number of each speaker or the control (chairman) of the TV conference system. In such a method, an individual microphone is required for each speaker, so an expensive system is necessary, or a video conference system administrator changes the imaging area every time the speaker changes. There was a hassle of having to control change.
In addition, as for the name display of the speaker, the microphone and the speaker name are usually linked to each other, and if the seat where the participant sits is changed halfway, resetting is necessary and the procedure is complicated.
There are simple systems that simply point the camera in the direction in which the sound is coming out, but the camera faces the direction of the person who is not suitable for imaging, or is used for ambient noise, such as a meeting. In response to the sound of the fan of the projector device, there is a problem that the imaging direction of the camera is directed toward the projector device.

Using the above-described voice collecting / imaging device communication device has various advantages such as accurate selection of a speaker and no need to install a microphone in the vicinity of a conference attendee. Can improve.
That is, the function illustrated in FIG. 5 is that a plurality of microphones MC1 to MC6 are arranged in all directions, and the first digital signal processor (DSP) 25 selects a microphone sound collection signal in a direction mainly speaking at present. 6 can be used to accurately select the speaker's microphone. For example, since the microphones are arranged at equal angles, if the microphone can be selected in the DSP 25, for example, the arrangement direction of the microphone can be determined in the DSP 25, and further, the direction of the speaker can be specified in the DSP 25.
More preferably, the authentication pass signal S32 output from the voiceprint authentication unit 32 by the call device in which the voiceprint authentication unit 32 is added to the call device illustrated in FIG. 6 described as the second embodiment with reference to FIG. If the microphone selection signal S251 output from the DSP 25 to the microprocessor 23 is used, the speaker can be accurately identified.
As illustrated in FIGS. 1B, 2, and 31, since the speaker is sitting in front of the corresponding microphone, the position of the speaker corresponding to the position of each microphone is registered in the DSP 25 in advance. deep. Further, the television camera devices 40A1 and 40A2 and the positions and directions of the speakers are registered in the DSP 25.
By using the direction and position of the above speakers, it is possible to determine the imaging area of the speaker to be captured by each of the television camera devices 40A1 and 40A2.

Therefore, in the third embodiment, as illustrated in FIGS. 26 and 27, the television camera devices 40A1 and 40A2 (typically, the television camera device 40) as the imaging means of the present invention, and the television camera device An imaging adjustment unit 36 serving as an imaging adjustment unit that adjusts 40 imaging conditions is added to the communication device described with reference to FIG.
FIG. 26 and FIG. 27 are configuration diagrams of an audio sound collection / video imaging apparatus as a third embodiment of the present invention. FIG. 27 is a configuration diagram of an audio sound collection / video imaging device in which the imaging adjustment unit 36 and the television camera device 40 (television camera devices 40A1 and 40A2) are added to the call device illustrated in FIG. FIG. 28 is a block diagram of a sound collection / video imaging apparatus in which the variable gain amplifiers 301 to 306 and the amplifier gain adjustment unit 34 are deleted from the voice collection / video imaging apparatus illustrated in FIG.

(1) As the third embodiment of the present invention, the microphone selection process by the DSP 25 described as the first embodiment is indispensable, and the imaging adjustment unit 36 of the television camera device 40 is based on the result of the microphone selection process in the DSP 25. Control imaging conditions.
(2) As a preferable mode of the third embodiment of the present invention, the configuration illustrated in FIG. 26 is described as the first embodiment like the second embodiment described with reference to FIG. In addition to the microphone selection process by the DSP 25, voice print authentication is performed in the voice print authentication unit 32, and only when the microphone selection process result and the voice print authentication match, the imaging adjustment unit 36 performs the television camera device 40 (TV camera device 40A1). , 40A2) is controlled.
(3) As a more preferable form of the third embodiment of the present invention, the structure illustrated in FIG. 27 is described as the first embodiment like the second embodiment described with reference to FIG. In addition to the microphone selection process performed by the DSP 25, voice print authentication is performed by the voice print authentication unit 32, and only when the microphone selection process result matches the voice print authentication, the imaging adjustment unit 36 performs the TV camera device 40 (TV camera device). 40A1 and 40A2) are controlled, and the gain control of the variable gain amplifiers 301 to 306 by the amplifier gain adjustment unit 34 described in the second embodiment is also performed.
The basic items of the third embodiment will be described below with reference to FIGS.

The imaging adjustment unit 36 has a built-in computer, and as illustrated in FIG. 2 and FIG. 31, up and down and left and right directions (up and down and left and right or tilt), pan, zoom, illumination conditions, etc. Can be adjusted.
In addition, the imaging adjustment unit 36, for each TV camera device 40A1, 40A2, in advance, for example, the first imaging condition information for imaging the direction and area MIC1AREA of the first microphone, the direction and area of the second microphone Second imaging condition information for imaging MIC2AREA is set in the memory portion of the computer. Preferably, the imaging condition information may include the name, title or title of the attendee of the meeting.
In the example illustrated in FIG. 2, as an initial state, the imaging adjustment unit 36 has the television camera devices 40A1 and 40A2 share the right and left of the conference room with the call device 10A in the conference room as the center, and the attendees as well. Everyone can be imaged.

  Each of the television camera devices 40A1 and 40A2 has an imaging condition given from the imaging adjustment unit 36, for example, an imaging condition (up and down, left and right direction), whether to zoom, and how much to zoom when zooming. If so, the image can be captured under the imaging conditions. The image signals captured by the TV camera devices 40A1 and 40A2 are displayed on the projector device 60A (or the television receiver 50A), and the projector device 60B (or the television receiver 50B) of the remote sound collection / video imaging device. ) Is displayed.

The amplifier gain adjustment unit 34 and the imaging adjustment unit 36 input a microphone selection signal S251 indicating the number of the microphone selected by the DSP 25 via the microprocessor 23.
The amplifier gain adjustment unit 34 and the image pickup adjustment unit 36 have a voice print that has been pre-registered after the voice print authentication is performed by the voice print authentication unit 32 on the selected voice signal S26 output by the echo cancellation processing of the collected sound signal selected by the DSP 25 by the DSP 26. The authentication pass signal S32 output as “1” is input.

  The amplifier gain adjustment unit 34 sets the gain of the variable gain amplifier corresponding to the microphone indicated by the microphone selection signal S251 to a large first gain by the method described in the second embodiment. The result is the same as that described in the second embodiment.

The imaging adjustment unit 36 reads imaging condition information set in advance in the imaging adjustment unit 36 corresponding to the microphone indicated by the microphone selection signal S251 from the memory, and based on the imaging condition information, the TV camera device 40A1. , 40A2 imaging conditions are adjusted.
For example, when the microphone selection signal S251 indicates the first microphone, based on the first imaging condition information for imaging the direction of the first microphone and the region MIC1 AREA (FIG. 26, for example, the left direction in FIG. 2). Thus, the direction or direction (up and down, left and right) of each of the television camera devices 40A1 and 40A2 is controlled so as to image the direction of the first microphone and the area MIC1AREA. When the first imaging condition information includes zoom information, the imaging adjustment unit 36 further instructs the television camera devices 40A1 and 40A2 to perform zoom processing.
The TV camera devices 40A1 and 40A2 perform image capturing under the conditions instructed by the image capturing adjustment unit 36, and send the result to the projector device of the remote sound collecting / video image capturing device using a line (not shown). In addition, the imaging results of the television camera devices 40A1 and 40A2 can be displayed on the projector device of the sound collection / video imaging device.
As described above, a microphone selected by the DSP 25 and further subjected to voice print authentication by the voice print authentication unit 32 is selected as a projector device serving as a monitor device in a room where the voice collecting / imaging device of the remote party is installed. The video of the attendees who used to speak is selected and displayed.

  The imaging adjustment unit 36 can superimpose information such as name and title included in the imaging condition information on the video signals captured by the television camera devices 40A1 and 40A2. As a result, the projector device as the indoor monitor device in which the communication device is installed, and the remote counterpart projector device, not only the images captured by the TV camera devices 40A1 and 40A2, but also information such as name and title Are superimposed and displayed.

Mode of Operation The mode of operation of the sound collection / video imaging apparatus of the third embodiment will be described with reference to FIGS.
1. As an initial state, the imaging adjustment unit 36 sets the TV camera devices 40A1 and 40A2 to a wide angle as illustrated in FIG.
2. FIG. 28, Step S51: When the conference starts and there is a speaker, the call device detects the voice of the speaker by the method described above.
3. Steps S52 to 53: Preferably, the voiceprint authentication unit 32 of the call device extracts the voiceprint of the speaker and performs voiceprint recognition processing. If the voiceprint is not registered in the voiceprint registration device 32A, the process proceeds to step S60.
4). Steps S60 to S64: Processing for a new voiceprint is performed. Details of this processing will be described later.
5). Step S54: The voiceprint authentication unit 32 checks whether it is the same voiceprint as the previous time or whether the microphone that detected the sound is the same as the previous time, and if the same voiceprint as the previous time or the same microphone as the previous time is selected, the step The process returns to S51.
If a voiceprint different from the previous time or a microphone different from the previous time is selected, the process proceeds to step S55.
6). Steps S55 to 59: Steps S60 to 64:
Before describing these processes, the processes of subroutine 1 shown in FIG. 29 and subroutine 2 shown in FIG. 30 will be described.

FIG. 29, subroutine 1
Step S70: When the sound collection / video imaging device is installed, the coordinate positions of the microphones of the communication device and the TV camera devices 40A1 and 40A2 are input to the imaging adjustment unit 36. These pieces of information specify the direction (microphone position) of the speaker's sound in the communication device. For example, when the speaker is further specified by voiceprint recognition, the direction and distance of the speaker from each TV camera device 40A1, 40A2. Is the information for calculating.

Step S71: The imaging adjustment unit 36 obtains sound source direction detection data calculated from the two selected microphones as a result of the DSP 25.
In the embodiment described above, it is detected which microphone the speaker is near, and the TV camera devices 40A1 and 40A2 suitable for photographing the vicinity of the microphone are selected.

In the first embodiment, the case of selecting the microphone that detects the highest sound using a pair of microphones arranged opposite to each other has been described as a preferred embodiment. In the embodiment, as illustrated in FIGS. 2 and 31, for example, it is assumed that there are eight conference attendees for six microphones.
In such a case, since the number of microphones and the number of conference attendees do not correspond one-to-one, there are conference attendees located between two adjacent microphones. In such a case, instead of selecting only one microphone as in the first embodiment, the first microphone that detects the highest sound and the second microphone that detects the next highest sound are selected, and these are selected. The direction of the sound source is detected from the two microphones. Therefore, the sound source direction data can be defined from the directions (arrangement, first arrangement condition) of two adjacent microphones.

  The relationship between the specification of the sound source direction and the shooting conditions of the TV camera devices 40A1 and 40A2 is, for example, as illustrated in FIG. 31, with the TV camera device 40A2 that can capture the front of the face of the conference participant A1 being the conference participant A1. The other TV camera device 40A1 shoots the entire right side of the conference room, the chairperson (for example, conference attendant A4), or all conference attendees.

  Step S72: The imaging adjustment unit 36 checks whether or not there is a change in the sound source direction detection data. If there is no change, the process returns to step S71, and if there is a change, the process proceeds to step S73.

Step S73: The imaging adjustment unit 36 calculates an intersection point from the directions (directions) of two adjacent microphones. Note that the data set in step S70 is used as data used to calculate the intersection.
Thereby, the position of the speaker can be estimated from the center of the communication device 10A.

Step S74: The imaging adjustment unit 36 calculates the distance, vertical and horizontal directions (or vertical and horizontal directions) of each of the television camera devices 40A1 and 40A2 to the calculated intersection. Note that the data set in step S70 is used as data used to calculate the distance and direction.
Steps S75 and 76: The imaging adjustment unit 36 pans the television camera devices 40A1 and 40A2 in the calculated direction (direction). Thereafter, the processing of the imaging adjustment unit 36 returns to the called step of FIG.

FIG. 30, subroutine 2
Steps S80 and 81: The results of the subroutine 2 of the main routine illustrated in FIG. 28 (imaging results of the television camera devices 40A1 and 40A2) are viewed. As a result, if there is no output, the process returns to step S80, and if there is an output, the process proceeds to step S82.

Steps S82 to 84: The imaging adjustment unit 36 searches for the outline of the imaging results (images) of the TV camera devices 40A1 and 40A2, that is, the outline of the conference attendee (step S82), and the outline fills the frame (frame) of the image. Thus, zoom control is performed on the television camera devices 40A1 and 40A2. As described above, for example, as illustrated in FIG. 31, when the conference attendee A1 is photographed, the face of the conference attendant A1 is photographed by the TV camera device 40A2 that can capture the front face of the conference attendee A1. Perform zoom processing. After the zoom process, the process returns to the next step S of the called main routine.
That is, from the imaging results of the TV camera devices 40A1 and 40A2, the imaging adjustment unit 36 recognizes the state of the speaker speaking, and the TV camera devices 40A1 and 40A1 are configured so that the outline of the speaker's face is at the center of the image frame. The zoom is performed by changing the direction of 40A2 by pan and tilt. At the same time, the voiceprint of the speaker is registered.

  At this time, if the image capturing adjustment unit 36 recognizes two or more images, the fact is displayed on the projector device 60A as a monitor device. For example, since all recognized faces are displayed, the speaker selects which one of them is his / herself, and if necessary, manually performs pan, tilt and zoom operations. To enter the image frame.

Steps S55 to 59: Steps S60 to 64:
These processes will be described with reference to the processes of the subroutine 1 shown in FIG. 29 and the subroutine 2 shown in FIG.
Steps S55 to 56, 60 to 61: The sound source direction detection data is passed to the subroutine 1, and the corresponding one of the TV camera devices 40A1 and 40A2 is panned.
Steps S57 to 58, 62 to 63: Subroutine 2 processing for performing image recognition processing is performed.
Steps S59 and S64: The voiceprint data by the voiceprint authentication unit 32 and the pan, tilt, and zoom data of the television camera devices 40A1 and 40A2 are stored as a pair in, for example, the database of the imaging adjustment unit 36 and used for the next processing. .
That is, a speaker's voiceprint and camera pan, tilt, and zoom data for clearly displaying the speaker are made to correspond one-to-one and registered as data. As a result, even if the speaker changes thereafter, the camera's pan, tilt and zoom operations are automatically performed to clearly show the speaker by comparing the speaker's voiceprint with the registered data.

  If the microphone is not selected properly, or if the microphone is selected but voiceprint authentication is not passed, the imaging adjustment unit 36 performs default processing. As such default processing, the imaging adjustment unit 36 sets the imaging conditions for imaging in the initial state illustrated in FIG. 2, that is, the television camera devices 40A1 and 40A2 share the right and left of the conference room. Give to 40A2. As a result, the television camera devices 40A1 and 40A2 capture an initial image.

  At the default, the amplifier gain adjusting unit 34 does not adjust the gain of the variable gain amplifiers 301 to 306.

An example of shooting a speaker will be described below.
Suppose that the meeting attendee A1 in the first microphone direction and area MIC1 AREA speaks using the first microphone MC1. The collected sound signal of the first microphone MC1 is converted into a digital signal by the A / D converter 271 and input to the DSP 25, and is selected by the method described in the first embodiment. At this time, the DSP 25 outputs to the microprocessor 23 a microphone selection signal S251 indicating that the first microphone MC1 has been selected. The microphone selection signal S251 is output from the microprocessor 23 to the imaging adjustment unit 36.
The collected sound signal of the first microphone selected by the DSP 25 is output to the DSP 26, echo-cancelled by the DSP 26, and input to the voiceprint authentication unit 32 via the D / A converter 282 and the amplifier 291 as the selected sound signal S26. Is done.
The voiceprint authentication unit 32 authenticates whether the selected voice signal S26 matches a voiceprint registered in advance in the voiceprint registration memory M2 in the voiceprint authentication unit 32. If the voice print of the attendee A1 has been registered in advance in the voice print registration memory M2 of the voice print authentication unit 32, an authentication pass signal S32 of “1” indicating acceptance from the voice print authentication unit 32 and the imaging adjustment with the amplifier gain adjustment unit 34 Is output to the unit 36.
On the other hand, if the voice print of the attendee A1 is not registered in advance in the voice print registration memory M2 of the voice print authentication unit 32, the authentication pass signal S32 of “0” indicating failure is sent from the voice print authentication unit 32 to the imaging adjustment unit 36. Is output.

When the authentication pass signal S32 of “1” is input, the imaging adjustment unit 36 sets the television camera devices 40A1 and 40A2 based on the first imaging condition information about the first microphone MC1 indicated by the microphone selection signal S251. Control. As a result, the direction of the first microphone and the area MIC1 AREA are imaged, and the conference attendee A1 is imaged.
The imaging adjustment unit 36 continues imaging the direction of the first microphone and the area MIC1 AREA with the television camera 40 based on the first imaging condition information while the conference attendee A1 is speaking.

Next, it is assumed that the conference attendee A3 using the third microphone MC3 who has no voiceprint registered in the voiceprint authentication unit 32 speaks and the DSP25 selects the speech.
From the DSP 25, a microphone selection signal S251 indicating the third microphone MC3 is output to the imaging adjustment unit 36 via the microprocessor 23. Of course, the collected sound signal of the third microphone MC3 is input to the DSP 26, subjected to echo cancellation processing, and output to the voiceprint authentication unit 32 as the DSP 26.
Since the voice print of the attendee A3 is not registered in the voice print authentication unit 32, the voice print authentication unit 32 outputs an authentication pass signal S32 of “0” indicating failure to the imaging adjustment unit 36.
The imaging adjustment unit 36 determines that it is the default when the authentication pass signal S32 of “0” is input. As a process in the case of default, for example, the imaging adjustment unit 36 continues the imaging conditions of the TV camera devices 40A1 and 40A2, or the left and right of the conference room and the entire conference attendee are imaged as an initial state.

  When a plurality of conference attendees speak at the same time, the DSP 25 selects the one with the higher sound level, and thereafter, the imaging conditions of the TV camera devices 40A1 and 40A2 via the imaging adjustment unit 36 according to the result of the voiceprint authentication described above. Is controlled.

The above processing is performed in exactly the same way in the audio collecting / imaging device at the other end of the remote conference.
If voiceprint registration / authentication cannot be used at the remote site, voiceprint registration of the conference attendee at the remote conference and voiceprint authentication during the conference will be performed on this side where the communication device is installed. It is also possible to control the imaging conditions of the TV camera apparatus.

  By using the audio collecting / imaging device of the third embodiment, it is of course transmitted to the other party of the remote conference by clear audio and video, but when the conference participant speaks, the voiceprint is authenticated, The TV camera device can be directed toward the voice-printed speaker.

  According to the third embodiment, it is not necessary to provide a separate microphone for each conference attendant, and it is not necessary to control the imaging conditions of the TV camera devices 40A1 and 40A2 by the system administrator, for example, the chairperson.

  Furthermore, even if the conference attendee moves during the conference, a valid microphone is selected by the microphone selection process in the DSP 25 and the voice print authentication by the voice print authentication unit 32 causes the TV camera devices 40A1 and 40A2 to be connected to the conference attendee. Can be directed in the direction and area.

  During the conference, the name of the speaker is automatically displayed on the television receiver or the television receiver without any action from the system administrator (for example, chairperson).

  As described above, as a preferable example of the third embodiment, referring to FIGS. 26 and 27, the microphone selection in the DSP 25 is performed, and the voice print authentication in the voice print authentication unit 32 is performed. Although the case where the devices 40A1 and 40A2 are controlled in accordance with the imaging conditions has been described, basically, the imaging control of the television camera devices 40A1 and 40A2 by the imaging adjustment unit 36 can be performed only for the microphone selection result by the DSP 25.

The implementation of the third embodiment is not limited to the case where the microphones are arranged radially at equal angles as described in the first embodiment. Even when the microphones are not arranged radially at an equal angle, the DSP 25 can select, for example, a microphone that exhibits the maximum amplitude, and the voiceprint authentication unit 32 determines whether or not it matches the voiceprint registered in advance. It can be authenticated.
Even in this case, the imaging adjustment unit 36 controls the imaging conditions of the television camera devices 40A1 and 40A2 based on the imaging condition information set in advance.

According to the third embodiment of the present invention, even if the speaker changes during the conference, the selection of the camera that displays the speaker and the pan, tilt, and zoom of the selected camera automatically change. Thus, it is not necessary to change the setting manually, and a clear image of the speaker can always be projected.
In addition, by using the speaker direction detection technology and the image recognition technology, panning, tilting, and zooming operations of a camera that displays the speaker are automatically performed, and a clear image of the speaker can be displayed. In particular, by performing speaker voiceprint matching, the camera pans, tilts, and zooms automatically whenever the speaker changes, and a new speaker can be clearly photographed.
Further, according to the third embodiment of the present invention, even if the relative positions of the microphone and the TV camera devices 40A1 and 40A2 are not strict, practical images and sounds can be recorded by the above-described image recognition processing or the like.

Fourth Embodiment The fourth embodiment of the present invention is an invention that extends the third embodiment described above.
(1) Types of imaging means In the third embodiment, the case where two television camera devices are used as the imaging means has been described. In the fourth embodiment, a small camera, for example, A CCD camera is used.

(2) Arrangement of Microphone and CCD Camera In FIGS. 32 and 33, the white circle represents the CCD camera, and the black circle represents the microphone.
FIG. 32 shows an example in which the microphone and the CCD camera are completely arranged in a one-to-one relationship, and FIG. 33 shows an example in which the microphone and the CCD camera are not completely arranged in a one-to-one relationship.

(3) Number of Imaging Means and Imaging Range As illustrated in FIGS. 32 and 33, a plurality of CCD cameras are arranged at intervals such that the imaging ranges do not overlap each other. It is not necessary for the sound collection range of each microphone, each imaging means, and the imaging range to completely match.
That is, since the microphone sound collection range and the CCD camera shooting range may be different, there is no need to match the position and quantity of the microphone and the CCD camera, but the direction selected by the microphone can be shot with a certain CCD camera. is necessary.

In the present embodiment, it is sufficient that the relationship between the microphone sound collection range and the CCD camera imaging range is clear, and the microphones are arranged at equal intervals as described in the first embodiment. The directivity as illustrated in FIG. 7 is not necessarily required.
However, as described in the first embodiment, it is easy and effective in terms of processing and design that the microphones are arranged at equal intervals and have directivity.

The reason why the CCD camera is used as the imaging means is that the microphone and the CCD camera are placed close to each other so that the number of CCD cameras is the same as or close to the number of microphones. A priced CCD camera was used.
In the present embodiment, it is practically difficult to install an imaging unit having a large size such as a television camera device together with a microphone, and a small CCD camera is used like the microphone.

FIG. 34 is a partial configuration diagram of a circuit for selecting and processing a microphone and a CCD camera.
The microphone selection processing unit 251 is a part that performs a microphone selection process among the processes of the DSP 25 described in the first to third embodiments.
The camera selection processing unit 252 is a part that performs processing added in the present embodiment in the DSP 25.
The video changeover switch circuit 37 is a part added in the present embodiment, and a switch circuit that selects and outputs one of video signals picked up by a plurality of CCD cameras in accordance with a command from the camera selection processing unit 252. It is.
The camera selection processing unit 252 selects a CCD camera according to a command from the microphone selection processing unit 251 and selects a CCD camera according to a camera selection instruction button (not shown) provided on the operation unit 15. There is. Whether to use the instruction of the camera selection instruction button or the instruction of the microphone selection processing unit 251 can be determined as appropriate. For example, the instruction of the camera selection instruction button can be given priority over the instruction of the microphone selection processing unit 251 or vice versa. Alternatively, the priority order may not be assigned, and the CCD camera may be switched each time a selection command is issued.
The image composition unit 38 is a part for compositing images captured by a plurality of CCD cameras in accordance with an image composition instruction button (not shown) provided on the operation unit 15. In addition to synthesizing images, the image synthesizing unit 38 also performs a process of dividing a plurality of video signals into a single screen.

FIG. 35 is a flowchart showing processing performed by the DSP 25 when the CCD camera and the microphone are installed at the same position.
FIG. 36 is a flowchart showing processing performed by the DSP 25 when it is assumed that the CCD camera and the microphone may not be provided at the same position.

Operation of the First Mode The operation of the first mode will be described with reference to FIG.
In step S91, when the microphone selection processing unit 251 of the DSP 25 detects the microphone from which sound is emitted by the method of the first embodiment described above, the microphone newly detected in step S92 has been selected up to the previous time. It is determined whether or not. If they are the same, the previously selected microphone is continuously selected, and the direction of the speaker can be specified by the selected microphone according to the method of the first to third embodiments described above.

  Note that the microphone selection processing unit 251 does not switch the selection of the microphone and does not perform the process of switching the CCD camera for an extremely short time of sound such as a simple match, so that the images are displayed one by one. It is possible to avoid the troublesomeness or unnaturalness of switching.

  In step S92, when the microphone selection processing unit 251 of the DSP 25 detects that the microphone is a new microphone, the microphone selection processing unit 251 of the DSP 25 selects a new microphone according to the above-described embodiment, and audio is output from the selected microphone. At the same time, information on the selected microphone is output to the camera selection processing unit 252 to notify that the selection of the microphone has been switched.

The camera selection processing unit 252 stores a CCD camera provided alongside the selected microphone in a memory in the camera selection processing unit 252. Therefore, the camera selection processing unit 252 outputs a switching command to the video changeover switch circuit 37 so as to select the CCD camera corresponding to the microphone selected and switched by the microphone selection processing unit 251.
The video changeover switch circuit 37 switches the selection of the video signal output of the CCD camera in accordance with a command from the camera selection processing unit 252.
In this example, there is no image composition instruction from the image composition instruction button. Therefore, the video signal output from the video changeover switch circuit 37 passes through the image synthesis unit 38 and is displayed on, for example, the television receiver 50A illustrated in FIG. 2 and sent to the conference room on the other side. Displayed on the television receiver in the other party's conference room. Of course, the selected voice is transmitted to the other party's conference room in the same manner as in the above-described embodiment, and can be heard by the conference attendee in the other party's conference room.

As described above, according to the first embodiment of the fourth embodiment, in addition to the quick selection of sound from the microphone, it is possible to quickly select the corresponding CCD camera at the same time. The video and audio of the speaker in use can be quickly switched and output.
In particular, in the first embodiment of the fourth embodiment, it is possible to take an image of a speaker in front of a microphone without adding a zoom function or the like to the CCD camera, and the zoom function as in the third embodiment. And the operation of determining the appropriate video of the speaker by detecting the contour is unnecessary, and the video can be switched simultaneously with the switching of the voice.

Operation of the Second Mode The operation of the second mode will be described with reference to FIG.
The processing in steps S91 to S92 is almost the same as the processing described with reference to FIG. However, in this example, as illustrated in FIG. 33, when one microphone is selected, the imaging results of the CCD cameras on both sides of the microphone can be synthesized and output. Therefore, the following processing is performed.

In step S93A, when microphone selection information is input from the microphone selection processing unit 251, the camera selection processing unit 252 searches the memory in the camera selection processing unit 252, and is adjacent to the selected microphone. It is determined whether or not the images from the two CCD cameras should be combined. When only one CCD camera is designated for the selected microphone in the memory in the camera selection processing unit 252, as described with reference to step S93, an instruction to select the CCD camera is displayed as an image. Output to the changeover switch circuit 37.
When two CCD cameras on both sides with respect to the selected microphone are designated in the memory in the camera selection processing unit 252, the camera selection processing unit 252 switches the image of the imaging signal from one CCD camera. The video signal is selected so that the video signals of the two CCD cameras are alternately selected and input to the image synthesis unit 38 at a time interval that can be selected by the switch circuit 37 and passed therethrough and received by the image synthesis unit 38. The changeover switch circuit 37 is instructed.

  In step S <b> 94, when the image composition unit 38 receives an image composition instruction from the camera selection processing unit 252, the image composition unit 38 outputs two consecutive imaging signals output from the video changeover switch circuit 37 at the predetermined interval. The image is stored in the frame memory, and the two kinds of video signals stored in the memory are combined and output as one image. The image synthesizing method is designated in advance in the memory of the image synthesizing unit 38 for each combination of two CCD cameras, and the image synthesizing unit 38 synthesizes two video signals according to the synthesizing method to generate one image. To do.

Further, as described above, the image composition unit 38 can not only synthesize two types of video signals but also output an image obtained by dividing one screen into two.
In the image composition unit 38, whether to synthesize two types of video signals into one image or to divide the image into two images for output may be designated from the camera selection processing unit 252 The user may specify using an image composition instruction button in the unit 15.

When two types of video signals are divided into one screen, the video signals may be divided into two equal parts, the first video signal is displayed on the full screen, and the second video signal is displayed in a small frame. A multi-display method may be used.
Such an image composition method may also be designated from the camera selection processing unit 252 or may be designated by the user using an image composition instruction button on the operation unit 15.

  As in the first example, the video signal output from the image composition unit 38 is displayed on, for example, the television receiver 50A illustrated in FIG. It is displayed on the television receiver in the room. Of course, the selected voice is sent to the other party's conference room in the same manner as the above-described embodiment, and can be heard by the conference attendee in the other party's conference room.

The number of CCD cameras to be selected in connection with the selection of one microphone is not limited to two CCD cameras adjacent to the selected microphone, and may be plural, for example, three or four.
For example, when four CCD cameras are selected, the imaging signals of the four CCD cameras can be displayed on one screen by being divided into four.

  As described above, according to the second embodiment of the fourth embodiment, in addition to the same effect as the first embodiment, the video signal of one or more CCD cameras related to the selection of one microphone is obtained. It can be output in a synthesized or divided state.

First Modification of Fourth Embodiment In the fourth embodiment described above, the case where the zoom function, pan function, etc. are not added to the CCD camera has been described. However, the CCD function of the present embodiment has a zoom function. The addition is easy, as has been realized in recent mobile phones with cameras.
Therefore, a zoom function or the like may be added to the CCD camera of this embodiment. With such a zoom function, for example, when a zoom button is added to the operation unit 15 and the user presses the zoom button, zooming can be performed at a predetermined speed during the pressing period.
That is, even when a CCD camera is used, various processes are performed except that the direction of the television camera device is changed as in the case where the television camera device is used as the imaging means in the third embodiment. Can do.
Such various imaging condition changing processes are performed in the same manner as in the third embodiment.

Second Modification of Fourth Embodiment Also in the fourth embodiment, as in the third embodiment, the second embodiment, that is, the voice print authentication unit 32 is added, and only the voice that has been voice print authenticated is a microphone. Can be selected. As a result, it is possible to select a more accurate microphone and to select a field imaging means such as a CCD camera.

  In the fourth embodiment, a CCD camera is exemplified as a small-sized image pickup means. However, an image pickup means such as another small camera that can be provided with a microphone and has the above-described functions and suitable for the above-described purpose is used. You can also.

As described above, in the first mode of the fourth embodiment, the microphone and the small-size imaging unit are provided. Therefore, if the speaker using the microphone can be selected, 1 or 2 suitable for photographing the speaker using the microphone, or other related small imaging means is selected. This can be uniquely selected from a small-sized imaging means CCD camera provided in the vicinity.
As a result, according to the first form of the fourth embodiment, in addition to the quick selection of the sound from the microphone, the corresponding or related one or more small imaging means can be quickly selected and selected. The voice of the speaker using the selected microphone and the video of the speaker and / or the speaker can be quickly switched and output.

Further, there is no need for the user to press the microphone selection button.
It should be noted that according to the selection method of the small imaging means, it is possible to project only a specific direction regardless of the speaker, or on the contrary not to display a video in a specific direction.
Furthermore, as described above, it is possible to avoid selecting a camera that can shoot the direction one by one by providing a time for which processing is not performed for a certain period of time for an extremely short time of pronunciation such as mere conflict.
In addition, it is possible to automatically switch between the microphone on this side and the small image pickup means in accordance with an instruction from the other party of the conference.

In carrying out the present invention, the plurality of embodiments described above can be combined as appropriate.
In addition, about the 1st-4th embodiment, it is possible to change a microphone and an imaging means of this side according to the hope from the meeting room of the other party, and to change a sound collection object person and an imaging object person. Similarly, it is possible to change the other party's microphone and imaging means from this side, and change the sound collection target person and the photographing target person.
As a result, the bidirectional sound collection / video imaging apparatus can be effectively utilized.

FIG. 1A is a diagram showing an outline of a conference system as an example to which the sound collection / video imaging apparatus of the present invention is applied, and FIG. 1B is a sound collection in FIG. -It is a figure which shows the state by which the communication apparatus of a video imaging device is mounted, FIG.1 (C) is a figure which shows arrangement | positioning with the communication apparatus mounted on the table, and a conference attendant. FIG. 2 is a plan configuration diagram of the sound collection / video imaging apparatus according to the embodiment of the present invention. FIG. 3 is a perspective view of the communication device according to the embodiment of the present invention. FIG. 4 is an internal cross-sectional view of the communication device illustrated in FIG. FIG. 5 is a plan view of the microphone / electronic circuit housing portion from which the upper cover of the communication device illustrated in FIG. 3 is removed. FIG. 6 is a diagram showing the configuration and connection state of the main circuit of the microphone / electronic circuit housing portion of the first embodiment, and the connection of the first digital signal processor (DSP1) and the second digital signal processor (DSP2). Shows the connection state. FIG. 7 is a characteristic diagram of the microphone illustrated in FIG. 8A to 8D are graphs showing the results of analyzing the directivity of a microphone having the characteristics illustrated in FIG. FIG. 9 is a partial configuration diagram of a modification of the communication device of the present invention. FIG. 10 is a graph showing an outline of the entire processing contents in the first digital signal processor (DSP 1). FIG. 11 is a diagram showing filtering processing in the communication device of the present invention. FIG. 12 is a frequency characteristic diagram showing the processing result of FIG. FIG. 13 is a block diagram showing bandpass filtering processing and level conversion processing according to the present invention. FIG. 14 is a flowchart showing the processing of FIG. FIG. 15 is a graph showing a process for determining the start and end of speech in the communication device of the present invention. FIG. 16 is a graph showing the flow of normal processing in the communication device of the present invention. FIG. 17 is a flowchart showing the flow of normal processing in the communication device of the present invention. FIG. 18 is a block diagram illustrating a microphone switching process in the communication device of the present invention. FIG. 19 is a block diagram illustrating a method of microphone switching processing in the communication device of the present invention. FIG. 20 is a diagram showing a configuration and connection state of main circuits of the microphone / electronic circuit housing portion of the second embodiment. FIG. 21 is a graph showing the processing of the voiceprint authentication unit illustrated in FIG. FIG. 22 is a first flowchart showing the processing of the voiceprint authentication unit illustrated in FIG. FIG. 23 is a second flowchart showing the processing of the voiceprint authentication unit illustrated in FIG. FIG. 24 is a third flowchart showing the processing of the voiceprint authentication unit illustrated in FIG. FIG. 25 is a fourth flowchart showing the processing of the voiceprint authentication unit illustrated in FIG. FIG. 26 is a configuration diagram of the conference apparatus according to the third embodiment. FIG. 27 is another configuration diagram of the conference apparatus according to the third embodiment. FIG. 28 is a flowchart showing the operation of the third embodiment. FIG. 29 is a flowchart showing the first embodiment and the flow of processing from detection to photographing (part 1). FIG. 30 is a flowchart showing the process flow (No. 2) from detection to photographing in the third embodiment. FIG. 31 is a diagram illustrating an imaging state of the television camera apparatus according to the third embodiment. FIG. 32 is a diagram showing an arrangement example of a microphone and a CCD camera as a first embodiment of the fourth embodiment. FIG. 33 is a diagram showing an arrangement example of a microphone and a CCD camera as a second embodiment of the fourth embodiment. FIG. 34 is a partial configuration diagram of a circuit for selecting and processing the microphone and the CCD camera according to the fourth embodiment. FIG. 35 is a flowchart showing processing performed by the DSP in the case of the arrangement of the microphone and the CCD camera illustrated in FIG. FIG. 36 is a flowchart showing processing performed by the DSP in the case of the arrangement of the microphone and the CCD camera illustrated in FIG.

Explanation of symbols

1A, 1B..Voice sound collection / video imaging device 10A, 10B.
11 .. Upper cover, 12 .... Sound reflector, 13 .... Connecting member
14 .. Speaker housing part, 15 .. Operation part, 16 ..
17 .. Restraining member 18.. Damper 2.. Microphone · Electronic circuit housing
MC1 ~ MC ・ ・ Microphone
21 .. Printed circuit board, 22 .. Microphone support member
23. Microprocessor, 24. Codec
25..First DSP
251 .. Microphone selection processing unit
252 .. Camera selection processing unit
26 .. Second DSP
27 ·· A / D converter block, 271 to 274 ·· A / D converter
28..D / A converter block, 29..Amplifier block
30 .. Microphone selection result display means
301-306 .. Variable gain amplifier
32. Voiceprint authentication department
34 .. Amplifier gain adjustment section
36. Imaging adjustment unit
37..Image changeover switch circuit
38. ・ Image composition part
40 (40A, 40B) .. TV camera device (imaging means)

Claims (9)

A plurality of microphones, each annularly arranged radially,
A plurality of first small-sized imaging means provided in correspondence with each of the plurality of microphones and disposed in proximity to the corresponding microphone so that a sound collection range of the corresponding microphone can be imaged;
At least one second small image pickup means disposed so as to be capable of picking up a sound collection range of the microphone at a predetermined position between each microphone and the corresponding first small image pickup means;
A first relationship between each microphone and each corresponding first small-sized image pickup means, and when the second small-size image pickup means located in the vicinity of the microphone is located, the second small-size image pickup means Storage means for storing the second relationship of
Microphone selection means for detecting sound collection signals of the plurality of microphones and selecting a microphone that has detected an effective sound collection signal among the detected sound collection signals;
Based on the first relation stored in the storage means, the first small image pickup means close to the selected microphone is selected, and the second relation stored in the storage means applies. An imaging means selecting means for selecting the second small imaging means when the second small imaging means is present;
When there is a first image signal picked up by the first small image pickup means selected by the image pickup means selection means and the corresponding second small image pickup means, the second image pickup image picked up by the second small image pickup means is present . Imaging signal selection means for selectively outputting the two imaging signals ;
The imaging signal selection unit is selected, and the first imaging signal and the second imaging signal are combined into one image combining unit or divided into one screen.
Audio collection / video imaging device. When the image pickup means selection means selects the first small image pickup means and the second small image pickup means,
The imaging signal selection means switches the first imaging signal and the second imaging signal continuously and inputs them to the image synthesis means,
The image synthesizing unit synthesizes the first imaging signal and the second imaging signal that are continuously input into one screen based on a predesignated condition, or within one screen. Divide multiple image signals,
The sound collection / video imaging apparatus according to claim 1. The first small-sized imaging means is a CCD camera;
The second small-sized imaging means is a CCD camera;
The sound collection / video imaging apparatus according to claim 1 or 2. The first small-sized imaging unit and / or the first small-sized imaging unit has a zoom function for performing zoom processing according to an instruction.
The sound collection / video imaging apparatus according to claim 3. The sound collection / video imaging device further includes voiceprint authentication means for authenticating voiceprints of a plurality of speakers using the plurality of microphones,
The microphone selection means selects the effective microphone of the collected sound when the voiceprint authentication is performed by the voiceprint authentication means;
The sound collection / video imaging apparatus according to claim 1. The microphone selecting means sets at least the first imaging means to a default state when the voiceprint authentication means is not authenticated by the voiceprint authentication means;
The sound collection / video imaging apparatus according to claim 5. The microphone selection means does not change at least the selection of the microphone and does not change the first imaging means as the default state.
The sound collection / video imaging apparatus according to claim 6. The microphone selection means sets at least the first imaging means as an initial imaging condition as the default state;
The sound collection / video imaging apparatus according to claim 7. Each of the microphones has a predetermined directivity,
The sound collection / video imaging apparatus according to claim 1.

Images