JP5866505B2 - Voice processing system and voice processing method - Google Patents

Description

  The present invention relates to an audio processing system and an audio processing method for reproducing recorded video data and audio data.

  Conventionally, in a surveillance system installed in a factory, a store (for example, a retail store, a bank) or a public place (for example, a library), a plurality of surveillance cameras (for example, a pan-tilt camera and an omnidirectional camera) are connected using a network. Thus, the image data (including still images and moving images; the same applies hereinafter) of the surrounding video to be monitored is improved in image quality and wide angle of view.

  In addition, since the amount of information that can be obtained by video-only monitoring is inevitably limited, a monitoring system that obtains video data and audio data around the monitoring target by arranging a microphone in addition to the monitoring camera has recently appeared. ing.

  As a prior art for obtaining sound data around a monitoring target, an image pickup unit that obtains a picked-up image and a plurality of microphones (sound pickup units) that pick up sound data are used, and sound data that each microphone picks up is used. A sound processing apparatus that generates sound data having directivity in a predetermined sound collection direction designated from a sound reproduction apparatus as a client is known (see, for example, Patent Document 1).

  In Patent Document 1, the sound processing device receives a sound data collected by a plurality of sound collection units (microphones) in a predetermined sound collection direction received in advance from a client (sound reproduction device) connected via a network. Synthesis is performed based on the control command, voice data having directivity in the same direction is generated, and the synthesized voice data is transmitted to the client (sound reproduction device).

Japanese Patent Laid-Open No. 2000-209589

  When the sound processing device shown in Patent Document 1 is applied to a manned monitoring system, the sound processing device designates a sound collection direction when any accident occurs during recording of a captured image around the monitoring target. Audio data having directivity in the sound collecting direction can be generated immediately after receiving from the client (sound reproduction device).

  However, when the sound processing device shown in Patent Document 1 is applied to, for example, an unmanned monitoring system, after the occurrence of an accident, the video data and audio data recorded before the occurrence of the accident are reproduced to reproduce information about the accident ( For example, it is desired to obtain audio data. In this case, the sound processing apparatus has directivity in the place where the accident has occurred, that is, in the desired sound collection direction, because the place where the accident has occurred is not necessarily the predetermined sound collection direction designated in advance by the client. It may be difficult to obtain audio data. That is, there is a problem that there is a high possibility that effective information regarding the accident cannot be obtained from the recorded video data and audio data.

  In order to solve the above-described conventional problems, the present invention provides audio data in a directional direction toward a position corresponding to one or more designated locations designated on a display screen on which captured video data is displayed. It is an object of the present invention to provide a voice processing system and a voice processing method that can be output with emphasis and that can easily recognize a parameter state of voice data.

The present invention includes an imaging unit that captures an image and a rectangular display area, and displays video data captured by the imaging unit in a circular video display area that is smaller than the rectangular display area. A display unit, a plurality of microphones, a sound collection unit that collects sound using the microphones, a sound output unit that outputs sound data collected by the sound collection unit, and a display unit An operation unit that accepts designation, and an emphasized voice that emphasizes voice in a directivity direction from the sound collection unit to a position corresponding to the designated location of the video data that has been designated by the operation unit, based on the audio data data generated or synthesized and a and a signal processing unit for audio output from the audio output unit, the signal processing unit, in response to designation of acceptance into the video display area outside by the operation unit, the voice The status display area or operating area for adjustment of the force, straddle the borders of the video display area from one of the four corners of the rectangular display area to the center of the image display region in a rectangular area whose diagonal it is displayed as a sound voice processing system.

In addition, the present invention provides a step of capturing an image in an image capturing unit, a step of capturing sound in a sound collecting unit including a plurality of microphones, and a video image data captured by the image capturing unit in a rectangular display area. a step of displaying the rectangular display small circular image display area than the area of the display unit with, the image data is displayed on the display unit, voice output audio data picked up by the sound pickup unit A step of outputting a sound to a unit, a step of receiving a designation to the display unit by an operation unit, and a designated portion of the video data that has been designated by the operation unit from the sound collection unit based on the audio data a step of voice output from the voice output unit generates or synthesizes the emphasized sound data that emphasizes the orientation of the sound toward the position corresponding to the operating unit Wherein in response to reception of the specified to the video display area outside with, the status display area or operating area for adjustment of the audio output from any of the four corners of the rectangular display area to the center of the video display area Displaying in a rectangular area having a diagonal line across the boundary of the video display area .

  According to the present invention, it is possible to emphasize and output audio data in a directivity direction toward a position corresponding to one or more designated locations designated in a display screen on which captured video data is displayed, The state of the parameters of the audio data can be easily displayed.

(A), (B) The block diagram which shows the system configuration | structure of the speech processing system of each embodiment. (A) External view of microphone array, (B) External view of microphone array according to the third embodiment, (C) A view showing a mounting state of the microphone array and the camera Illustration of the principle of directivity control processing using a microphone array Flow chart for explaining the operation procedure at the time of recording in the voice processing system The flowchart explaining the operation | movement procedure at the time of reproduction | regeneration of a voice processing system in the case of designating one or more designation | designated places. The schematic diagram which shows an example of the usage pattern of the audio | voice processing system of 1st Embodiment, (A) The mode that one camera and one microphone array were installed in the position which separated from the ceiling of the indoor hall, for example, for example (B) The figure which shows a mode that video data is displayed on a display, and audio | voice data is audio-outputted in the speaker. Schematic diagram showing an example of a usage pattern of the speech processing system of the second embodiment, (A) For example, two microphones and one microphone array at an intermediate position between the two cameras on the ceiling of an indoor hall And (B) a diagram showing a state in which video data picked up by the camera 10 is displayed on the display 63 and audio data is output as audio from the speaker 65. (C) The figure which shows a mode that the video data imaged with the camera 10A is displayed on the display 63, and audio | voice data are output by the speaker 65 as audio | voice. The schematic diagram which shows an example of the usage pattern of the speech processing system of 4th Embodiment, (A) A mode that one camera, one microphone array, and the speaker were installed in the ceiling of the indoor hall, for example (B) Explanatory drawing of operation | movement outline | summary of audio | voice processing system in case the some designation | designated location is designated in the video data displayed on the display. Schematic diagram showing an example of a usage pattern of a sound processing system, (A) A donut-shaped microphone array, a camera integrated with the microphone array, and a speaker, for example, installed on the ceiling of an indoor hall FIG. 5B is a diagram showing a state in which two persons 91 and 92 are selected in the video data captured by the camera 10E, and FIG. 5C is a diagram showing video data of the two persons 91 and 92 after image conversion. (D) The state in which two persons 93 and 94 are selected in the video data captured by the camera 10E. (E) Video data of two persons 93 and 94 after image conversion are displayed on the display, and voice data of conversations of the persons 93 and 94 is displayed on the speaker 65. Shows a state in which the audio output Te (A), (B), (C) External view of other microphone arrays 20D, 20E, 20F Schematic diagram showing the operation of the display 63 and the speaker 65 when a plurality of designated locations are designated. Disassembled perspective view of the housing structure of the microphone array of each embodiment (A) Plan view of the housing structure of the microphone array shown in FIG. 12, (B) AA sectional view of FIG. The principal part enlarged view of the dotted-line range of FIG.13 (B) (A) The perspective view which shows a mode that a punching metal cover is fixed to a main housing | casing, (B) It is sectional drawing which shows a mode that a punching metal cover is fixed to a main housing | casing. Schematic diagram of microphone mounting structure Top view of microphone board (A) A diagram of a microphone substrate circuit in which one ripple removal circuit is provided in a plurality of microphone circuits, (B) a diagram of a microphone substrate circuit in which a ripple removal circuit is provided in each of the plurality of microphone circuits. (A) The perspective view of the housing | casing structure of the microphone array to which the omnidirectional camera was attached without attaching the camera adapter, (B) The perspective view of the housing | casing structure of the microphone array to which the outdoor omnidirectional camera was attached with the camera adapter. Figure An exploded perspective view of a microphone array housing structure to which an indoor omnidirectional camera is attached An exploded perspective view of a microphone array housing structure to which an outdoor omnidirectional camera is attached (A) Side view of housing structure of microphone array to which outdoor omnidirectional camera is attached, (B) BB sectional view of FIG. 22 (A) The principal part enlarged view of the dotted line range of FIG. Exploded perspective view of the microphone array housing structure to which the lid is attached Disassembled perspective view of a housing structure that can be attached to the ceiling using mounting brackets (A) Side view of the base metal plate fixing pin before being inserted into the base metal plate fixing hole, (B) Side view of the base metal plate fixing pin inserted into the base metal plate fixing hole, (C) Base metal plate fixing Plan view of the base sheet metal side fixing pin inserted into the hole, (D) Side view of the base sheet metal side fixing pin moved to the small diameter hole of the base sheet metal fixing hole, (E) Movement to the small diameter hole of the base sheet metal fixing hole Plan view of fixed base sheet metal side fixing pin Sectional drawing of the housing | casing structure of the microphone array in which the taper was provided in the recessed part for ECM Cross-sectional view of microphone array housing structure with wind countermeasures (A) A cross-sectional view of the housing structure of the microphone array showing the relationship between the inner diameter and depth of the ECM recess, (B) a cross-sectional view of the microphone array housing structure in which the inner wall of the ECM recess is an inclined wall, (C) Sectional view of the housing structure of the microphone array in which the inner peripheral corner of the ECM recess is an R portion (A) Explanatory drawing showing the isobaric surface of the ECM recess not forming the taper, (B) Explanatory drawing showing the isobaric surface of the ECM recess forming the taper (A) Explanatory drawing of an example of use of the speech processing system of the fourth embodiment, (B) a first identification shape displayed around the first designated location, and displayed around the second designated location. A state in which an example of the second identification shape is displayed and the first directional sound that is directed to the first sound position corresponding to the first designated location specified by the first identification shape is emphasized in the first. Output from the second speaker, and the second directivity direction sound toward the second sound position corresponding to the second designated location specified by the second identification shape is emphasized and output from the second speaker. Figure showing the situation The figure which shows a mode that the operation box for adjustment is displayed according to the click operation outside the display area of the video data displayed on the display in the state where the video data shown in FIG. 31 (B) is displayed. (A) Explanatory drawing of an example of use of the speech processing system of the fourth embodiment, (B) a first identification shape displayed around the first designated location, and displayed around the second designated location. A state in which an example of the second identification shape is displayed and the first directional sound that is directed to the first sound position corresponding to the first designated location specified by the first identification shape is emphasized in the first. Output from the second speaker, and the second directivity direction sound toward the second sound position corresponding to the second designated location specified by the second identification shape is emphasized and output from the second speaker. Figure showing the situation In the state where the video data shown in FIG. 31 (B) is being displayed, the video data captured by the omnidirectional camera and the adjustment operation box for each click operation outside the display area of the video data displayed on the display, Showing how to switch and display The figure which shows a mode that the box for a status indication is displayed according to the click operation outside the display area of the video data displayed on the display in the state where the video data shown in FIG. 31 (B) is displayed. (A) Explanatory drawing of an example of use of the speech processing system of the fourth embodiment, (B) a first identification shape displayed around the first designated location, and displayed around the second designated location. An example of displaying the second identification shape, the third identification shape displayed around the third designated location, the fourth identification shape displayed around the fourth designated location, and the first In the voice data that emphasizes the voice in the first directivity direction toward the first voice position corresponding to the first designated location specified by the identification shape and the second designated location specified by the second identification shape The voice data in which the voice in the second directivity direction toward the corresponding second voice position is emphasized, and the third toward the third voice position corresponding to the third designated position specified by the third identification shape. Voice data that emphasizes voice in the directional direction is output from each of the first and second speakers. Shows a state in which the force The figure which shows a mode that the operation box for adjustment is displayed according to simultaneous pressing operation of the several specific key of a keyboard in the state as which the video data shown to FIG. 36 (B) is displayed. The figure which shows a mode that the operation box for adjustment is displayed according to the click operation outside the display area of the video data displayed on the display in the state where the video data shown in FIG. 36 (B) is displayed. (A) Explanatory drawing of an example of use of the speech processing system of the fourth embodiment, (B) a first identification shape displayed around the first designated location, and displayed around the second designated location. An example of displaying the second identification shape, the third identification shape displayed around the third designated location, the fourth identification shape displayed around the fourth designated location, and the first In the voice data that emphasizes the voice in the first directivity direction toward the first voice position corresponding to the first designated location specified by the identification shape and the second designated location specified by the second identification shape A state of synthesizing and outputting from the first speaker the voice data in which the voice in the second directivity direction toward the corresponding second voice position is emphasized, and the third designated portion specified by the third identification shape A sound that emphasizes the sound in the third directivity direction toward the third sound position corresponding to Shows how the output data from the second speaker In the state where the video data shown in FIG. 39B is displayed, the adjustment operation box is displayed in response to a touch outside the display area of the video data displayed on the display provided with the touch panel. Illustration

  Hereinafter, embodiments of a voice processing system and a voice processing method according to the present invention will be described with reference to the drawings. The voice processing system of each embodiment is applied to a monitoring system (including a manned monitoring system and an unmanned monitoring system) installed in a factory, a public facility (for example, a library or an event venue), or a store (for example, a retail store or a bank). The

(First embodiment)
FIG. 1A and FIG. 1B are block diagrams showing the system configuration of the speech processing systems 5A and 5B of the embodiments. The voice processing system 5 </ b> A includes a monitoring camera 10, 10 </ b> A, a microphone array 20, and a voice processing device 40. The cameras 10, 10 </ b> A, the microphone array 20, and the sound processing device 40 are connected to each other via the network 30.

  The audio processing system 5B includes a monitoring camera 10B, 10C, a microphone array 20A, a recorder 45A, and a PC (Personal Computer) 70. The cameras 10B and 10C, the microphone array 20A, the recorder 45A, and the PC 70 are connected to each other via the network 30A.

  Hereinafter, the operation of each part of the voice processing system 5A will be mainly described, and the contents of the operation of each part of the voice processing system 5B will be described differently from the operation of the voice processing system 5A.

  The cameras 10 and 10 </ b> A as imaging units are monitoring cameras installed on the ceiling (for example, see FIG. 6) of the event venue, for example, and are remote from a monitoring system control room (not shown) connected via the network 30. It has a pan / tilt function, a zoom-in function, and a zoom-out function that can be operated, and captures images (including still images and moving images; the same applies hereinafter) around the point (place) to be monitored. The cameras 10 and 10 </ b> A record captured video data (video data) in the recorder 45 via the network 30.

  The microphone array 20 as a sound collection unit is a microphone that is installed on, for example, a ceiling (for example, see FIG. 6) in an event venue, and is uniformly provided with a plurality of microphones 22 (see, for example, FIG. 2). The microphone array 20 uses each microphone 22 to collect the sound around the point (location) to be monitored, and the sound data (voice data) collected by each microphone 22 via the network. Record in the recorder 45. The structure of the microphone array 20 will be described later with reference to FIG.

  The audio processing device 40 includes a recorder 45, a signal processing unit 50, an operation unit 55, and a playback unit 60. The recorder 45 includes a control unit (not shown) for controlling each process such as data recording in the recorder 45 and a recording unit (not shown) for storing video data and audio data. The recorder 45 records the video data captured by the cameras 10 and 10A and the audio data collected by the microphone array 20 in association with each other.

  The signal processing unit 50 is configured using, for example, a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP), and controls for overall control of operations of each unit of the audio processing device 40. Processing, data input / output processing with other units, data calculation (calculation) processing, and data storage processing are executed.

  The signal processing unit 50 uses the sound data recorded in the recorder 45 to add the sound data collected by each microphone by the sound data directivity control process described later, and each microphone of the microphone array 20. In order to emphasize (amplify) the sound (volume level) in the specific direction from the position 22, sound data in which directivity in the specific direction is formed is generated. In addition, the signal processing unit 50 uses the audio data transmitted from the microphone array 20 to emphasize (amplify) the volume level of the audio from the microphone array 20 in the specific direction (directing direction). Audio data having directivity may be generated. The specific direction is a direction from the microphone array 20 to a position corresponding to a predetermined designated location designated from the operation unit 55, and is designated by the user to emphasize (amplify) the volume level of the audio data. Direction.

  When the video data recorded in the recorder 45 is imaged by an omnidirectional camera (see later), the signal processing unit 50 uses a coordinate system (for example, x-axis, y-axis, z) of the video data recorded in the recorder 45. Conversion processing (two-dimensional or three-dimensional coordinate conversion of the axes) is performed, and the video data after the conversion processing is displayed on the display 63 (see FIGS. 9C and 9E).

  The operation unit 55 is arranged using, for example, a touch panel or a touch pad that is arranged corresponding to the screen of the display 63 and can be input by a user's finger 95 or a stylus pen. The operation unit 55 outputs, to the signal processing unit 50, data on the coordinates of one or more designated locations for which enhancement (amplification) of the volume level of the audio data is desired in accordance with a user operation. The operation unit 55 may be configured using a pointing device such as a mouse or a keyboard.

  The playback unit 60 includes a display 63 and a speaker 65. The playback unit 60 displays video data recorded on the recorder 45 on the display 63, and further causes the speaker 65 to output audio data recorded on the recorder 45. . The display 63 and the speaker 65 may be configured separately from the playback unit 60.

  The display 63 as a display unit displays video data captured by the cameras 10 and 10 </ b> A and recorded in the recorder 45.

  The speaker 65 as an audio output unit is audio data picked up by the microphone array 20 and recorded in the recorder 45, or audio that has been emphasized in a specific direction by the signal processing unit 50 based on the audio data. Output data as audio.

  Here, the voice processing device 40 may be configured as a device in which the recorder 45 and other units in the voice processing device 40 are different (see FIG. 1B). Specifically, the audio processing device 40 illustrated in FIG. 1A may include a recorder 45A illustrated in FIG. 1B and a PC 70 illustrated in FIG. That is, the PC 70 is configured using a general-purpose computer, and includes a signal processing unit 71, a playback unit 72 including a display 73 and a speaker 75, and an operation unit 78. The recorder 45A and the PC 70 correspond to the voice processing device 40 in the voice processing system 5A, and realize the same functions and operations.

  The functions of the cameras 10B and 10C and the microphone array 20A are the same as the functions of the cameras 10 and 10A and the microphone array 20 in the sound processing system 5A, respectively.

  Note that the number of cameras installed in the voice processing systems 5A and 5B is arbitrary. Further, the networks 30 and 30A may be connected to each other so that data can be transferred between the voice processing systems 5A and 5B.

  FIG. 2A is an external view of the microphone array 20. The microphone array 20 includes a plurality of microphones 22 arranged in a disk-shaped housing 21. The plurality of microphones 22 are arranged along the surface of the housing 21, and are arranged along two concentric circles, a small circle having the same center as the housing 21 and a large circle. The plurality of microphones 22A arranged along a small circle are narrow in distance from each other and have characteristics suitable for a high sound range. On the other hand, the plurality of microphones 22B arranged along a large circle have a large diameter and characteristics suitable for a low sound range.

  FIG. 2B is a diagram showing the appearance of the microphone array 20C and the attachment state of the microphone array 20C and the omnidirectional camera 10E (see FIG. 9A) in the third embodiment. A microphone array 20C shown in FIG. 2B includes a donut-shaped housing 21C having an opening 21a formed inside, and a plurality of microphones 22C provided uniformly in the housing 21C. is there. The plurality of microphones 22C are arranged along a concentric circle with respect to the housing 21C.

  In FIG. 2C, the omnidirectional camera 10E shown in FIG. 9A is attached inside the opening 21a of the casing 21C. In the present embodiment, the omnidirectional camera 10E is, for example, a camera equipped with a fisheye lens, and is attached so as to image a wide range of the floor of the hall. As described above, the omnidirectional camera 10E and the microphone array 20C are arranged on the same axis so that the center of the casing 21C of the microphone array 20C is common, so that the same coordinate system can be used.

  FIG. 3 is an explanatory diagram of the principle of directivity control processing using the microphone array 20. In FIG. 3, the principle of directivity control processing using the delay sum method will be briefly described. When sound waves emitted from the sound source 80 are incident on the microphones 22a, 22b, 22c,..., 22n-1, 22n of the microphone array 20 at a certain angle (incident angle = (90−θ) [degrees]). To do. It is assumed that the sound source 80 is disposed in the direction of the predetermined angle θ with respect to the surface of the casing 21 of the microphone array 20. The intervals between the microphones 22a, 22b, 22c,..., 22n-1, and 22n are constant.

  The sound wave emitted from the sound source 80 first reaches the microphone 22a and is collected, then reaches the microphone 22b and is collected one after another, and finally reaches the microphone 22n and is collected. . Note that the direction from the position of each microphone 22a, 22b, 22c,..., 22n-1, 22n of the microphone array 20 toward the sound source 80 is, for example, when the sound source 80 is a voice during a conversation of a person or surrounding music. Assuming the case, it may be considered that the direction is the same as the direction corresponding to the predetermined range designated from the operation unit 55 in order to emphasize (amplify) the volume level of the voice data of the person's conversation or the sound data of the surrounding music. it can.

  Here, the arrival time difference τ1, τ2, τ3,..., Τn− from the time when the sound wave reaches the microphones 22a, 22b, 22c,. 1 is produced. For this reason, when the sound data collected by the respective microphones 22a, 22b, 22c,..., 22n-1, 22n are added as they are, they are added while being out of phase. It will be weakening as a whole.

  Note that τ1 is a difference time between the time when the sound wave reaches the microphone 22a and the time when the sound wave reaches the microphone 22n, and τ2 is the time when the sound wave reaches the microphone 22b and the time when the sound wave reaches the microphone 22n. It is a difference time, and τn-1 is a difference time between the time when the sound wave reaches the microphone 22n-1 and the time when the sound wave reaches the microphone 22n.

  On the other hand, in each embodiment including this embodiment, the signal processing unit 50 includes A / D converters 51a, 51b, 51c provided corresponding to the microphones 22a, 22b, 22c,..., 22n-1, 22n. ,..., 51n-1, 51n, delay units 52a, 52b, 52c,..., 52n-1, 52n, and an adder 57 (see FIG. 3).

  That is, the signal processing unit 50 converts the analog audio data collected by the microphones 22a, 22b, 22c,..., 22n-1, 22n into A / D converters 51a, 51b, 51c,. , 51n, digital audio data is obtained by AD conversion. Further, the signal processing unit 50 uses the delay units 52a, 52b, 52c,..., 52n-1, 52n to set delay times corresponding to the arrival time differences in the respective microphones 22a, 22b, 22c,. After providing the signals and aligning the phases, the adder 57 adds the audio data after the delay processing. As a result, the signal processing unit 50 can generate audio data in which the audio data in the direction of the predetermined angle θ from the installation positions of the microphones 22a, 22b, 22c,..., 22n-1, 22n are emphasized. For example, in FIG. 3, the delay times D1, D2, D3,..., Dn-1, Dn set in the delay units 52a, 52b, 53c,. ,..., .Tau.n-1, and is represented by the formula (1).

  L1 is the difference in the sound wave arrival distance between the microphone 22a and the microphone 22n. L2 is the difference in sound wave arrival distance between the microphone 22b and the microphone 22n. L3 is a difference in sound wave arrival distance between the microphone 22c and the microphone 22n. Ln-1 is a difference in sound wave arrival distance between the microphone 22n-1 and the microphone 22n. Vs is the speed of sound. L1, L2, L3,..., Ln−1, Vs are known values. In FIG. 3, the delay time Dn set in the delay device 52n is 0 (zero).

  Thus, the signal processing unit 50 changes the delay times D1, D2, D3,..., Dn-1, Dn set in the delay units 52a, 52b, 52c,. Audio data recorded in the recorder 45 can be used to generate audio data in which audio data in an arbitrary direction with respect to the installation position of the microphone array 20 is used as a reference, and the audio data in the audio processing systems 5A and 5B can be generated. Directivity control processing can be performed easily.

  Next, each operation at the time of recording and reproduction of the audio processing systems 5A and 5B of the present embodiment will be described. Here, a case where the voice processing system 5A is applied to a monitoring system will be described. FIG. 4 is a flowchart for explaining an operation procedure during recording of the voice processing system 5A.

  In FIG. 4, for example, by remote operation from a user in a monitoring system control room (not shown), the cameras 10 and 10 </ b> A start capturing an image around a monitored point (place) (S <b> 1). Simultaneously or substantially simultaneously with the start of imaging by the cameras 10 and 10A, the microphone array 20 starts collecting sound around the point (location) to be monitored (S2). The cameras 10 and 10 </ b> A transfer the captured video data to the recorder 45 connected via the network 30. The microphone array 20 transfers the collected audio data to the recorder 45 connected via the network 30.

  The recorder 45 associates and stores all the video data transferred from the cameras 10 and 10A and the audio data transferred from the microphone array 20 in a recording medium (S3). The operation at the time of recording with the cameras 10 and 10A, the microphone array 20, and the recorder 45 is completed by a remote operation from the user.

  FIG. 5 is a flowchart for explaining an operation procedure during reproduction of the voice processing systems 5A and 5B when one or more designated portions are designated.

  In FIG. 5, the recorder 45 of the audio processing device 40 accepts designation of video data to be reproduced by direct operation or remote operation from the user (S11). For the designation of the video data, for example, recorded date and camera type are used as conditions. The reproduction unit 60 reproduces video data corresponding to the condition specified in step S <b> 11 and displays it on the screen of the display 63. Furthermore, the reproduction unit 60 reproduces audio data stored in the recorder 45 in association with the reproduced video data, and outputs the audio from the speaker 65.

  Here, during the playback or pause of the video data being played back by the playback unit 60, the user can make a sound (volume level) in the video data displayed on the screen of the display 63 via the operation unit 55. ) Is specified at one or more specified locations to emphasize (amplify). The signal processing unit 50 accepts designation of one or more designated portions that emphasize (amplify) audio (volume level) in the contents of the video data in accordance with a user's designation operation (S12).

  In the following, the designated location designated by the user in order to form directivity in the direction (directing direction) in which the sound (volume level) is emphasized (amplified) with reference to the microphone arrays 20 and 20A via the operation unit 55. Is abbreviated as “specified location”. In step S12, for example, when the user touches the screen of the display 63 with the finger 95, the designated portion of the video data displayed on the screen of the display 63 or a predetermined rectangular sound centering on the touched designated portion is used. Assume that the highlight range is specified.

  Based on one or more designated locations or voice enhancement ranges designated via the operation unit 55, the signal processing unit 50 performs one or more designated locations or sounds from the center position of each microphone 22 of the microphone array 20. The direction (each pointing direction) toward each position (each voice position) corresponding to, for example, the center of the enhancement range is the direction of the predetermined angles θ1, θ2,..., Θn described with reference to FIG. The sound (volume level) is calculated as each direction (each direction) in which the sound (volume level) is emphasized (amplified). Further, the signal processing unit 50 is directed to the calculated predetermined angles θ1, θ2,..., Θn with respect to the audio data stored in the recorder 45 in association with the video data currently reproduced by the reproduction unit 60, respectively. , That is, voice data in which voices (volume levels) at predetermined angles θ1, θ2,..., Θn are emphasized (amplified) are generated (S13).

  In the present embodiment, the signal processing unit 50 is directed in a direction from the center position of each microphone 22 of the microphone array 20 toward each sound position corresponding to, for example, the center of one or more specified locations or sound enhancement ranges. Is generated or synthesized, but is further greatly deviated from the direction (predetermined angles θ1, θ2,..., Θn) toward each audio position corresponding to one or more designated locations or audio enhancement ranges ( For example, audio data for a predetermined angle θ1, θ2,.

  In step S11, the reproduction unit 60 emphasizes (amplifies) the sound (volume level) in the direction toward each sound position corresponding to one or more designated locations or sound enhancement ranges by the signal processing unit 50. In accordance with the designation, audio is output from the speaker 65 in synchronization with the video data displayed on the display 63 (S14). Thereby, the operation | movement at the time of reproduction | regeneration of the audio processing apparatus 40 is complete | finished.

  FIG. 6 is a schematic diagram illustrating an example of a usage pattern of the voice processing system 5A according to the first embodiment. FIG. 6A is a diagram showing a state in which one camera 10 and one microphone array 20 are installed at a distance from a ceiling 85 of a hall as an indoor event venue, for example.

  In FIG. 6A, two persons 91 and 92 are standing on the floor 87 of the hall and having a conversation. A speaker 82 is placed on the floor 87 at a position slightly away from the two persons 91 and 92, and music flows from the speaker 82. Further, the camera 10 captures images of persons 91 and 92 around a monitoring target point (place) set in advance in the camera 10. Furthermore, the microphone array 20 collects the sound of the entire hall.

  FIG. 6B is a diagram illustrating a state in which video data is displayed on the display 63 and audio data is output as audio from the speaker 65. Video data captured by the camera 10 is displayed on the screen of the display 63. In addition, the speaker 65 outputs the conversation between the two persons 91 and 92 or the music in the hall.

  For example, it is assumed that the user touches the vicinity of the center of the video data of the two persons 91 and 92 displayed on the screen of the display 63 with the finger 95. The touch point 63a is a designated location designated by the user. The signal processing unit 50 uses the sound picked up by the microphone array 20, that is, the sound data picked up by each microphone 22, from the position of each microphone 22 of the microphone array 20, and the touch point 63a designated by the user or Audio data in which directivity is formed in the directivity direction (direction indicated by the symbol e shown in FIG. 6A) toward the audio position corresponding to the center of the rectangular range 63b is generated.

  In other words, the signal processing unit 50 uses the audio data collected by each microphone 22 and the audio corresponding to the center of the touch point 63a or the rectangular range 63b specified by the user from the position of each microphone 22 of the microphone array 20. Sound data in which the sound (volume level) in the directivity direction toward the position is emphasized (amplified) is generated. The reproduction unit 60 outputs the audio data generated by the signal processing unit 50 from the speaker 65 in synchronization with the video data captured by the camera 10.

  As a result, the voice data at the touch point 63a or the rectangular range 63b specified by the user is emphasized, and the conversation between the two persons 91 and 92 from the speaker 65 (see, for example, “Hello” shown in FIG. 6A) is large. Sound is output according to the volume. On the other hand, compared to the two persons 91 and 92, music that is placed from a speaker 82 that is placed closer to the microphone array 20 but is not the touch point 63a specified by the user (shown in FIG. 6A). The voice is output with a lower sound volume than the conversation between the two persons 91 and 92.

  As described above, in the present embodiment, the audio processing system 5A or 5B emphasizes the audio data in the video for an arbitrary reproduction time specified by the user during the reproduction of the video data and audio data recorded in the recorder 45. Can be output. As a result, the user simply touches and designates a location where the audio data is to be emphasized while watching the video data displayed on the screen of the display 63, and the designated location including the designated location (audio enhancement) Audio data in the range) can be emphasized and output as audio. As described above, in the audio processing system 5A or 5B according to the present embodiment, the user can easily obtain the audio information in the necessary range while viewing the video data captured by the camera 10 on the display 63. it can.

  For example, the speech processing system 5A or 5B according to the present embodiment has directivity in the direction from the position of each microphone 22 of the microphone array 20 toward the occurrence point of the accident, even when some accident occurs or after the occurrence of the accident. By generating the voice data that forms, the user can confirm the conversation or voice at the time of occurrence of the accident.

  Further, in the audio processing system 5A or 5B of the present embodiment, since the camera 10 and the microphone array 20 are installed on the ceiling 85 such as an indoor hall, it is possible to monitor everywhere in the hall. .

(Second Embodiment)
In the first embodiment, an example of the usage pattern of the audio processing system 5A when there is one camera has been described. In the second embodiment, an example of a usage pattern of the voice processing system 5C when there are a plurality of cameras (for example, two cameras) will be described.

  Note that the voice processing system 5C of the second embodiment has the same configuration as the voice processing system 5A or 5B of the first embodiment, except that there are a plurality of cameras (for example, two). The same components as those of the speech processing system 5A or 5B of the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  FIG. 7 is a schematic diagram illustrating an example of a usage pattern of the voice processing system 5C of the second embodiment. In FIG. 7A, for example, two cameras 10 and 10A, one microphone array 20 at an intermediate position between the two cameras 10 and 10A, and a speaker 83 are installed on the ceiling 85 of an indoor hall. FIG.

  In addition, four persons 91, 92, 93, and 94 are standing on the floor 87 of the hall, the person 91 and the person 92 are talking, and the person 93 and the person 94 are talking. A speaker 82 is placed on the floor 87 at a position between these two sets, and music flows. The speaker 83 is installed on a ceiling 85 almost directly above the person 93 and the person 94.

  The camera 10 images two persons 91 and 92 from positions slightly away from the four persons 91, 92, 93 and 94, and the microphone array 20 is installed on the ceiling 85 almost directly above the speaker 82. The sound of the whole hall is collected. The camera 10 </ b> A images the persons 93 and 94 from positions slightly apart from the four persons 91, 92, 93, and 94.

  FIG. 7B is a diagram illustrating a state in which video data captured by the camera 10 is displayed on the display 63 and audio data is output as audio from the speaker 65. Video data captured by the camera 10 is displayed on the screen of the display 63. In addition, the speaker 65 outputs the conversation between the two persons 91 and 92 or the music in the hall.

  For example, it is assumed that the user touches the vicinity of the center of the video data of the two persons 91 and 92 displayed on the screen of the display 63 with the finger 95. The signal processing unit 50 uses the sound picked up by the microphone array 20, that is, the sound data picked up by each microphone 22, from the position of each microphone 22 of the microphone array 20, and the touch point 63a designated by the user or Audio data in which directivity is formed in the directivity direction (direction indicated by the symbol e shown in FIG. 7A) toward the audio position corresponding to the center of the rectangular range 63b is generated.

  In other words, the signal processing unit 50 uses the audio data collected by each microphone 22 and the audio corresponding to the center of the touch point 63a or the rectangular range 63b specified by the user from the position of each microphone 22 of the microphone array 20. Sound data in which the sound (volume level) in the directivity direction toward the position is emphasized (amplified) is generated. The reproduction unit 60 outputs the audio data generated by the signal processing unit 50 from the speaker 65 in synchronization with the video data captured by the camera 10.

  As a result, the audio data at the touch point 63a or the rectangular range 63b designated by the user is emphasized, and the conversation between the two persons 91 and 92 from the speaker 65 (see, for example, “Hello” shown in FIG. 7A) is large. Sound is output according to the volume. On the other hand, music that is placed from a speaker 82 that is placed closer to the microphone array 20 than the two persons 91 and 92 but is not included in the rectangular range 63b specified by the user (FIG. 7A). The voice is not output with emphasis, but is output with a lower sound volume than the conversation between the two persons 91 and 92.

  FIG. 7C is a diagram illustrating a state in which video data captured by the camera 10 </ b> A is displayed on the display 63 and audio data is output as audio from the speaker 65. On the screen of the display 63, video data captured by the camera 10A is displayed. Further, the speaker 65 outputs voice of conversation between two persons 93 and 94 or music in the hall.

  It is assumed that the user touches the vicinity of the center of the video data of the two persons 93 and 94 displayed on the screen of the display 63 with the finger 95, for example. The signal processing unit 50 uses the sound picked up by the microphone array 20, that is, the sound data picked up by each microphone 22, from the position of each microphone 22 of the microphone array 20, and the touch point 63c specified by the user or Audio data in which directivity is formed in the directivity direction (direction indicated by the symbol f shown in FIG. 7A) toward the audio position corresponding to the center of the rectangular range 63d is generated.

  In other words, the signal processing unit 50 uses the audio data collected by each microphone 22 and the audio corresponding to the center of the touch point 63c or the rectangular range 63d designated by the user from the position of each microphone 22 of the microphone array 20. Sound data in which the sound (volume level) in the directivity direction toward the position is emphasized (amplified) is generated. The reproduction unit 60 outputs the audio data generated by the signal processing unit 50 from the speaker 65 in synchronization with the video data captured by the camera 10A.

  As a result, the voice data at the touch point 63c or the rectangular range 63d designated by the user is emphasized, and the conversation between the two persons 91 and 92 from the speaker 65 (see, for example, “Hi” shown in FIG. 7A) is large. Sound is output according to the volume. On the other hand, compared to the two persons 93 and 94, music that is placed from a speaker 82 that is placed closer to the microphone array 20 but is not included in the rectangular range 63d designated by the user (FIG. 7A). Are emphasized and are not output in voice, but are output with a lower sound volume than in the conversation between two persons 93 and 94.

  As described above, in the present embodiment, the audio processing system 5C is designated for the video data in one of the cameras 10 or 10A designated by the user during the reproduction of the video data and audio data recorded in the recorder 45. In addition, audio data in video for an arbitrary reproduction time can be emphasized and output. As a result, the user simply touches and designates a portion where the sound (volume level) is to be emphasized (amplified) while viewing the video data captured by the camera 10 or 10A on the display 63, and the designated designation can be easily performed. The voice data in the designated range including the location or the designated location can be emphasized and outputted. As described above, in the audio processing system 5C according to the present embodiment, the user can easily obtain audio information within a necessary range while viewing the video data captured by the camera 10 or 10A on the display 63. it can.

  Further, in the present embodiment, a plurality of cameras may be installed in the sound processing system 5C as compared with the first embodiment, so that it is not necessary to increase the number of microphone arrays in accordance with the number of cameras, and the cost can be reduced. The possible voice processing system 5C can be constructed, and the space for the voice processing system 5C can be saved. Further, the voice processing system 5C can be obtained by adding the second camera 10A to the voice processing system 5A or 5B in which the first camera 10 has already been installed. Operations and effects similar to those of 5B can be obtained, and the expandability of the voice processing system can be improved.

(Third embodiment)
In each of the first and second embodiments, an example of a usage pattern of the voice processing system 5A or 5B in which the camera and the microphone array are installed in different places on the ceiling has been described. In the third embodiment, an example of a usage pattern of a voice processing system 5D in which an omnidirectional camera and a microphone array are integrally installed on the same axis will be described.

  The voice processing system 5D of the third embodiment is the same as the voice processing system 5A or the voice processing system 5B of the first embodiment, except that the omnidirectional camera and the microphone array are integrally installed on the same axis. Since they have the same configuration, the same components as those in the speech processing system 5A or 5B of the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  FIG. 9 is a schematic diagram illustrating an example of a usage pattern of the voice processing system 5D. FIG. 9A shows a state in which, for example, a donut-shaped microphone array 20C, an omnidirectional camera 10E integrated with the microphone array 20C, and a speaker 83 are installed on the ceiling 85 of an indoor hall. FIG. In FIG. 9A, it is assumed that the conversation situations of the persons 91, 92, 93, and 94 and the operation situations of the speakers 82 and 83 are the same as those in the second embodiment.

  FIG. 9B is a diagram illustrating a state in which two persons 91 and 92 are selected in the video data captured by the omnidirectional camera 10E. In FIG. 9B, video data using a coordinate system in the omnidirectional camera 10E, that is, video data captured by the omnidirectional camera 10E is displayed on the screen of the display 63 as it is. FIG. 9C is a diagram showing a state in which the video data of the two persons 91 and 92 after the image conversion is displayed on the display, and the voice data of the conversations of the persons 91 and 92 is output as audio from the speaker 65. is there.

  For example, it is assumed that the user touches the designated portion near the upper left of the video data of the four persons 91, 92, 93, 94 displayed on the screen of the display 63 with the finger 95. In addition to the same operation as that of the second embodiment, the signal processing unit 50 performs processing of video data in a range of code g including a designated portion designated by the user from a wide range of video data captured by the omnidirectional camera 10E. Convert the coordinate system. The reproduction unit 60 causes the display 63 to display the video data obtained by converting the coordinate system by the signal processing unit 50 (see FIG. 9C). Note that the range g is automatically generated from the touch point of the finger 95. The description of the operation of the signal processing unit 50 similar to that of the second embodiment is omitted.

  As a result, the voice data in the range g designated by the user is emphasized, and the conversation between the two persons 91 and 92 (see, for example, “Hello” shown in FIG. 9A) is output from the speaker 65 with a high volume. The On the other hand, it flows from the speaker 82 that is placed closer to the microphone array 20C than the two persons 91 and 92 but is not included in the specified location designated by the user or the designated range g including the designated location. The music (see “♪ ˜” shown in FIG. 9A) is not emphasized and is not output as a sound, but is output as a sound with a smaller volume compared to the conversation between the two persons 91 and 92.

  FIG. 9D is a diagram showing a state in which two persons 93 and 94 are selected in the video data captured by the omnidirectional camera 10E. In FIG. 9D, video data using a coordinate system in the omnidirectional camera 10E, that is, video data captured by the omnidirectional camera 10E is displayed on the screen of the display 63 as it is. FIG. 9E is a diagram showing a state in which the video data of the two persons 93 and 94 after the image conversion is displayed on the display, and the voice data of the conversations of the persons 93 and 94 is output as audio from the speaker 65. is there.

  For example, it is assumed that the user touches a designated portion near the lower right of the video data of four persons 91, 92, 93, 94 displayed on the screen of the display 63 with the finger 95. In addition to the same operation as that of the second embodiment, the signal processing unit 50 performs processing of video data in a range of code h including a designated portion designated by the user from a wide range of video data captured by the omnidirectional camera 10E. Convert the coordinate system. The reproducing unit 60 causes the display 63 to display the video data obtained by converting the coordinate system by the signal processing unit 50 (see FIG. 9E). The range h is automatically generated from the touch point of the finger 95. The description of the operation of the signal processing unit 50 similar to that of the second embodiment is omitted.

  As a result, the audio data in the range h designated by the user is emphasized, and the conversation between the two persons 93 and 94 (see, for example, “Hi” shown in FIG. 9A) is output from the speaker 65 with a high sound volume. The On the other hand, it flows from the speaker 82 that is placed closer to the microphone array 20C than the two persons 93 and 94 but is not included in the specified location designated by the user or the designated range h including the designated location. The existing music (see “♪ ˜” shown in FIG. 9A) is not emphasized and is not output as a sound, but is output as a sound with a smaller volume compared to the conversation between the two persons 93 and 94.

  As described above, in this embodiment, in the audio processing system 5D, the omnidirectional camera 10E and the microphone array 20C are arranged on the same axis, so that the coordinate systems of the omnidirectional camera 10E and the microphone array 20C are the same. Can do. Thereby, in addition to the effects of the first and second embodiments, the audio processing system 5D adds the position of the subject in the video data picked up by the omnidirectional camera 10E and the person of the subject picked up by the microphone array 20C. Compared to the first and second embodiments, the coordinate system conversion process for associating the audio direction can be facilitated, and the load of the reproduction process that synchronizes the video data and the audio data in the reproduction unit 60 can be reduced. .

  Also, the audio processing system 5D converts the video data included in the designated area designated by the user or the designated range g including the designated place or the designated range h into video data that matches the screen size of the display 63. The video data captured by the omnidirectional camera 10 </ b> E can be displayed in a video data display mode in which the aspect ratio is natural for the display 63.

  For example, the shape and configuration of the microphone array are not limited to those of the above-described embodiments, and various shapes and configurations may be used. FIGS. 10A to 10C are external views of other microphone arrays 20D, 20E, and 20F.

  In the microphone array 20D shown in FIG. 10A, the diameter of the disk-shaped housing 21D is smaller than that of the microphone array 20 shown in FIG. A plurality of microphones 22D are uniformly arranged along a circle on the surface of the housing 21D. Since the interval between the microphones 22D is shortened, the microphone array 20D has characteristics suitable for a high sound range.

  In the microphone array 20E shown in FIG. 10B, a plurality of microphones 22E are uniformly arranged along the rectangle on the surface of the rectangular casing 21E. Since the casing 21E is formed in a rectangular shape, the microphone array 20E can be easily installed even at a corner or the like.

  In the microphone array 20F shown in FIG. 10C, a plurality of microphones 22F are uniformly arranged vertically and horizontally on the surface of the disk-shaped casing 21F. Since the plurality of microphones 22F are arranged in a straight line, the sound enhancement processing in the signal processing unit 50 can be simplified. A plurality of microphones 22F may be arranged in only one column in the vertical direction or the horizontal direction.

  Further, in each of the above-described embodiments, the user designates a designated portion where the user wants to emphasize audio or a designated range including the designated portion by touching with the finger 95 while viewing the video data displayed on the display 63. For example, the screen of the display 63 may be divided in advance into a plurality of sections (for example, four sections on the top, bottom, left, and right), and any one section may be selected as a range in which the voice is to be emphasized.

  In each of the above-described embodiments, a case has been described in which the camera records (records) video and the display displays recorded video data. However, the camera captures still images at a predetermined cycle, and the display The present invention can also be applied when displaying still images picked up at intervals, that is, when picking up sound by picking up video in real time. That is, the user can designate a predetermined range in the still image displayed on the screen of the display and emphasize the sound in the vicinity thereof.

  In each of the above-described embodiments, the user touches the screen with the finger 95 to specify a specified range (for example, an ellipse or a rectangular range) including the touch point where the finger 95 is touched. A predetermined range may be designated by drawing a circle, a polygon or the like at 95.

  In each of the above-described embodiments, the signal processing unit 50 may receive a designation of a plurality of designated locations or a designated range (speech enhancement range) including each designated location from the operation unit 55. In this case, the signal processing unit 50 performs audio data enhancement processing according to each designated location or designated range. FIG. 11 is a schematic diagram illustrating operations of the display 63 and the speaker 65 when a plurality of predetermined designated locations or designated ranges (speech enhancement ranges) are designated. In order to simplify the description, the operation status of the camera and the microphone array in which the voice processing system is used is the same as the operation status of the camera 10 and the microphone array 20 shown in FIG.

  In this case, the signal processing unit 50 receives two persons from the positions of the microphones 22 of the microphone array 20 in accordance with the designation of the voice emphasizing ranges 63e and 63f including two predetermined different designated places or different designated places from the speaker 65. Voice data having directivity formed in the directivity direction toward the voice position corresponding to the center of the persons 91 and 92, and further, the voice position corresponding to the center of the speaker 82 from the position of each microphone 22 of the microphone array 20 Generate voice data with directivity in the direction toward.

  As a result, both the conversation between the two persons 91 and 92 (see “Hello” shown in FIG. 11) and the music flowing from the speaker 82 (see “♪ ˜” shown in FIG. 11) are output as audio with a high volume. The Thereby, the voice processing system can emphasize two or more voices on one display.

  Next, an example of the housing structure of the microphone array 20 and the circuit configuration of the microphone array 20 in each embodiment described above will be described with reference to FIGS.

(Mic array housing: quadruple housing structure)
FIG. 12 is an exploded perspective view of the housing structure of the microphone array 20 of each embodiment described above. FIG. 13A is a plan view of the housing structure of the microphone array 20 shown in FIG. FIG. 13B is a cross-sectional view taken along the line AA in FIG. FIG. 14 is an enlarged view of a main part of the dotted line range in FIG.

  The housing structure of the microphone array 20 shown in FIG. 12 has a configuration in which a main housing 101, a punching metal cover 103, a microphone sheet metal 105, and a base sheet metal 107 are stacked in the vertical direction. The main casing 101, the punching metal cover 103, the microphone sheet metal 105, and the base sheet metal 107 constitute an impact resistant casing 109 (vandal-resistant casing) having four layers.

  The main casing 101 is integrally formed using, for example, a resin as a material. The main casing 101 is formed in a bottomed cylindrical shape in which a plurality of microphone laying holes 113 are provided concentrically on the annular bottom 111. A central portion of the annular bottom portion 111 becomes a camera mounting space 115. The main casing 101 has a main casing outer peripheral wall 117 having a maximum outer diameter in the casing structure of the microphone array 20 shown in FIG.

  The punching metal cover 103 is formed into an integral annular shape using, for example, a metal as a material. The punching metal cover 103 is attached to the main casing 101 so as to cover the annular bottom portion 111 of the main casing 101. The punching metal cover 103 has a large number of through holes (not shown) through which sound waves are incident. On the outer periphery of the punching metal cover 103, a rising edge portion 119 rising toward the main casing 101 is formed by drawing or the like. The standing edge 119 is inserted into a circumferential groove 121 (see FIG. 14) formed on the outer periphery of the lower surface of the main housing 101. A plurality of elastic locking claws 123 protrude further upward (upward in FIG. 12 or FIG. 14) at equal intervals in the circumferential direction on the standing edge 119.

  FIG. 15A is a perspective view showing how the punching metal cover 103 is fixed to the main casing 101. FIG. 15B is a cross-sectional view showing a state in which the punching metal cover 103 is fixed to the main housing 101. The elastic locking claw 123 is locked to the claw locking portion 125 by rotating through a locking hole 125 a provided on the back side of the circumferential groove 121. The punching metal cover 103 is fixed to the main casing 101 by locking the elastic locking claws 123 to the claw locking portions 125.

  The microphone sheet metal 105 is formed, for example, by pressing a metal plate. The microphone sheet metal 105 is formed in a shape obtained by dividing an annular shape into four equal parts in the circumferential direction. The microphone sheet metal 105 is fixed to the main housing 101 by a microphone sheet metal fixing screw (not shown). The microphone sheet metal 105 fixed to the main casing 101 holds the microphone casing 129 holding the microphone substrate 127 between the annular bottom 111 of the main casing 101.

  The microphone casing 129 is integrally formed using, for example, a resin. The microphone casing 129 is formed in a shape obtained by dividing an annular shape into four equal parts in the circumferential direction. Four high sound quality small electret condenser microphones (ECM) are mounted on the same surface of the microphone substrate 127. A microphone substrate 127 is attached to the microphone casing 129 in a state where the ECM 131 is below in FIG. A rubber component is sandwiched between the microphone substrate 127 and the microphone casing 129 (see FIG. 14). One microphone substrate 127 is attached to the microphone casing 129. Accordingly, a total of four microphone substrates 127 are attached in the entire housing structure of the microphone array 20, and a total of 16 ECMs 131 are equipped in the entire housing structure of the microphone array 20.

  Therefore, in the case structure of the microphone array 20 shown in FIG. 12, the punching metal cover 103, the main case 101, the microphone case 129, the microphone sheet metal 105, and the base sheet metal 107 are sequentially shown in FIG. It is arranged toward the direction. These members constitute a structure that opposes the external force (impact force) from the lower side of the microphone array 20 shown in FIG. For example, since the main housing 101 and the microphone housing 129 are not integrated but separated, the external force (impact force) from below shown in FIG. And the deformation | transformation of the microphone housing | casing 129 can be prevented. Thereby, even after an external force is applied, it is possible to maintain the shape of the microphone array 20 during sound collection, and it is possible to prevent deterioration of the acoustic characteristics during sound collection of the microphone array 20.

  The base sheet metal 107 is integrally formed by, for example, pressing (drawing) a metal material. The base metal plate 107 has an annular top plate portion 133 and is formed in a bottomed cylindrical shape. That is, the base sheet metal outer peripheral wall 135 is bent downward from the outer periphery of the annular bottom portion 111. The base sheet metal outer peripheral wall 135 is obtained by drawing a base plate of the large-diameter annular top plate portion 133. The base sheet metal 107 obtained by drawing the outer peripheral wall 135 of the base sheet metal has higher strength than other components.

  The base metal plate 107 is fixed to the main housing 101 with a base metal plate fixing screw (not shown). For example, a component for controlling the processing of the microphone array 20 is mounted between the base sheet metal 107 and the microphone sheet metal 105, and a component for supplying power to each part of the microphone array 20, for example. And a power supply board 141 on which are mounted. The main board 139 and the power supply board 141 are provided one by one in the entire housing structure of the microphone array 20 shown in FIG.

  From the microphone sheet metal 105, a plurality of fitting portions 143 stand up at equal intervals in the circumferential direction. The fitting portion 143 includes a pair of holding pieces (an outer holding piece 145 and an inner holding piece 147) that are spaced apart in the radial direction. The fitting portion 143 is disposed with a gap 149 inside the outer peripheral wall 117 of the main housing. A base sheet metal outer peripheral wall 135 is fitted to the fitting portion 143. That is, in the case structure of the microphone array 20 shown in FIG. 12, the main case outer peripheral wall 117, the gap 149, the outer holding piece 145, the base sheet metal outer peripheral wall 135, and the inner holding piece 147 are arranged in the radial direction in this order from the outer side. It is arranged toward the inside. A plurality of these stacked members constitute a structure that resists external force (impact force) from the side of the microphone array 20.

  Further, the microphone sheet metal 105 has a stopper portion 137 that stands up and protrudes, and is usually located away from the base sheet metal 107, but when the main casing 101 is deformed by an external force, the stopper portion 137 hits the base metal plate 107 and works so that a large distortion does not occur in the main casing 101.

(Direct mounting structure of ECM)
FIG. 16 is a schematic diagram of an ECM mounting structure. In the housing structure of the microphone array 20 shown in FIG. 12, the microphone substrate 127 is disposed below the microphone sheet metal 105, and the main substrate 139 and the power supply substrate 141 are disposed above the microphone sheet metal 105. That is, the microphone substrate 127, the main substrate 139, and the power supply substrate 141 are arranged in a two-story structure. Here, in the four microphone boards 127, the first microphone board 127, the second microphone board 127, the third microphone board 127, and the fourth microphone board 127 are sequentially arranged in one direction around the circumference. And In this case, the main board 139 is connected to the first microphone board 127 and the fourth microphone board 127 by the power supply wiring 151. The first microphone substrate 127 is connected to the second microphone substrate 127. The fourth microphone substrate 127 is connected to the third microphone substrate 127.

  An ECM 131 is attached to the lower surface side of the microphone substrate 127. A pair of pin terminals 153 protrudes from the ECM 131. Each pin terminal 153 of the ECM 131 is inserted into a terminal pin insertion hole (not shown) provided in a predetermined circuit of the microphone substrate 127, and is directly connected and fixed by, for example, solder. Thereby, the thickness (low profile) of the ECM 131 with respect to the microphone substrate 127 is realized. In addition, the material cost is reduced by directly attaching the ECM 131 to the microphone substrate 127.

(ADC converter arrangement)
FIG. 17 is a plan view of the microphone substrate 127. Four ECMs 131 are attached to one microphone substrate 127 shown in FIG. In the circuit of the microphone substrate 127 (microphone substrate circuit), the difference in the line length connected to each ECM 131 causes a phase difference in the sound wave signal, and as a result, this phase difference becomes a deviation of the directivity angle. For this reason, the line length connected to each ECM 131 needs to be as equal as possible.

  Therefore, in the microphone substrate 127, a microphone substrate circuit is configured by a combination of two ECMs 131 and one AD converter 155. The microphone substrate circuit is configured such that one AD converter 155 is arranged between two ECMs 131 at an equal distance from each ECM 131, so that an analog line 157 between the AD converter 155 and the ECM 131 passes through an amplifier circuit. It is wired so that it is the shortest and has the same line length. Thereby, the microphone substrate circuit can equalize the level of the noise signal in the microphone substrate 127 in each ECM, and can reduce the deviation of the directivity angle.

(Microphone board circuit)
FIG. 18A shows a diagram of a microphone substrate circuit in which one ripple removing circuit 161 is provided for a plurality of microphone circuits 159. FIG. 18B is a diagram of a microphone substrate circuit in which a ripple removal circuit 161 is provided in each of the plurality of microphone circuits 159.

  In the microphone substrate circuit of the microphone substrate 127, a ripple removing circuit 161 is provided between the microphone circuit 159 where the ECM is arranged and the power supply substrate 141. The ripple removal circuit 161 is a filter that passes a DC signal but cuts an AC signal having a specific frequency. As shown in FIG. 18A, one ripple removing circuit 161 can be provided between the four microphone circuits 159 connected in parallel and the power supply substrate 141. In this case, the manufacturing cost of the microphone array 20 can be reduced.

  On the other hand, the ripple removal circuit 161 may be provided between each of the four microphone circuits 159 and the power supply board 141 as shown in FIG. In this case, signal inflow between different ECMs is reduced, and so-called crosstalk 163 can be suppressed.

(Measures for structural gaps between the microphone array and the camera)
FIG. 19A is a perspective view of the housing structure of the microphone array 20 in which the omnidirectional camera is attached without the camera adapter being attached. FIG. 19B is a perspective view of the housing structure of the microphone array 20 to which the outdoor omnidirectional camera 165 is attached together with the camera adapter. FIG. 20 is an exploded perspective view of the housing structure of the microphone array 20 to which the indoor omnidirectional camera 167 is attached. FIG. 21 is an exploded perspective view of the housing structure of the microphone array 20 to which the outdoor omnidirectional camera 165 is attached. FIG. 22A is a side view of the housing structure of the microphone array 20 to which the outdoor omnidirectional camera 165 is attached. FIG. 22B is a cross-sectional view taken along the line BB in FIG. FIG. 23 is an enlarged view of a main part of FIG.

  In the housing structure of the microphone array 20, for example, an omnidirectional camera can be incorporated in the camera mounting space 115 in the center. The omnidirectional camera includes an outdoor omnidirectional camera 165 and an indoor omnidirectional camera 167. As shown in FIG. 19A, when the indoor omnidirectional camera 167 is attached to the camera mounting space 115 as the housing structure of the microphone array 20, for example, the main housing 101 of the microphone array 20 and the indoor omnidirectional camera are arranged. A gap 169 is generated between the microphone array 167 and the inside of the microphone array 20 can be seen. The state where the inside can be seen is not only the deterioration of the appearance of the product and the entry of dust etc., but also the sound enters the internal space of the microphone array 20 and causes resonance and reflection, which causes deterioration of acoustic performance. turn into.

  There are various sizes of omnidirectional cameras depending on applications and functions. For each omnidirectional camera, it is inevitable to increase the manufacturing cost to prepare the main casing 101 having a different size. Manufacturing cost can be reduced by fixing the main casing 101 to one size and closing the gap by using a camera adapter to prevent the gap between the omnidirectional camera models.

  Therefore, as shown in FIG. 19B, for example, when the outdoor omnidirectional camera 165 is attached to the camera mounting space 115, the outdoor camera adapter 171 is attached around the outdoor omnidirectional camera 165. As shown in FIG. 20, when the indoor omnidirectional camera 167 is attached to the camera mounting space 115, the indoor camera adapter 173 is attached around the indoor omnidirectional camera 167. The indoor camera adapter 173 is formed in a cylindrical shape using, for example, resin. The indoor camera adapter 173 has a flange 175 for concealing a gap formed at the lower end of the indoor camera adapter 173, and the flange 175 has the indoor omnidirectional camera 167 and the main casing 101 that are generated when the indoor omnidirectional camera 167 is attached to the camera mounting space 115. The gap 169 between the two is hidden.

  In the indoor camera adapter 173, a plurality of peripheral wall elastic claws 177 are formed in the plurality of cuts 179 at equal intervals along the circumferential direction. The indoor camera adapter 173 is attached by locking the peripheral wall elastic claw 177 to the camera casing 181 of the indoor omnidirectional camera 167. On the base sheet metal 107, a plurality of camera fixing sheet metal parts 183 shown in FIG. 22 are formed at equal intervals along the circumferential direction. The camera fixing sheet metal portion 183 has a dharma hole 185 and is disposed above the camera mounting space 115. An engaging pin (not shown) having a large-diameter head (not shown) that engages with the dharma hole 185 of the camera fixing sheet metal part 183 is projected on the upper surface of the camera housing 181. The indoor omnidirectional camera 167 to which the indoor camera adapter 173 is attached is inserted into the camera attachment space 115 and rotated, whereby the engagement pin engages with the dharma hole 185 and is supported by the fall being restricted. The At this rotational position, the indoor omnidirectional camera 167 is locked to the main casing 101 or the like of the microphone array 20 by a camera rotation restricting screw (not shown). Further, in a state where the indoor omnidirectional camera 167 is locked, the peripheral wall elastic claws 177 are prevented from being locked by the camera fixing sheet metal portion 183 because the inner peripheral wall of the main housing 101 becomes an obstacle.

  On the other hand, on the outer periphery of the outdoor camera adapter 171 shown in FIG. 21, a bayonet plate 187 having a free end is provided. At the free end of the bayonet plate 187, an adapter rotation restricting claw 189 (see FIG. 23) protruding inward in the radial direction is formed. The adapter rotation restricting claw 189 engages with a bayonet engaging groove 191 formed in the camera housing 181. Other structures are the same as the indoor camera adapter 173. When the outdoor camera adapter 171 incorporated in the camera mounting space 115 is rotated, the adapter rotation restricting claw 189 engages with the bayonet engaging groove 191 and the rotation is restricted as shown in FIG. That is, relative rotation between the outdoor camera adapter 171 and the outdoor omnidirectional camera 165 is restricted. A tool insertion groove 193 is formed in the flange 175 of the outdoor camera adapter 171. When the outdoor omnidirectional camera 165 is pushed into the camera mounting space 115, there is no means for rotating. Therefore, a screwdriver or the like can be inserted into the tool insertion groove 193 and rotated.

(Lid used when using separate microphone array and omnidirectional camera)
FIG. 24 is an exploded perspective view of the housing structure of the microphone array 20 to which the lid 195 is attached. The microphone array 20 and the omnidirectional camera may be used by being integrally attached as shown in FIG. 7A, for example, but are attached separately as shown in FIG. 9A, for example. Sometimes used. In this case, the camera mounting space 115 is closed by a lid 195 shown in FIG. The lid 195 is integrally formed using, for example, a resin as a material. Further, the lid 195 is integrally combined by a locking structure with a metal lid sheet metal 197 or the like. The lid 195 is combined with the lid sheet metal 197 to disperse external force (impact force) to the lid sheet metal 197. As a result, the lid 195 is prevented from being cracked and the like by preventing large deformation of the lid 195 itself. The lid 195 is inserted into the camera mounting space 115 in combination with the lid sheet metal 197, and the lid sheet metal 197 is supported by engaging the camera fixing sheet metal portion 183 for fixing the omnidirectional camera. In this state, the lid 195 is rotationally stopped and fixed to the camera fixing sheet metal portion 183 by the lid rotation set screw 199.

(Mounting bracket)
FIG. 25 is an exploded perspective view of the housing structure of the microphone array 20 that is attached to the ceiling using the mounting bracket 201. FIG. 26A is a side view of the base sheet metal side fixing pin 205 before being inserted into the base sheet metal fixing hole 203. FIG. 26B is a side view of the base sheet metal side fixing pin 205 inserted into the base sheet metal fixing hole 203. FIG. 26C is a plan view of the base metal plate-side fixing pin 205 inserted into the base metal plate fixing hole 203. FIG. 26D is a side view of the base sheet metal side fixing pin 205 moved to the small diameter hole 207 of the base sheet metal fixing hole 203. FIG. 26E is a plan view of the base sheet metal side fixing pin 205 moved to the small diameter hole 207 of the base sheet metal fixing hole 203.

  The impact-resistant housing 109 (see FIG. 12) is attached to a ceiling surface (not shown) as an example of an installation surface using a mounting bracket 201. That is, the mounting bracket 201 is fixed to the ceiling surface, and an impact resistant casing 109 having a casing structure is mounted on the mounting bracket 201.

  As shown in FIG. 25, the mounting bracket 201 as an example of a mounting tool has a circular bracket base. However, the fixture is not limited to the metal fixture 201, and the material of the fixture may be, for example, ceramics or synthetic resin (for example, plastic or elastomer). A plurality of (for example, three) base metal plate fixing holes 203 are formed in the metal base. The base sheet metal fixing hole 203 is formed in a dharma shape or a loofah shape in which a small diameter hole 207 and a large diameter hole 209 are connected.

  On the other hand, on the surface of the base sheet metal 107 facing the ceiling surface, a base sheet metal side fixing pin 205 protrudes corresponding to the base sheet metal fixing hole 203. As shown in FIG. 26A, the base metal plate fixing pin 205 has a large-diameter pin head 211 at the protruding tip. The large-diameter pin head 211 can be inserted into the large-diameter hole 209, and the small-diameter hole 207 is restricted from being detached and can be locked.

Next, a method for attaching the impact-resistant housing 109 will be described.
First, in order to attach the impact-resistant housing 109 to a ceiling surface as an example of an installation surface, the mounting bracket 201 is fixed to a predetermined position on the ceiling surface with a ceiling fixing screw (not shown). The impact-resistant housing 109 is concentrically aligned with the mounting bracket 201 fixed to the ceiling surface.

  Next, as shown in FIGS. 26B and 26C, the large-diameter pin head 211 of the base metal plate-side fixing pin 205 is inserted into the large-diameter hole 209 of the base metal plate fixing hole 203 (see FIG. 26 (B) and FIG. 26 (C)).

  Thereafter, as shown in FIGS. 26 (D) and 26 (E), the impact-resistant housing 109 is rotated, and the large-diameter pin head 211 is moved to the small-diameter hole 207. The side fixing pins 205 are simultaneously fixed in the base sheet metal fixing holes 203. As described above, the outdoor omnidirectional camera 165 and the indoor omnidirectional camera 167 are attached to the camera mounting space 115 of the impact-resistant housing 109 fixed to the ceiling surface via the mounting bracket 201.

  As described above, in the case structure of the microphone array 20, the omnidirectional camera is directly attached to the impact-resistant case 109 fixed to the ceiling surface by the mounting bracket 201. Thereby, since the omnidirectional camera is directly attached to the base sheet metal 107 to which the microphone sheet metal 105 is fixed, the housing structure of the microphone array 20 can improve the positional accuracy of the ECM 131 and the omnidirectional camera.

(Suppression of reflected sound)
FIG. 27 is a cross-sectional view of the housing structure of the microphone array 20 in which the taper 223 is provided in the ECM recess 213. As shown in FIG. 27, the housing structure of the microphone array 20 has a taper 223 in which the inner peripheral surface of the ECM recess 213 is reduced in diameter toward the ECM 131. The taper 223 has a minimum diameter that substantially matches the outer diameter of the circular convex portion of the cushioning material 217 into which the ECM 131 is inserted, and a maximum diameter that substantially matches the microphone laying hole 113 in the annular bottom portion 111. In the ECM recess 213 formed with the taper 223, the resonance point of the air column rises. Further, the reflected wave on the inner peripheral surface of the ECM recess 213 does not go to the ECM 131. Furthermore, it reaches the ECM 131 in a state where the sound waves from the lateral direction of the casing are not disturbed. As a result, the usable sound range is widened, and the acoustic characteristics of the microphone array 20 during sound collection are improved. Further, a nonwoven fabric 221 for reducing wind noise is sandwiched between the punching metal cover 103 and the annular bottom portion 111.

(Wind measures)
FIG. 28 is a cross-sectional view of the housing structure of the microphone array 20 that is provided with wind countermeasures. In the microphone array 20, a plurality of ECM recesses 213 are formed in the microphone casing 129 in accordance with the ECM 131. The ECM recess 213 is formed, for example, in a circular shape, and a through hole 215 for exposing the ECM 131 is formed at the center. The ECM 131 is attached to the microphone casing 129 with a buffer material 217 such as rubber wound around the outer periphery, and the tip of the ECM 131 is inserted into the through hole 215. The ECM recess 213 is arranged concentrically with the microphone laying hole 113 formed in the annular bottom 111. The ECM recess 213 can be filled with a sound absorbing material 219 for wind countermeasures. The surface of the sound absorbing material 219 is covered with a nonwoven fabric 221. The nonwoven fabric 221 is sandwiched between the punching metal cover 103 and the annular bottom portion 111.

  Next, modified examples of the ECM recess 213 will be described with reference to FIGS. FIG. 29A is a cross-sectional view of the housing structure of the microphone array 20 showing the relationship between the inner diameter and depth of the ECM recess 213. FIG. 29B is a cross-sectional view of the housing structure of the microphone array 20 in which the inner wall of the ECM recess 213 is an inclined wall 225. FIG. 29C is a cross-sectional view of the housing structure of the microphone array 20 in which the inner peripheral corner of the ECM recess 213 is an R portion 227.

  As shown in FIG. 29A, it is preferable that the diameter D and the depth H of the ECM recess 213 have a predetermined relationship. For example, by satisfying the relationship of H / D <1/10, the peak can be suppressed in the vicinity of the resonance frequency of the ECM recess 213, so that the acoustic performance is not adversely affected.

  As shown in FIG. 29B, the ECM recess 213 may be formed by a flat recess bottom surface 229 and a tapered inclined wall 225. As a result, the resonance frequency of the ECM recess 213 can be made higher than the operating frequency band, and reflected waves from the inner peripheral surface of the ECM recess 213 toward the ECM 131 can be reduced.

  As shown in FIG. 29C, the ECM recess 213 may have an inner peripheral corner as an R portion 227. Also by this, the resonance frequency of the ECM recess 213 can be made higher than the use frequency band, and the reflected wave from the inner peripheral surface of the ECM recess 213 toward the ECM 131 can be reduced.

  FIG. 30A is an explanatory view showing the isobaric surface of the ECM recess 213 in which the taper 223 is not formed. FIG. 30B is an explanatory diagram showing the isobaric surface of the ECM recess 213 in which the taper 223 is formed.

  The sound in the vicinity of the ECM 131 can be simulated, for example, by analyzing the sound transmitted through the space based on the wave equation by the finite element method. In this case, in a model in which the ECM recess 213 is not provided with the taper 223, the space between the isobaric surfaces is different between the housing surface 231 and the ECM portion 233, as shown in FIG. On the other hand, in the model in which the ECM recess 213 is provided with the taper 223, the space between the isobaric surfaces is the same between the housing surface 231 and the ECM portion 233, as shown in FIG. Thereby, the taper 223 is provided in the ECM recess 213, so that the sound wave reaches the ECM 131 without being disturbed.

Next, the effect | action of the housing structure of the microphone array 20 of each embodiment mentioned above is demonstrated.
In the case structure of the microphone array 20 of each of the embodiments described above, a metal microphone sheet metal 105 and a bottomed cylindrical metal base sheet metal are formed on a resin main casing 101 formed in a bottomed cylindrical shape. 107 is fixed. The metal microphone sheet metal 105 has a stopper 137 standing on the base sheet metal 107 side. A metal punching metal cover 103 is fixed to the main casing 101 on the opposite side of the microphone sheet metal 105 with the main casing 101 interposed therebetween.

  In the case structure of the microphone array 20 of each embodiment described above, impact energy from the outside is absorbed by deforming the resin main case 101. Impact energy exceeding the breaking strength of the main casing 101 is absorbed by deforming the metal microphone sheet metal 105. Further, the impact energy that plastically deforms the microphone sheet metal 105 to a predetermined amount or more is applied to the base sheet metal 107 through the contact stop portion 137 and is finally released to the building frame or the like to which the base sheet metal 107 is attached.

  Further, in the case structure of the microphone array 20 of each embodiment described above, the punching metal cover 103, the main case 101, the microphone sheet metal 105, and the base sheet metal 107 made of separate members are integrally fixed and assembled. . For this reason, the impact energy from the outside is absorbed and reduced by the gap 149 between these members and friction caused by friction.

  Further, in the case structure of the microphone array 20 of each embodiment described above, the microphone substrate 127 is sandwiched between the punching metal cover 103 and the microphone sheet metal 105. The main board 139 and the power board 141 are sandwiched between the microphone sheet metal 105 and the base sheet metal 107. That is, the microphone sheet metal 105 is electromagnetically shielded by the conductive outer shell formed by the metal punching metal cover 103 and the metal microphone sheet metal 105. The main board 139 and the power supply board 141 are electromagnetically shielded by a conductive outer shell formed by the metal microphone sheet metal 105 and the metal base sheet metal 107.

  In the case structure of the microphone array 20 of each embodiment described above, the microphone case 129 sandwiched between the resin main case 101 and the metal microphone sheet metal 105 is made of a resin material. A plurality of microphones are fixed to the microphone casing 129. The microphone fixed to the microphone casing 129 is opened to the outside through a microphone laying hole 113 opened in the annular bottom portion 111 of the main casing 101. The microphone laying hole 113 is covered with a punching metal cover 103 that covers the annular bottom portion 111.

  For example, when the impact-resistant housing 109 is fixed to the ceiling surface, the punching metal cover 103 is disposed on the side facing the ground. First, an impact such as impact applied to the impact-resistant housing 109 from the ground side is applied to the punching metal cover 103. The metal punching metal cover 103 is plastically deformed by an impact exceeding the elastic limit and absorbs the impact energy. The impact energy that has not been absorbed by the plastic deformation of the punching metal cover 103 is applied to the annular bottom 111 of the main housing 101. The impact energy deforms the annular bottom portion 111 and is applied to the microphone sheet metal 105 and the base sheet metal 107. Since the microphone casing 129 is fixed to the microphone sheet metal, a large impact energy is not applied.

  If the impact energy at this time is equal to or greater than the elastic limit of the resin main casing 101, the main casing 101 causes whitening, cracks, and the like, and absorbs the impact energy. The main casing 101 is whitened or cracked, but is restored to its original shape with whitening or cracking unless the entire case is completely destroyed. That is, the main casing 101 does not greatly affect the acoustic characteristics of the microphone even if whitening or cracking occurs. Further, the punching metal cover 103 that has been plastically deformed also has a high aperture ratio, so that even if it is deformed, it does not affect the acoustic characteristics of the microphone. For this reason, the acoustic characteristics of the microphone are unlikely to deteriorate against an external impact.

  If the main casing 101 is made of aluminum, plastic deformation is likely to occur due to an impact from the punching metal cover 103. In particular, when the shape around the microphone is plastically deformed, the acoustic characteristics deteriorate. Therefore, according to the housing structure of the microphone array 20 of each embodiment described above, deterioration of acoustic characteristics due to such plastic deformation is suppressed.

  Further, in the case structure, the microphone sheet metal 105 is disposed inside the main case 101. A fitting portion 143 rises from the microphone sheet metal 105. The fitting portion 143 is disposed inside the main casing outer peripheral wall 117 with a gap 149. This fitting part 143 has a pair of clamping pieces spaced apart in the radial direction (the thickness direction of the main casing outer peripheral wall 117). Between the pair of sandwiching pieces of the fitting portion 143, the base sheet metal outer peripheral wall 135 of the base sheet metal 107 is inserted and fitted (fitted). That is, in this case structure, the side part of the impact-resistant case 109 is arranged in the order of the main case outer peripheral wall 117, the gap 149, the outer holding piece 145, the base sheet metal outer peripheral wall 135, and the inner holding piece 147 from the outside. It is configured to overlap.

  Impact energy such as impact applied to the impact-resistant casing 109 from the outside of the side portion is first applied to the outer peripheral wall 117 of the main casing. The main casing outer peripheral wall 117 elastically deforms between the gaps 149 to absorb impact energy. Impact energy exceeding the elastic limit is applied to the fitting portion 143. The impact energy applied to the fitting portion 143 is absorbed by elastically deforming the outer holding piece 145, the base metal outer peripheral wall 135, and the inner holding piece 147. Further, the impact energy applied to the fitting portion 143 is also effectively absorbed and reduced by the friction between the outer sandwiching piece 145 and the base sheet metal outer peripheral wall 135, and the base sheet metal outer peripheral wall 135 and the inner sandwiching piece 147.

  Therefore, according to the casing structure of the microphone array 20 of each embodiment described above, the impact resistance can be improved.

(Fourth embodiment)
In each of the first to third embodiments, the description has been made assuming the operation of the audio processing system when one designated place is designated by the user in the video data displayed on the displays 63 and 73. In the fourth embodiment, the operation of the sound processing system when a plurality of (for example, two) designated locations are designated by the user in the video data similarly displayed on the displays 63 and 73 will be described. Since the system configuration of the voice processing system of this embodiment is the same as the system configuration of the voice processing system 5A shown in FIG. 1A, description will be made with reference to the reference numerals of the respective parts of the voice processing system 5A.

  The audio processing system according to the present embodiment, for example, when two designated locations are designated by the user in the video data displayed on the displays 63 and 73, the two designated locations are properly distinguished and distinguished. In order to make it clearly visible to the user, a different identification shape for each designated location is displayed around each designated location. Furthermore, the voice processing system of the present embodiment uses the voice data of the voice collected by the microphone array 20 to form directivity in the direction from the microphone array 20 toward the voice position corresponding to each designated location, Audio is output according to a method defined in advance in association with each identification shape.

  FIG. 8 is a schematic diagram illustrating an example of a usage pattern of the voice processing system 5A according to the fourth embodiment. FIG. 8A is a diagram showing a state in which one camera 10, one microphone array 20, and a speaker 82 are installed on the ceiling 85 of an indoor hall, for example. FIG. 8B is an explanatory diagram of an outline of the operation of the audio processing system 5 </ b> A when a plurality of designated locations are designated in the video data displayed on the display 63.

  In FIG. 8A, two persons 91a and 92a are standing on the floor 87 of the hall and having a conversation. A speaker 82 is placed in contact with the floor 87 at a position slightly away from the two persons 91a and 92a, and music flows from the speaker 82. In addition, the camera 10 captures images of persons 91 a and 92 a around a monitoring target point (place) set in advance in the camera 10. Furthermore, the microphone array 20 collects the sound of the entire hall. Video data captured by the camera 10 is displayed on the screen 68 of the display 63. In addition, the speaker 65 outputs the conversation between the two persons 91 and 92 or the music in the hall.

  For example, it is assumed that the user continuously touches the vicinity of the heads of the two persons 91 a and 92 a displayed on the screen 68 of the display 63 with the finger 95. The touch points 63a1 and 63a2 are a plurality of designated locations designated by the user. The signal processing unit 50 uses the sound picked up by the microphone array 20, that is, the sound data picked up by each microphone 22, from the position of each microphone 22 of the microphone array 20, and touch points 63a1, designated by the user. Each voice data in which directivity is formed in each directivity direction (direction indicated by reference signs e1 and e2 shown in FIG. 8A) toward each voice position corresponding to 63a2 is generated and synthesized.

  In other words, the signal processing unit 50 uses each audio data collected by each microphone 22 from each microphone 22 in the microphone array 20 to each audio position corresponding to the touch points 63a1 and 63a2 designated by the user. Generate and synthesize audio data that emphasizes (amplifies) audio in each directional direction (volume level). The playback unit 60 outputs the audio data synthesized by the signal processing unit 50 from the speaker 65 in synchronization with the video data captured by the camera 10.

  As a result, the voice at each voice position corresponding to the touch points 63a1 and 63a2 designated by the user is emphasized, and the conversation between the two persons 91a and 92a from the speaker 65 (for example, “Hello” shown in FIG. 8A) (See “Hi!”) Is output at a high volume. On the other hand, music flowing from the speaker 82 that is placed closer to the microphone array 20 than the two persons 91a and 92a but is not the touch points 63a1 and 63a2 designated by the user (FIG. 8A). The voice is not output with emphasis, but is output with a lower sound volume than the conversation between the two persons 91a and 92a.

  Next, when a plurality of designated locations are designated by the user, the audio processing system according to the present embodiment displays different identification shapes for each designated location in the video data displayed on the display 63 around each designated location. An example of displaying the voice and an example of outputting the voice according to a method defined in advance in association with each identification shape will be described in detail with reference to FIGS. In order to make the description of FIGS. 31 to 40 of the present embodiment easier to understand, description will be made assuming an audio processing system 5D in which the omnidirectional camera 10E and the microphone array 20C are integrated (FIG. 9A). However, in the voice processing system 5D of the present embodiment, it is assumed that a plurality of (for example, two) speakers 65L and 65R are provided in the voice processing device 40 or the PC 70.

  FIG. 31A is an explanatory diagram of a usage example of the voice processing system 5D of the fourth embodiment. FIG. 31B shows an example of displaying an example of the first identification shape 91M displayed around the first designated location, and the second identification shape 92M displayed around the second designated location, A state in which the sound in the first directivity direction toward the first sound position corresponding to the first designated location specified by the first identification shape 91M is emphasized and output from the first speaker 65L; It is a figure which shows a mode that the audio | voice of the 2nd directivity direction which goes to the 2nd audio | voice position corresponding to the 2nd designated location specified by the identification shape 92M is emphasized, and it outputs from the 2nd speaker 65R.

  In FIG. 31A, for example, a donut-shaped microphone array 20C, an omnidirectional camera 10E integrated with the microphone array 20C, and a speaker 83 are installed on the ceiling 85 of an indoor hall. Further, in FIG. 31A, four persons 91a, 92a, 93a, 94a are talking on the floor 87 of the hall, more specifically, the persons 91a, 92a are talking, Persons 93a and 94a are having a conversation. A speaker 82 is placed on the floor 87 at a position slightly away from the persons 92a and 93a, and music flows from the speaker 82. Further, the omnidirectional camera 10E images the persons 91a, 92a, 93a, 94a and the speaker 82 that exist within a predetermined viewing angle. Furthermore, the microphone array 20C collects the sound of the entire hall. On the screen 68 of the display 63, video data captured by the omnidirectional camera 10E is displayed.

(Combination of the specified location specification method and the audio output method associated with the specified method)
Hereinafter, in the voice processing system 5D of the present embodiment, a plurality of examples are used for a combination of a method for specifying a plurality of designated locations of a user and a voice output method associated with an identification shape displayed for each designated location. I will explain. However, the combination of the following designation location designation method and voice output method is merely an example, and each designation location may be combined using another designation location designation method or voice output method.

(Combination of first designation method and audio output method)
The first designation method is a method of designating a designated portion by, for example, a left click operation and a right click operation using a mouse. The first audio output method is a simple stereo 2ch (channel) output method in which one audio data at a specified location is output from one speaker and the other audio data at the specified location is output from the other speaker.

  The user, for example, near the overhead of the person 91a displayed on the screen 68 (see FIG. 31B) of the display 63 by left-clicking the operation unit 55 (for example, a mouse), and further, near the overhead of the person 92a. It is assumed that each is designated continuously by a right click operation of 55 (for example, a mouse). The locations specified by the left click operation and the right click operation are a plurality of specified locations specified by the user. When a plurality of designated locations are designated, the signal processing unit 50 displays a different identification shape for each designated location around each designated location in order to properly distinguish each designated location.

  Specifically, the signal processing unit 50 displays an identification shape 91M for visually indicating that the person 91a has been designated around the person 91a designated by the left click operation. An identification shape 92M for visually indicating that the person 92a is designated is displayed around the person 92a designated by the click operation. The identification shapes 91M and 92M are, for example, green and red rectangles, respectively, but the color and shape are not limited to green, red, and rectangle.

  In addition, the signal processing unit 50 uses the sound data of the sound collected by the microphone array 20C to direct each direction from the installation position of the microphone array 20C to each sound position corresponding to two designated locations designated by the user. Each piece of audio data in which directivity is formed in the direction (the direction indicated by reference signs e1 and e2 shown in FIG. 31A) is generated. The reproducing unit 60 synchronizes with the video data imaged by the omnidirectional camera 10E, and emphasizes the audio data in the first directivity direction (see symbol e1 shown in FIG. 31A) specified by the identification shape 91M. Is output from the speaker 65L, and the audio data emphasizing the sound in the second directivity direction (see symbol e2 shown in FIG. 31A) specified by the identification shape 92M is output from the speaker 65R. Accordingly, the conversation voice (“Hello”) of the person 91a is emphasized and output from the speaker 65L, and the conversation voice (“Hi!”) Of the person 92a is emphasized and output from the speaker 65R.

  In FIG. 32, the adjustment operation box OPB is displayed in response to a click operation outside the display area of the video data displayed on the display 63 in a state where the video data shown in FIG. 31B is displayed. It is a figure which shows a mode. For example, when the video data shown in FIG. 31B is displayed on the display 63, the user moves the cursor MPT out of the video data display area using the operation unit 55 (for example, a mouse) and then performs a click operation. (For example, right click operation). The signal processing unit 50 causes the display 63 to display an adjustment operation box OPB for adjusting an audio parameter (for example, a volume level) output from the speaker 65L or 65R according to a user's click operation. The adjustment operation box OPB has been described as being used, for example, for adjusting the volume level. However, other than that, adjustment of equalizer settings at the time of sound output and switching between directional sound and omnidirectional sound can be performed. It may be used for adjustment.

  When the user selects the first identification shape 91M and presses the “+” button of the adjustment operation box OPB a plurality of times, the conversation voice of the person 91a output from the speaker 65L is further increased. Is done. On the other hand, when the user selects the second identification shape 92M and presses the “−” button of the adjustment operation box OPB a plurality of times, the conversation voice of the person 92a output from the speaker 65R is further reduced. Is output.

  The first identification shape 91M and the second identification shape 92M are both solid lines, but are distinguished by different colors. For example, the colors are the same and are distinguished by a solid line and a dotted line. (See FIG. 33B). FIG. 33A is an explanatory diagram of a usage example of the speech processing system 5D of the fourth embodiment. FIG. 33B shows an example of displaying an example of the first identification shape 91N displayed around the first designated location and the second identification shape 92N displayed around the second designated location; A state in which the sound in the first directivity direction toward the first sound position corresponding to the first designated location specified by the first identification shape 91N is emphasized and output from the first speaker 65L; It is a figure which shows a mode that the audio | voice of the 2nd directivity direction which goes to the 2nd audio | voice position corresponding to the 2nd designated location specified by the identification shape 92N is emphasized, and it outputs from the 2nd speaker 65R.

  Note that FIG. 33A is similar to FIG. 31A, and thus the description of FIG. 33A is omitted. Further, in FIG. 31B, the colors of the identification shapes 91M and 92M are different and both are solid lines. However, in FIG. 33B, the colors of the identification shapes 91N and 92N are the same, and one (first) Since there is no difference between FIG. 33 (B) and FIG. 31 (B) except that one identification shape 91N) is a solid line and the other (second identification shape 92N) is a dotted line, FIG. I will omit the explanation.

  FIG. 34 shows video data captured by the omnidirectional camera 10E for each click operation outside the display area of the video data displayed on the display 63 in a state where the video data shown in FIG. 31 (B) is displayed. It is a figure which shows a mode that it switches and displays operation box OPB for adjustment. For example, when the video data shown in FIG. 31B is displayed on the display 63, the user moves the cursor MPT out of the video data display area using the operation unit 55 (for example, a mouse) and then performs a click operation. (For example, right click operation). In response to the user's click operation, the signal processing unit 50 switches the screen of the video data captured by the omnidirectional camera 10E to the adjustment operation box OPB and causes the display 63 to display the screen.

  On the other hand, when the adjustment operation box OPB is displayed on the display 63, the user moves the cursor MPT out of the display area of the video data using the operation unit 55 (for example, a mouse) and then performs a click operation (for example, the right). Click operation). In response to the user's click operation, the signal processing unit 50 switches the adjustment operation box OPB to a screen of video data captured by the omnidirectional camera 10E and displays it on the display 63. It has been described that the switching between the adjustment operation box OPB and the screen of the video data captured by the omnidirectional camera 10E is executed by a click operation outside the display area of the video data of the cursor MPT, but is limited to the click operation. Instead, it may be executed by a predetermined input operation. The predetermined input operation is, for example, an operation in which the user simultaneously presses a plurality of specific keys with different keyboards.

  In FIG. 35, in the state where the video data shown in FIG. 31 (B) is displayed, a state indication box IND is displayed in response to a click operation outside the display area of the video data displayed on the display 63. It is a figure which shows a mode. For example, when the video data shown in FIG. 31B is displayed on the display 63, the user moves the cursor MPT out of the video data display area using the operation unit 55 (for example, a mouse) and then performs a click operation. (For example, right click operation). The signal processing unit 50 displays, on the display 63, a status indication box IND for indicating the status of a parameter (for example, a volume level) of audio output from the speaker 65L or 65R in response to a user's click operation. Display.

  Although the user cannot operate the status indication box IND, if any identification shape displayed on the display 63 is designated by the user, the voice of the person corresponding to the designated identification shape is not displayed. The contents of the sound volume level are visually specified by a state indication box IND. Further, in order to change the contents of the state indication box IND, for example, the user presses a specific key of another operation unit (for example, a keyboard) while the first identification shape 91M is selected. The result of increasing or decreasing the volume level of the conversation voice of the person 91a that is outputting the sound from 65L or the process leading to the result is visually specified in the state indication box IND. Note that the state indication box IND is described as indicating, for example, the state of the volume level. However, other than the setting contents of the equalizer at the time of sound output, the state of switching between directional sound and omnidirectional sound May be used for marking. Further, the state indication box IND may be always displayed on the display 63.

(Combination of second designation method and audio output method)
The second designation method is a method of designating a designated portion by, for example, pressing a numeric key on the keyboard and left-clicking the mouse. The second audio output method is a synthetic monaural 2ch (channel) output method that outputs audio data of all designated locations from both speakers.

  FIG. 36A is an explanatory diagram of a usage example of the voice processing system 5D of the fourth embodiment. FIG. 36B shows the first identification shape 91K displayed around the first designated location, the second identification shape 92K displayed around the second designated location, and the circumference of the third designated location. An example of displaying the third identification shape 93K displayed on the screen, an example of the fourth identification shape 94K displayed around the fourth designated location, and the first designation specified by the first identification shape 91K Heading to the second voice position corresponding to the second designated location specified by the second identification shape 92K and the voice data in which the voice in the first directivity direction toward the first voice position corresponding to the location is emphasized. Voice data in which the voice in the second directivity direction is emphasized, and voice in which the voice in the third directivity direction toward the third voice position corresponding to the third designated position specified by the third identification shape 93K is emphasized. Data, the first and second speakers 65L, Is a diagram showing how the output from the 5R. Note that FIG. 36A is similar to FIG. 31A, and thus the description of FIG. 36A is omitted.

  For example, the user operates the operation unit 55 (for example, pressing the number “1” key on the keyboard and left clicking the mouse) near the head of the person 91a displayed on the screen 68 of the display 63 (see FIG. 36B). The operation portion 55 (for example, pressing the number “2” key on the keyboard and the left click of the mouse) is operated simultaneously near the head of the person 92a, and the operation portion 55 (for example, the number “3” key on the keyboard) The simultaneous operation of pressing and left clicking of the mouse is performed, and the vicinity of the head of the person 94a is successively designated by the simultaneous operation of the operation unit 55 (for example, pressing of the number “4” key on the keyboard and left clicking of the mouse). . Each location specified by pressing the number key and each left-click operation becomes a plurality of specified locations specified by the user. When a plurality of designated locations are designated, the signal processing unit 50 displays a different identification shape for each designated location around each designated location in order to properly distinguish each designated location.

  Specifically, the signal processing unit 50 visually identifies that the person 91a has been designated around the person 91a designated by the pressing operation of the number “1” key and the left click operation. 91K is displayed, and an identification shape 92K for visually indicating that the person 92a has been designated is displayed around the person 92a designated by the pressing operation of the number “2” key and the left click operation. An identification shape 93K for visually indicating that the person 93a is designated is displayed around the person 93a designated by the pressing operation of the “3” key and the left click operation, and the number “4” key is pressed. An identification shape 94K for visually indicating that the person 94a is designated is displayed around the person 94a designated by the operation and the left click operation. The identification shapes 91K, 92K, 93K, and 94K are, for example, black rectangles, but the color and shape are not limited to black and rectangles.

  In addition, the signal processing unit 50 uses the sound data of the sound collected by the microphone array 20C to direct each direction from the installation position of the microphone array 20C to each sound position corresponding to the four designated locations designated by the user. Each voice data in which directivity is formed in the direction (directions indicated by reference signs e1, e2, and e3 shown in FIG. 36A) is generated and synthesized. The reproduction unit 60 synchronizes with the video data captured by the omnidirectional camera 10E, and enhances audio data in the first directivity direction (see reference numeral e1 shown in FIG. 36A) specified by the identification shape 91K. And voice data emphasizing the voice in the second directivity direction (see symbol e2 shown in FIG. 36A) specified by the identification shape 92K, and the third directivity direction (FIG. 36) specified by the identification shape 93K. The voice data obtained by synthesizing the voice data in which the voice of (e3) shown in (A) is emphasized is output from the speakers 65L and 65R. Accordingly, the conversation voice of the person 91a (“Hello”), the conversation voice of the person 92a (“Hi!”), And the conversation voice of the person 93a (“Good morning!”) Are emphasized from the speakers 65L and 65R and outputted. . In FIG. 36A, since the person 94a does not speak, the conversation voice of the person 94a is not emphasized and output from the speakers 65L and 65R. In the case where the voice of the person 94a is output, the voice of the person 94a is also output from the speakers 65L and 65R.

  FIG. 37 is a diagram illustrating a state in which the adjustment operation box OPB is displayed in response to the simultaneous pressing operation of a plurality of specific keys on the keyboard in the state where the video data illustrated in FIG. 36B is displayed. . For example, when the video data shown in FIG. 36B is displayed on the display 63, the user performs the simultaneous pressing operation of the operation unit 55 (for example, the “Shift” key and the number “1” key on the keyboard). To do. The signal processing unit 50 causes the display 63 to display an adjustment operation box OPB for adjusting the volume level of the sound output from the speaker 65L or 65R according to the simultaneous pressing operation by the user.

  In FIG. 39, the adjustment operation box OPB is displayed in response to a click operation outside the display area of the video data displayed on the display 63 while the video data shown in FIG. It is a figure which shows a mode. For example, when the video data shown in FIG. 36B is displayed on the display 63, the user moves the cursor MPT out of the video data display area using the operation unit 55 (for example, a mouse) and then performs a click operation. (For example, right click operation). The signal processing unit 50 causes the display 63 to display an adjustment operation box OPB for adjusting the volume level of the sound output from the speaker 65L or 65R according to the user's click operation.

(Combination of the third designation method and audio output method)
The third designation method is a method of designating a designated portion by a drawing operation of a different identification shape by a user's finger or a stylus pen on a display 63 provided with a touch panel or a touch device (for example, a touch pad) different from the touch panel, for example. It is. In the third audio output method, audio data of one or a plurality of designated locations designated by the user is output from one speaker, and similarly, the audio data of one or more designated locations designated by the user is output. This is a synthetic stereo 2ch (channel) output method for outputting sound from the other speaker. Hereinafter, in order to make the explanation easy to understand, it is assumed that the designated portion is designated by the user's drawing operation on the display 63 provided with the touch panel.

  FIG. 39A is an explanatory diagram of a usage example of the speech processing system 5D of the fourth embodiment. FIG. 39B shows the first identification shape 91L displayed around the first designated location, the second identification shape 92L displayed around the second designated location, and the circumference of the third designated location. The third identification shape 93L displayed on the screen, an example of the fourth identification shape 94L displayed around the fourth designated location, and the first designation specified by the first identification shape 91L Heading to the second voice position corresponding to the second designated location specified by the second identification shape 92L and the voice data in which the voice in the first directivity direction toward the first voice position corresponding to the location is emphasized. The voice data in which the voice in the second directivity direction is emphasized is synthesized and output from the first speaker 65L, and the third voice corresponding to the third designated portion specified by the third identification shape 93L Emphasizing the voice in the third direction toward the position It is a diagram showing a state of outputting the voice data from the second speaker 65R. Note that FIG. 39A is similar to FIG. 31A, and thus the description of FIG. 39A is omitted.

  The user, for example, draws a round shape by touching and dragging near the head of the person 91a displayed on the screen 68 (see FIG. 40B) of the display 63, and a rectangular shape by touching and dragging near the head of the person 92a. Assume that the drawing operation, the triangular drawing operation by touching and dragging near the head of the person 93a, and the hexagonal drawing operation by touching and dragging near the head of the person 94a are respectively designated continuously. Each location designated by the drawing operation of each shape by touch and drag becomes a plurality of designated locations designated by the user. When a plurality of designated places are designated, the signal processing unit 50 uses a shape drawn by a different drawing operation for each designated place as an identification shape around each designated place in order to appropriately distinguish each designated place. Display.

  Specifically, the signal processing unit 50 displays an identification shape 91L for visually indicating that the person 91a is designated around the person 91a designated by the circular drawing operation, thereby obtaining a rectangular shape. An identification shape 92L for visually indicating that the person 92a has been designated is displayed around the person 92a designated by the drawing operation, and around the person 93a designated by the triangular drawing operation, An identification shape 93L for visually indicating that the person 93a has been specified is displayed, and it is visually indicated that the person 94a has been specified around the person 94a specified by the hexagonal drawing operation. An identification shape 94L is displayed. The identification shapes 91K, 92K, 93K, and 94K are merely examples, and are not limited to each shape. In FIG. 39B, each identification shape is illustrated by a dotted line, but is not limited to a dotted line, and is illustrated by, for example, a solid line. May be.

  In addition, the signal processing unit 50 uses the sound data of the sound collected by the microphone array 20C to direct each direction from the installation position of the microphone array 20C to each sound position corresponding to the four designated locations designated by the user. Each piece of audio data having directivity in the direction (direction indicated by reference signs e1, e2, and e3 shown in FIG. 39A) is generated and synthesized. For example, the reproducing unit 60 groups the identification shapes 91L and 92L drawn in the display area on the left side from the center of the display 63 as one audio output group, and synchronizes with the video data captured by the omnidirectional camera 10E to identify the identification shapes. The voice data in which the voice in the first directivity direction (see reference numeral e1 shown in FIG. 39A) specified by 91L is emphasized and the second directivity direction (see FIG. 39A) specified by the identification shape 92L. The voice data obtained by synthesizing the voice data emphasizing the voice of the reference numeral e2 is output from the speaker 65L. Further, the reproducing unit 60 groups the identification shape 93L drawn in the display area on the right side from the center of the display 63, for example, as one audio output group, and synchronizes with the video data captured by the omnidirectional camera 10E. Audio data emphasizing the audio in the third directivity direction (see symbol e3 shown in FIG. 39A) specified by 93L is output as audio from the speaker 65R. Accordingly, the conversation voice (“Hello”) of the person 91a and the conversation voice (“Hi!”) Of the person 92a are emphasized and output from the speaker 65L, and the conversation voice (“Good morning!”) Of the person 93a is output from the speaker 65R. Is output as a sound. In FIG. 36A, since the person 94a does not speak, the conversation voice of the person 94a is not emphasized and output from the speakers 65L and 65R. In the case where the voice of the person 94a is output, the voice of the person 94a is also output from the speakers 65L and 65R.

  Further, in the above description, the playback unit 60 forms the audio output groups after dividing the set of identification shapes displayed in the left display area and the right display area from the center of the display 63. However, it is not limited to this method. For example, the user may arbitrarily specify the audio output group. For example, the first identification shape 91L and the third identification shape 93L are designated as one audio output group for outputting sound from the speaker 65L, and the second identification shape 92L is 1 for outputting sound from the speaker 65R. It may be specified as one audio output group. In this case, the reproducing unit 60 emphasizes the sound in the first directivity direction (see the symbol e1 shown in FIG. 39A) specified by the identification shape 91L in synchronization with the video data captured by the omnidirectional camera 10E. Audio data obtained by synthesizing the audio data and the audio data emphasizing the audio in the third directivity direction (see symbol e3 shown in FIG. 39A) specified by the identification shape 93L is output from the speaker 65L. . Furthermore, the reproducing unit 60 emphasizes the voice in the second directivity direction (see symbol e2 shown in FIG. 39A) specified by the identification shape 92L in synchronization with the video data captured by the omnidirectional camera 10E. Audio data is output as audio from the speaker 65R. Therefore, the conversation voice (“Hello”) of the person 91a and the conversation voice (“Good morning!”) Of the person 93a are emphasized and output from the speaker 65L, and the conversation voice (“Hi!”) Of the person 92a is output from the speaker 65R. Is output as a sound.

  FIG. 40 shows an adjustment operation box OPB in response to a touch outside the display area of the video data displayed on the display 63 provided with the touch panel in a state where the video data shown in FIG. 39B is displayed. It is a figure which shows a mode that is displayed. For example, when the video data shown in FIG. 39B is displayed on the display 63 provided with the touch panel, the user touches outside the display area of the video data. The signal processing unit 50 causes the display 63 to display an adjustment operation box OPB for adjusting the volume level of the sound output from the speaker 65L or 65R according to the user's touch.

  As described above, in the fourth embodiment, when the user designates a plurality of different designated locations (for example, two locations) for the video data displayed on the display 63, the signal processing unit 50 includes the video data in the video data. Different identification shapes (for example, identification shapes 91L and 92L) are displayed at different designated locations.

  Thereby, the audio processing system 5D can distinguish and recognize a plurality of different designated places designated by the user in the video data displayed on the display 63. For example, as different identification shapes for each designated designated place, By displaying a rectangular identification shape 91L around one designated location and displaying a round identification shape 92L around the other designated location, the user is visually informed that a plurality of designated locations have been distinguished and recognized. It can be clearly indicated.

  In addition, for example, two speakers are provided in the audio processing system 5D, and the reproducing unit 60 has a first directivity direction from the microphone array 20 to a position corresponding to the first designated location (first audio position). The first voice data emphasizing the voice is outputted from the first speaker 65L, and the voice in the second directivity direction from the microphone array 20 toward the position corresponding to the second designated position (second voice position) is obtained. The emphasized second audio data is output as audio from the second speaker 65R.

  Thereby, for example, when two speakers are provided, the audio processing system 5D emphasizes the sound in the direction of the direction from the microphone array 20 toward the audio position corresponding to each designated location for each designated location. Can be output from each speaker 65L, 65R independently.

  Hereinafter, the configuration, operation, and effect of the above-described voice processing system and voice processing method according to the present invention will be described.

  One embodiment of the present invention includes at least one imaging unit that captures an image, a display unit that displays image data captured by the imaging unit, and a plurality of microphones, and collects sound using the microphones. A sound collection unit, a sound output unit that outputs sound data collected by the sound collection unit, the video data picked up by the image pickup unit, and the sound data picked up by the sound collection unit A recording unit for recording the video data recorded on the recording unit on the display unit, and a playback unit for outputting the audio data recorded on the recording unit to the audio output unit, On the basis of the audio data recorded on the recording unit, an operation unit that accepts designation of one or more designated locations of the video data displayed on the display unit, the designated video data from the sound collection unit A signal processing unit which generates or synthesizes the audio data that emphasizes the orientation of the sound toward the positions corresponding to one or more specified locations, a voice processing system comprising a.

  According to this configuration, the audio processing system uses each audio data collected by each microphone of the microphone array in accordance with designation of a predetermined designated location from the operation unit during reproduction of the already recorded video data. Then, the signal processing unit generates or synthesizes audio data in which directivity is formed in a directivity direction from the microphone array toward a position corresponding to one or more designated locations.

  Thereby, the audio processing system can emphasize and output the audio data in the video for the designated arbitrary reproduction time during the reproduction of the recorded video data and audio data.

  Also, in one embodiment of the present invention, the reproduction unit causes the audio output unit to output audio data in which sound in a directivity direction from the sound collection unit toward a position corresponding to the one or more specified locations is emphasized. This is a voice processing system for outputting.

  Accordingly, the audio processing system can output audio data having directivity formed in the directivity direction from the microphone array toward the position corresponding to the one or more designated locations by the signal processing unit.

  In one embodiment of the present invention, the imaging unit is an omnidirectional camera, and the signal processing unit is specified while the video data captured by the omnidirectional camera is displayed on the display unit. According to the one or more designated locations, the coordinate system of the video data including the one or more designated locations is image-converted, and the playback unit causes the display unit to display the video data after the image conversion. A sound processing system for outputting sound data in which sound in a directivity direction directed to a position corresponding to the one or more designated locations is output from the sound collection unit.

  According to this configuration, the sound processing system performs a coordinate system conversion process for associating the position of the subject in the video data captured by the omnidirectional camera with the direction of the sound of the subject person picked up by the microphone array. Processing in the case where video data captured by a camera and audio data in which directivity is formed in a directivity direction toward a position corresponding to one or more designated locations can be easily reproduced in the reproduction unit. The load can be reduced.

  Moreover, one Embodiment of this invention is an audio processing system with which the said imaging part and the said sound collection part are arrange | positioned coaxially.

  Thereby, the audio processing system is installed so that the omnidirectional camera and the microphone array in the audio processing system have the same central axis, so that the coordinate systems of the omnidirectional camera 10E and the microphone array 20C are made the same. Can do.

  Moreover, one Embodiment of this invention is an audio processing system with which the said imaging part and the said sound collection part are arrange | positioned at an indoor ceiling.

  Thereby, installation of the voice processing system can be simplified.

  Further, according to an embodiment of the present invention, the signal processing unit has different identification shapes for each designated location in the video data in response to designation of different locations for the video data displayed on the display unit. This is a voice processing system to be displayed.

  According to this configuration, when the user designates a plurality of different designated locations (for example, two locations) for the video data displayed on the display, each signal designated in the designated location in the video data, Different identification shapes are displayed.

  As a result, the audio processing system can distinguish and recognize a plurality of different designated locations designated by the user in the video data displayed on the display. By displaying a rectangular identification shape around the specified location and displaying a round identification shape around the other specified location, it is clearly indicated to the user that a plurality of specified locations have been recognized and recognized. be able to.

  In one embodiment of the present invention, the audio output unit includes a first audio output unit and a second audio output unit, and the reproduction unit is moved from the sound collection unit to the first designated location. The first sound data in which the sound in the first directivity direction toward the corresponding position is emphasized is output from the first sound output unit, and the first sound data toward the position corresponding to the second designated location is output from the sound collection unit. In the speech processing system, the second speech data in which the speech in the two directivity directions is emphasized is output from the second speech output unit.

  According to this configuration, the audio processing system is provided with, for example, two speakers, and the reproduction unit has a first directivity direction from the microphone array to a position corresponding to the first designated location (first audio position). The first voice data with enhanced voice is output from the first speaker, and the voice in the second directivity direction from the microphone array toward the position corresponding to the second designated position (second voice position) is emphasized. The second audio data is output from the second speaker.

  Thereby, for example, when two speakers are provided, the voice processing system, for each designated place, each voice data that emphasizes the voice in the directivity direction from the microphone array to the voice position corresponding to each designated place, Audio can be output independently from each speaker.

  In one embodiment of the present invention, the audio output unit includes a first audio output unit and a second audio output unit, and the playback unit is provided at a plurality of designated locations different from the sound collection unit. Voice data obtained by synthesizing voice data that emphasizes voices in different directivity directions toward the corresponding position is output from the first voice output unit, and the remaining one or more designated locations are output from the sound pickup unit. In the speech processing system, speech data that emphasizes speech in one or more remaining directivity directions toward a corresponding position is output from the second speech output unit as speech or synthesized speech.

  According to this configuration, the audio processing system is provided with, for example, two speakers, and the playback unit moves from the microphone array to positions corresponding to a plurality of different designated locations (for example, first and second audio positions). Audio data obtained by synthesizing audio data that emphasizes audio in the first and second directivity directions is output from the first speaker, and the position corresponding to one or more remaining designated locations from the microphone array (for example, Audio data that emphasizes the remaining voice in one or more directivity directions toward the third audio position) is output from the second speaker.

  As a result, when two speakers are provided, for example, the audio processing system synthesizes each audio data in which a plurality of (for example, two) directional sounds are emphasized from the microphone array and outputs the audio from one speaker. In addition, voice data in which voice in another direction is emphasized can be outputted from the other speaker.

  In addition, one embodiment of the present invention includes one or more audio output units, and the reproduction unit outputs audio in different directivity directions from the sound collection unit to positions corresponding to different designated locations. It is an audio processing system for outputting audio data obtained by synthesizing emphasized audio data from one or more audio output units.

  According to this configuration, the audio processing system is provided with, for example, one or more speakers, and the reproduction unit has the first directivity from the microphone array toward the position corresponding to the first designated location (first audio position). First voice data in which voice in the direction is emphasized, second voice data in which voice in the second directivity direction from the microphone array toward the position corresponding to the second designated location (second voice position) is emphasized, and , Voice data obtained by synthesizing the third voice data that emphasizes the voice in the third directivity direction from the microphone array toward the position corresponding to the third designated position (third voice position). Sound is output from the speaker.

  As a result, the audio processing system, for example, when one or more speakers are provided, synthesizes each audio data in which a plurality of (for example, three) directional sounds are emphasized from the microphone array and outputs the audio from the speakers. In addition, when a plurality of speakers are provided, the synthesized audio data can be output simultaneously.

  In one embodiment of the present invention, the signal processing unit outputs audio from the audio output unit in response to a predetermined input operation or a designation operation outside the display area of the video data displayed on the display unit. The voice processing system displays a parameter adjustment operation medium for the voice data.

  According to this configuration, the audio processing system can perform a speaker operation by performing a predetermined input operation (for example, a right click operation of the mouse) or a designation operation (for example, a left click operation of the mouse) outside the display area of the video data displayed on the display. It is possible to easily display an adjustment operation box that accepts an adjustment operation of a parameter (for example, volume level) of audio data that is output from the audio.

  Further, in one embodiment of the present invention, the signal processing unit may perform the audio output unit constantly or in response to a predetermined input operation or a designation operation outside the display area of the video data displayed on the display unit. It is a voice processing system for displaying a parameter status indicating medium for the voice data output from the voice.

  According to this configuration, the audio processing system always or predetermined input operation (for example, right click operation of the mouse) or designation operation outside the display area of the video data displayed on the display (for example, left click operation of the mouse) Thus, it is possible to easily display a state indication box as an indicator for indicating the state of a parameter (for example, a volume level) of audio data output from the speaker.

  In one embodiment of the present invention, the signal processing unit captures a video imaged by the imaging unit every time a predetermined input operation or a designation operation outside the display area of the video data displayed on the display unit is performed. The voice processing system is configured to switch to a parameter adjustment operation medium for data or voice data output from the voice output unit and to display the medium on the display unit.

  According to this configuration, the audio processing system performs video data captured by the camera for each predetermined input operation or designation operation (for example, a left click operation of the mouse) outside the display area of the video data displayed on the display. Alternatively, an adjustment operation box that accepts an adjustment operation of a parameter (for example, a volume level) of audio data output from a speaker can be easily switched and displayed.

  Further, according to an embodiment of the present invention, the signal processing unit may perform the specified location from the sound collection unit according to a drawing operation of a predetermined shape centered on the specified location of the video data displayed on the display unit. Is a voice processing system that generates or synthesizes voice data that emphasizes voice in a directivity direction toward a position corresponding to.

  According to this configuration, the audio processing system can perform a simple drawing operation (for example, a touch operation and a slide in a state in which the touch operation is performed) that draws a predetermined shape (for example, a rectangular shape) centering on a designated portion of the video data displayed on the display. Operation) can generate or synthesize audio data in which sound in a directivity direction from the microphone array toward the position corresponding to the designated location is emphasized.

  In one embodiment of the present invention, the signal processing unit displays the identification shape redesignated from the sound collection unit in response to the redesignation of the identification shape displayed for each designated location. A speech processing system that generates or synthesizes speech data in which enhancement of speech in a directivity direction toward a position corresponding to a designated location is stopped.

  According to this configuration, when the identification shape displayed for each designated location is redesignated, the voice processing system is directed toward the position corresponding to the designated location where the identification shape redesignated from the microphone array is displayed. It is possible to easily generate or synthesize voice data for which the enhancement of voice is stopped.

  In one embodiment of the present invention, at least one image pickup unit picks up an image, the sound pickup unit including a plurality of microphones picks up sound, and the image data picked up by the image pickup unit is recorded. Displaying on the display unit, recording the video data captured by the imaging unit and the audio data collected by the sound collection unit, and displaying the recorded video data on the display unit, Based on the recorded audio data, a step of outputting the recorded audio data to an audio output unit, a step of accepting designation of one or more designated locations of the video data displayed on the display unit, and the recorded audio data And generating sound data that emphasizes sound in a directivity direction toward a position corresponding to one or more specified locations of the specified video data from the sound collection unit Having the steps of synthesizes a speech processing method.

  According to this method, the audio processing system uses each audio data collected by each microphone of the microphone array in accordance with designation of a predetermined designated location from the operation unit during reproduction of already recorded video data. Then, the signal processing unit generates or synthesizes audio data in which directivity is formed in a directivity direction from the microphone array toward a position corresponding to one or more designated locations.

  Thereby, the audio processing system can emphasize and output the audio data in the video for the designated arbitrary reproduction time during the reproduction of the recorded video data and audio data.

  While various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  The present invention can emphasize and output audio data in a directional direction toward a position corresponding to one or more designated locations designated on a display screen on which captured video data is displayed. It is useful as a speech processing system and speech processing method that can easily recognize the state of the parameters.

5A, 5B, 5C, 5D Audio processing system 10, 10A, 10B, 10C Camera 10E Omnidirectional camera 20, 20A, 20C, 20D, 20E, 20F Microphone array 22, 22A, 22B, 22C, 22D, 22E, 22F, 22a 22b, 22c, 22n-1, 22n Microphone 30, 30A Network 40 Audio processing device 45, 45A Recorder 50, 71 Signal processing unit 51a, 51b, 51c, 51n-1, 51n A / D converters 52a, 52b, 52c 52n-1, 52n Delay unit 55, 78 Operation unit 57 Adder 60, 60A, 60B Playback unit 63, 73 Display 65, 75, 82, 83 Speaker 101 Main housing 103 Punching metal cover 105 Microphone sheet metal 107 Base sheet metal 111 Annular bottom 113 m Equal hole 117 Main casing outer peripheral wall 127 Microphone board 129 Microphone casing 133 Annular top panel 135 Base sheet metal outer peripheral wall 139 Main board 141 Power supply board 143 Fitting part 145 Outer clamping piece 147 Inner clamping piece 149 Gap

Claims (9)

An imaging unit for imaging video;
A display unit having a rectangular display area, and displaying video data captured by the imaging unit in a circular video display area smaller than the rectangular display area ;
A sound collection unit that includes a plurality of microphones and collects sound using the microphones;
An audio output unit for outputting audio data collected by the sound collection unit;
An operation unit for accepting designation to the display unit;
Based on the audio data, generating or synthesizing emphasized audio data that emphasizes audio in a directivity direction toward a position corresponding to a specified location of the video data, which is specified by the operation unit, from the sound collection unit. A signal processing unit for outputting audio from the audio output unit,
Wherein the signal processing unit, in response to said reception of assignment to the video display area outside by the operation unit, a status display area or operating area for adjustment of the audio output, any of the four corners of the rectangular display area Displaying the image display area so as to straddle the boundary line of the video display area in a rectangular area diagonally extending from the center to the video display area .
Voice processing system. The speech processing system according to claim 1 ,
When the operation unit accepts the sound output adjustment operation in a state where the state display area or the adjustment operation area is displayed, the signal processing unit outputs the emphasized sound data to the sound output unit. adjust,
Voice processing system. The speech processing system according to claim 2 ,
The audio output adjustment operation includes a directional switching operation for switching between the emphasized audio data and the non-emphasized audio data and outputting the audio to the audio output unit.
Voice processing system. The speech processing system according to claim 1 ,
The signal processing unit, when the designation of acceptance of the status display area or adjusting the operation area of the audio output of the enhanced speech data to the video display area outside in the state shown Table is performed by the operation unit, the enhanced speech Hide the status display area or adjustment operation area for data audio output.
Voice processing system. An imaging unit for imaging video;
A display unit having a rectangular display area, and displaying video data captured by the imaging unit in a circular video display area smaller than the rectangular display area ;
A sound collection unit that includes a plurality of microphones and collects sound using the microphones;
An audio output unit for outputting audio data collected by the sound collection unit;
An operation unit for accepting designation to the display unit;
Based on the voice data, a plurality of emphasized voice data is generated or synthesized by emphasizing voice in a directivity direction toward a position corresponding to each of a plurality of designated locations that are designated by the operation unit from the sound collection unit. And
Depending on the specification of acceptance into the video display area outside by the operation unit, the adjustment operation area of the enhanced speech data, diagonals to the center of the video display area from one of the four corners of the rectangular display area In the rectangular area to be displayed so as to straddle the boundary line of the video display area ,
When the sound output adjustment operation is accepted by the operation unit in a state where one of the plurality of designated portions is selected by the operation unit, the pointing direction toward the position corresponding to the designated portion in the selected state is changed. A signal processing unit that adjusts audio output to the audio output unit of the emphasized audio data that emphasizes audio , and
Voice processing system. The speech processing system according to claim 5 ,
The audio output adjustment operation includes a directional switching operation for switching between the emphasized audio data and the non-emphasized audio data and outputting the audio to the audio output unit.
Voice processing system. The speech processing system according to claim 5 ,
The signal processing unit, when the designation of acceptance of the status display area or adjusting the operation area of the audio output of the enhanced speech data to the video display area outside in the state shown Table is performed by the operation unit, the enhanced speech Hide the status display area or adjustment operation area for data audio output.
Voice processing system. Capturing an image in the imaging unit;
Collecting sound in a sound collecting unit including a plurality of microphones;
Displaying the video data captured by the imaging unit within a circular video display area smaller than the rectangular display area of the display unit having a rectangular display area ;
Displaying the video data on the display unit, and outputting audio data collected by the sound collection unit to an audio output unit;
Receiving a designation to the display unit by an operation unit;
Based on the audio data, generating or synthesizing emphasized audio data that emphasizes audio in a directivity direction toward a position corresponding to a specified location of the video data, which is specified by the operation unit, from the sound collection unit. Outputting audio from the audio output unit;
Depending on the specification of acceptance into the video display area outside by the operation unit, the state display area or operating area for adjustment of the audio output, the video display area from one of the four corners of the rectangular display area having a step of displaying to straddle the perimeter of the image display region within the rectangular region of from the center to the diagonal of the,
Audio processing method. Capturing an image in the imaging unit;
Collecting sound in a sound collecting unit including a plurality of microphones;
Displaying the video data captured by the imaging unit within a circular video display area smaller than the rectangular display area of the display unit having a rectangular display area ;
Displaying the video data on the display unit, and outputting audio data collected by the sound collection unit to an audio output unit;
Receiving a designation to the display unit by an operation unit;
Based on the voice data, a plurality of emphasized voice data is generated or synthesized by emphasizing voice in a directivity direction toward a position corresponding to each of a plurality of designated locations that are designated by the operation unit from the sound collection unit. And
In response to the designation of the outside of the video display area by the operation unit, the operation area for adjusting the emphasized audio data is a diagonal line from any one of the four corners of the rectangular display area to the center of the video display area. In the rectangular area to be displayed so as to straddle the boundary line of the video display area ,
When the sound output adjustment operation is accepted by the operation unit in a state where one of the plurality of designated portions is selected by the operation unit, the pointing direction toward the position corresponding to the designated portion in the selected state is changed. Adjusting the audio output to the audio output unit of the emphasized audio data in which the audio is emphasized ,
Audio processing method.