JP2016152557A - Sound collection system and sound collection setting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound collection system and a sound collection setting method that can properly form directivity at a predetermined imaging position and clearly output sounds at the predetermined imaging position even when the positional relationship between a camera and a microphone array is unknown.SOLUTION: When a microphone array device MA is secured from the rear side in a shop 10 where camera devices C1 to C4 are installed, in preset processing, sound sources are disposed at preset positions P1 to Pn as the imaging centers of the camera devices C1 to Cn, and sounds are produced. When the microphone array device MA collects sounds of a predetermined sound volume or more output from the sound sources and transmit the audio data thereof to an audio processing device 50, a sound source direction detector 52 makes a display 58 display sound source marks SD to SD4 representing the orientation directions (horizontal angle θ and vertical angle φ), thereby promoting a user to select an orientation direction and input camera information. The sound processing device 50 transmits the input camera information and the orientation direction to the microphone array device MA. The microphone array MA stores them as a preset information table 90 in the storage part 24.SELECTED DRAWING: Figure 7

Description

  The present invention relates to a sound collection system and a sound collection setting method for forming and outputting directivity at a predetermined position with respect to collected sound.

  Conventionally, in a surveillance system installed at a predetermined position (for example, a ceiling or a wall) in a factory, a store (for example, a retail store or a bank), a shopping street, or a public place (for example, a station or a library), a plurality of camera devices are connected via a network. Are connected, and video data (including still images and moving images; the same applies hereinafter) within a predetermined range to be monitored is monitored by a monitoring device installed at one location.

  However, since the amount of information that can be obtained is inevitably limited in video-only monitoring, there is an increasing demand for a monitoring system that obtains audio data in order to perform audio monitoring.

  In response to this request, some camera devices include a product that is equipped with a microphone and transmits audio data to video data. However, the microphones used in such products are often omnidirectional, and even if they are unidirectional microphones, their directional characteristics are wide-angle. Therefore, it is often the case that the sound desired to be heard during monitoring is drowned out by noise and cannot be heard.

  In recent years, in a surveillance camera system, development of a microphone array that responds to a request to detect a direction in which an abnormal sound has occurred or to listen to only a sound in a specific direction from among noises has been developed (for example, Patent Documents). 1). The microphone array of Patent Document 1 includes a plurality of microphone units, uses sound collected by each microphone unit, and a user's screen with respect to a screen on which an image captured by a surveillance camera (for example, an omnidirectional camera) is displayed. Used to emphasize and reproduce audio from a specified location.

JP 2014-143678 A

  In Patent Document 1, when the omnidirectional camera and the microphone array are mounted on the same axis, the coordinates of the video and the audio coincide with each other (that is, the imaging direction from the omnidirectional camera and the audio collected by the microphone array are the same). The direction to be emphasized is the same), so no particular problem occurs. However, when the surveillance camera and the microphone array are mounted apart from each other, the positional relationship between them is unknown. For this reason, for example, when the correspondence between the coordinate system of the monitoring camera and the coordinate system of the microphone array is not obtained at the time of initial setting, the directivity of the sound in the direction toward the predetermined position captured by the monitoring camera during monitoring There is a problem that it is difficult to form.

  In particular, if the installation position of the surveillance camera or the microphone array is restricted due to the strength of the building or the structure or the structural reason, information on the actual installation position cannot be obtained accurately. For this reason, in order to obtain the correspondence between the coordinate system of the camera and the coordinate system of the microphone array, it is necessary to measure the mounting position, direction, etc. on site, and it takes a lot of time and effort other than dedicated measuring instruments. Work becomes necessary.

  In order to solve the above-described conventional problems, the present invention appropriately forms directivity at a predetermined imaging position even when the positional relationship between the camera and the microphone array is unknown, and at the predetermined imaging position. An object of the present invention is to provide a sound collection system and a sound collection setting method for outputting sound clearly.

  The present invention has a plurality of sound collection elements, a sound collection section for collecting sound by the sound collection elements, at least one image pickup section for picking up an image of a predetermined position, and sound data of the collected sound. A sound source detection unit that detects a sound source direction from the sound collection unit, and a display unit that displays a detected sound source direction from the sound collection unit according to a predetermined sound output at the predetermined position. In response to the designation of the sound source direction from the sound collection unit, an operation unit that receives input of information related to the image pickup unit that images the predetermined position, the input information related to the image pickup unit, and the sound source from the sound collection unit And a storage unit that stores correspondence information in which directions are associated with each other.

  The present invention is also a sound collection setting method in a sound collection system including at least one imaging unit that images a predetermined position and a sound collection unit, wherein a plurality of predetermined output sounds of a sound source placed at the predetermined position are A step of collecting sound by the sound collecting unit having the sound collecting element, a step of detecting a sound source direction from the sound collecting unit based on sound data of the sound collected by the sound collecting unit, and The sound source direction from the sound collection unit is displayed on a display unit, the sound source direction from the sound collection unit displayed on the display unit is designated, and according to the designation of the sound source direction, A step of inputting information related to the imaging unit that images a predetermined position; a step of storing correspondence information in which the input information related to the imaging unit and the sound source direction from the sound collection unit are associated with each other in the storage unit; Having sound collection A constant way.

  According to the present invention, even when the positional relationship between the camera and the microphone array is unknown, the directivity can be appropriately formed at the predetermined imaging position, so that the sound at the predetermined imaging position can be clearly output.

The block diagram which shows the structure of the sound collection system in 1st Embodiment. Block diagram showing the configuration of the speech processing apparatus Block diagram showing the configuration of the microphone array device The figure which shows the structure of the packet of the audio | voice data transmitted to a sound processing apparatus from a microphone array apparatus The figure which shows the layout in the store where the sound collection system is installed Diagram explaining the outline of preset processing Flow chart showing sound collection procedure during preset processing and monitoring The figure which shows the screen of the display where the voice map is displayed at the time of preset processing The figure which shows the registration content of the preset information table stored in the microphone array apparatus The figure which shows the screen of the display which is displayed after the preset processing The figure which shows the sound generation operation | movement of the display screen and speaker displayed at the time of monitoring The block diagram which shows the structure of the sound collection system in 2nd Embodiment. Flow chart showing sound collection procedure during preset processing and monitoring FIG. 13 is a flowchart showing a sound collection procedure during preset processing and monitoring following FIG. Diagram showing the display screen displayed during preset processing The figure which shows the sound generation operation | movement of the display screen and speaker displayed at the time of monitoring The block diagram which shows the structure of the sound collection system in 3rd Embodiment. The figure which shows the registration contents of the preset information table stored in the table memory Flow chart showing preset processing procedure Diagram showing the display screen displayed during preset processing Flow chart showing the sound collection procedure during monitoring The figure which shows the sounding operation | movement of the display screen and speaker which are displayed at the time of monitoring The figure which shows the screen of the display displayed at the time of monitoring in the modification 1 of 3rd Embodiment. A table showing registered contents of the preset information table in the third modification of the third embodiment

  Embodiments that specifically disclose a sound collection system and a sound collection setting method according to the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram illustrating a configuration of a sound collection system 5 according to the first embodiment. The sound collection system 5 is installed in a store such as a convenience store, for example, and a plurality of monitoring camera devices C1 to Cn, a microphone array device MA, a recorder device 40, and a PC (Personal Computer) 30 are connected to the network 15. Are connected to each other.

  Each of the camera devices C1 to Cn is a fixed camera having a fixed angle of view, and captures images (including still images and moving images, and so on) around a predetermined position in each imaging target area. n is a positive value corresponding to the identification number of the camera device. The difference between the camera devices C1 to Cn is that the imaging target areas are different and all have the same configuration, so the configuration and operation of the camera device C1 will be described as an example. In the case of specifications different from those of the camera device C1, the camera device will be described each time. The camera device C <b> 1 transfers captured video data (video data) to the PC 30 via the network 15 and records it in the recorder device 40.

  The microphone array device MA is installed on the ceiling of the store 10 (see FIG. 5), for example, and a plurality of microphones M1 to Mn (see FIG. 3) are concentrically arranged downward (for example, eight). The sound in the store can be picked up. The microphone array device MA collects sound around the area to be imaged using each of the microphones M1 to Mn, and uses the network 15 to collect sound data (voice data) collected by each of the microphones M1 to Mn. To the PC 30 and to be recorded in the recorder device 40. Each of the microphones M1 to Mn may be an omnidirectional microphone, a bi-directional microphone, a unidirectional microphone, or a sharp directional microphone.

  The recorder device 40 includes a control unit (not shown) for controlling each process such as data recording, and a recording unit (not shown) for storing video data and audio data. The recorder device 40 records each video data imaged by the camera devices C1 to Cn in association with the audio data collected by the microphone array device MA.

  The PC 30 is configured to monitor an image picked up by the camera devices C1 to Cn and a sound collected by the microphone array device MA, and includes an audio processing device 50 and a video processing device 70.

  FIG. 2 is a block diagram showing a configuration of the audio processing device 50. The audio processing device 50 includes a signal processing unit 51, a memory 55, a communication unit 56, an operation unit 57, a display 58, and a speaker 59. The communication unit 56 receives the packet PKT (see FIG. 4) transmitted from the microphone array device MA or the recorder device 40 via the network 15 and outputs the packet PKT to the signal processing unit 51, and the signal processing unit 51 generates the communication unit 56. The preset information (see FIG. 9) is transmitted to the microphone array device MA. The memory 55 is configured using, for example, a RAM (Random Access Memory), functions as a work memory during operation by each unit of the audio processing device 50, and further stores data necessary for operation by each unit of the audio processing device 50. .

  The signal processing unit 51 is configured using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor), and a sound source direction detecting unit 52, a directivity forming unit 53, and an input / output control unit 54. Control processing for overall control of the operations of each unit of the PC 30, data input / output processing with other units, data calculation (calculation) processing, and data storage processing are executed.

  The sound source direction detection unit 52 estimates in which direction the sound source is present with respect to the sound collected by the microphone array apparatus MA. In the present embodiment, the sound source direction is represented by a horizontal angle θ and a vertical angle φ (see FIG. 6) centered on the microphone array apparatus MA, and is estimated using, for example, a sound volume. The horizontal angle θ is an angle in a horizontal plane (XY plane) in the real space with the center of the microphone array device MA as the origin, and the vertical angle φ is an inclination from the Z axis passing through the center of the microphone array device MA. . For example, when the sound source is near the microphone array apparatus MA, the vertical angle φ is detected as a small value.

  In addition, in order to perform a preset process, which will be described later, when sound is generated, a method of specifying a sound generation location (sound source direction) based not only on sound volume but also on sound characteristics is effective. As a method for characterizing the sound, for example, a sine wave with a constant frequency, a sine wave with a frequency changing at a constant period, white noise switching on / off at a constant period, a registered utterance, etc. The sound with the characteristic can be obtained. As a result, even when preset processing is performed in a place where noise is high (for example, indoors or shopping streets where interior work is being performed), the sound source direction detection unit 52 can select from the sounds collected by the microphone array device MA. , It can identify the sound with the characteristics. In addition, even when preset processing is performed in a quiet night and sound is generated, the sound source direction detection unit 52 is characterized even if the volume is low from the sound collected by the microphone array device MA. Sound can be identified.

  In addition, there are the following two methods for specifying the direction of a sound source that generates a characteristic sound. As a first method, the sound source direction detection unit 52 analyzes the sound characteristics in order from the place where the volume is high with respect to the sound picked up by the microphone array device MA, and determines the direction of the sound having the matched characteristics as the sound source. Judge the direction. As a second method, the sound source direction detection unit 52 divides the imaging target area, searches for a sound feature for each divided area (also referred to as a divided area), and determines the direction of the divided area where the features match as a sound source. Judging from the direction. In the second method, the sound source direction detection unit 52 retrieves sound data after a certain sound collection time and searches for the characteristics of the sound. Therefore, the sound volume is irrelevant and a small sound volume is sufficient. Accordingly, it is possible to specify a sound having a characteristic even in a noisy place where there is no inconvenience to the surroundings. As a third method, it is possible to make a determination based on the cross-correlation with the generated signal. As will be described later, the voice uttered from the sound source during the preset processing includes a voice uttered by a human, a buzzer sound, a sound output from a speaker, and the like.

  The directivity forming unit 53 uses the audio data directly transferred from the microphone array device MA or the audio data recorded in the recorder device 40 to collect sound by each of the microphones M1 to Mn by the audio data directivity control processing. In order to emphasize (amplify) the sound (volume level) in the specific direction from the positions of the microphones M1 to Mn of the microphone array device MA, the sound data in which directivity in the specific direction is formed is added. Is generated. The specific direction is a direction (also referred to as a pointing direction) from the microphone array device MA toward a position in the real space corresponding to the position designated by the operation unit 57. In addition, the technique regarding the directivity control process of the audio | voice data for forming the directivity of the audio | voice collected by the microphone array apparatus MA is shown by Unexamined-Japanese-Patent No. 2014-143678 (patent document 1 mentioned above) etc., for example. Thus, it is a known technique.

  The input / output control unit 54 controls input / output of various data to the operation unit 57, the display 58, and the speaker 59. The operation unit 57 is arranged corresponding to the screen of the display 58, for example, and is configured using a touch panel or a touch pad that can be input by a user's finger or stylus pen. The operation unit 57 outputs data of one or more designated locations (coordinates) for which enhancement (amplification) of the volume level of the audio data is desired to the signal processing unit 51 in accordance with a user operation. The operation unit 57 may be configured using a pointing device such as a mouse or a keyboard.

  The display 58 displays an audio map 65 indicating the sound source position estimated by the sound source direction detection unit 52. The speaker 59 is picked up by the microphone array device MA and transferred in a specific direction by the directivity forming unit 53 based on the sound data transferred via the network 15 or the sound data recorded in the recorder device 40 or the sound data. Audio data that has been subjected to the emphasis process is output.

  On the other hand, the video processing device 70 operates independently without being linked to the audio processing device 50, and performs control to display video data captured by the fixed camera devices C1 to Cn on the display 58 in accordance with an operation instruction from the user. . That is, when the user selects a camera device to be used, the video processing device 70 displays a video image from the selected camera device on the mounted camera monitor 71.

  FIG. 3 is a block diagram showing the configuration of the microphone array apparatus MA. The microphone array device MA collects sound in all directions (360 degrees), and includes a plurality of microphone units (also simply referred to as microphones) M1 to Mn (here, n = 8) and a plurality of microphone units M1 to Mn. A plurality of amplifiers (amplifiers) PA1 to PAn for amplifying output signals, a plurality of A / D converters A1 to An for converting analog signals output from the amplifiers PA1 to PAn into digital signals, an encoding unit 25, A storage unit 24 and a transmission unit 26 are included.

  The storage unit 24 stores a predetermined position (preset positions P1 to Pn) picked up by the camera devices C1 to Cn and a directivity direction from the microphone array device MA (specifically, a set of a horizontal angle θ and a vertical angle φ). The preset information representing the correspondence relationship is stored. The encoding unit 25 adds the preset information stored in the storage unit 24 to the digital audio signal output from the A / D converters A1 to An to generate a packet PKT of audio data. The transmission unit 26 transmits the packet PKT of the audio data generated by the encoding unit 25 to the audio processing device 50 via the network 15.

  As described above, the microphone array apparatus MA amplifies the output signals of the microphones M1 to Mn by the amplifiers PA1 to PAn, converts them into digital audio signals by the A / D converters A1 to An, and then stores them in the storage unit 24. The audio data packet PKT is generated by adding the preset information to the digital audio signal, and the audio data packet PKT is transmitted to the audio processing device 50 in the PC 30 via the network 15.

  FIG. 4 is a diagram showing the structure of a packet PKT of audio data transmitted from the microphone array device MA to the audio processing device 50. The audio data packet PKT is composed of audio data as a header and a payload. This header includes the preset information described above.

  In the present embodiment, the preset information is transmitted from the microphone array device MA to the sound processing device 50 by including the preset information in the packet PKT of the sound data. However, the sound processing device 50 acquires the preset information by another method. May be. For example, when the audio processing device 50 reads the initial information of the microphone array device MA, the preset information may be read together. Further, the microphone array apparatus MA may transmit the voice processing apparatus 50 in response to a request.

  FIG. 5 is a diagram showing a layout in the store 10 in which the sound collection system 5 is installed. As an example, a store 10 such as a convenience store includes “entrance / exit”, “register R1”, “register R2”, three rows of “product shelf”, “bento shelf”, “delivery shelf”, “beverage (drink) shelf ”And“ Magazine shelf ”are arranged. A microphone array device MA is installed on the ceiling of the store 10, and a plurality of camera devices C <b> 1 to Cn (here, n = 4) are installed on the upper wall or ceiling of the store 10.

  In FIG. 5, the plurality of camera devices C <b> 1 to Cn are directed to image a plurality of preset positions P <b> 1 to Pn set in the store 10 in advance. The plurality of preset positions P1 to Pn are determined by the user as an imaging target area to be monitored. The imaging ranges of the images captured by the respective camera devices C1 to Cn are represented by CR1 to CRn, respectively, and preset positions P1 to Pn exist at substantially the centers of the imaging ranges CR1 to CRn, respectively.

  The camera devices C1 and C2 capture images of the preset position P1 of “Register R1” and the preset position P2 of “Register R2”, which are monitoring target areas (sound collection areas), respectively. The camera device C3 captures an image of the preset position P3 of the “magazine shelf” that is the sound collection area. Thus, when the microphone array apparatus MA is attached later in the store 10 where the camera apparatuses C1 to C4 are already installed, a preset process described later is performed.

  Operation | movement of the sound collection system 5 which has the said structure is shown. First, a preset process performed before the operation (monitoring) of the sound collection system 5 is started will be described. Here, the preset processing is to set the directions from the microphone array device MA toward the predetermined positions (preset positions) P1 to Pn in the store 10 (that is, the horizontal angle θ and the vertical angle φ from the center of the microphone array device MA). It is processing to do. FIG. 6 is a diagram for explaining the outline of the preset process. On the ceiling RF of the store 10, a microphone array device MA and camera devices C1 to Cn are installed. Camera devices C1 to Cn, which are fixed cameras, are directed to preset positions P1 to Pn, and images captured by the camera devices C1 to Cn are displayed on a camera monitor 71 mounted on the video processing device 70. In the work associated with the preset processing, a sound source (sound source) is placed at the preset positions P1 to Pn. As described above, the sound source includes, for example, a human voice, a buzzer sound, a sound output from a speaker, and the like. FIG. 6 shows a case where a speaker 81 standing on the floor surface FLR speaks at preset positions P1 to Pn. When the microphone array device MA picks up the sound, it transmits the sound data to the sound processing device 50. The sound processing device 50 displays the sound source position of the collected sound as a sound source mark (marker) SD on the screen of the display 58 (a sound map 65 described later).

  The camera monitor 71 may be replaced with a display 58 included in the audio processing device 50. In addition, when the audio processing device 50 and the video processing device 70 are integrated as a monitoring device, the display can be used as a camera monitor by switching the display screen (window), and both can be simultaneously displayed in divided display. It is also possible to display.

  FIG. 7 is a flowchart showing a sound collection procedure during preset processing and monitoring. After the microphone array device MA is attached to the ceiling of the store 10, preset processing is performed. First, the voice processing device 50 performs initial setting of the microphone array device MA (S1). In this initial setting, the voice processing device 50 sets the IP address of the microphone array device MA and makes the microphone array device MA communicable. Further, the audio processing device 50 enters the preset mode and displays the audio map 65 (see FIG. 8) on the display 58.

  When the initial setting of the microphone array device MA is completed, a sound source (sound generation source) is placed at preset positions P1 to Pn to which the camera devices C1 to Cn are directed, and a sound source is sounded at a predetermined volume or more for a predetermined time (S2). Here, the speaker 81 utters at the preset positions P1 to Pn as a sound source. The microphone array device MA collects the sound and transmits the sound data to the sound processing device 50.

  The communication unit 56 in the sound processing device 50 receives the sound data transmitted from the microphone array device MA (S3). The sound source direction detection unit 52 in the sound processing device 50 obtains the directivity direction (horizontal angle θ and vertical angle φ) from the microphone array device MA toward the sound source based on the volume of the received sound data, and represents the sound source position. The sound source mark SD1 is displayed on the voice map 65 displayed on the display 58 (S4). FIG. 8 is a diagram showing a screen of the display 58 on which the audio map 65 is displayed during the preset process.

  The audio map 65 is drawn with three concentric circles 65h, 65i, 65j having a center point O at the position of the microphone array device MA and a line segment 65m having a radius that divides these central angles into 12 equal parts. Of the three concentric circles, the innermost concentric circle 65h corresponds to a vertical angle φ = 30 °, the middle concentric circle 65i corresponds to a vertical angle φ = 60 °, and the outermost concentric circle 65j has a vertical angle φ = 90 °. Equivalent to. Therefore, the closer to the sound source mark SD1, the closer to the microphone array device MA. A line segment 65m extending in the horizontal direction on the right side from the center point O has a center angle of 0 ° and corresponds to a horizontal angle θ = 0 °. A line segment 65m divided into 12 equals represents a horizontal angle of 0 ° to 360 ° in increments of 30 ° from the central angle of 0 °. Here, the sound source mark SD1 is drawn at coordinates (240 °, 70 °) as coordinates (θ, φ) on the audio map 65. Since this sound source mark SD1 is before being determined as the sound source at the preset position, it is drawn in a rectangle. In FIG. 8, for the sake of explanation, angles such as 30 ° and 60 ° are given, but they may not be displayed. In addition, the scale display may be performed with a vertical angle of, for example, every 15 °.

  The user (speaker) selects the sound source mark SD1 displayed on the display 58 with the cursor 87, and inputs information (camera information) of the camera device C1 corresponding to the sound source mark SD1 (S5). When the sound source mark SD1 is selected, a camera information input field 88 is displayed in the lower right corner of the screen of the display 58. In the camera information input field 88, a location (for example, cash register R1) and a camera IP address can be input by a user operation. Note that instead of the user selecting a sound source mark, if the sound volume picked up by the microphone array device continues for a predetermined time, the sound processing device automatically recognizes the direction of the sound. Then, a sound source mark may be displayed on the display to prompt the user to input camera information.

  The sound processing device 50 reads out the horizontal angle θ and the vertical angle φ of the sound source mark SD1 (S6), and uses the camera information input at step S5 and the horizontal angle θ and the vertical angle φ of the sound source mark SD1 as the microphone array device MA. (S7). The microphone array device MA registers the preset information transmitted from the sound processing device 50 in the preset information table 90 (see FIG. 9) and stores it in the storage unit 24.

  FIG. 9 is a diagram showing registration contents of the preset information table 90 stored in the microphone array apparatus MA. In the preset information table 90, a camera IP address, a location, and a directing direction (horizontal angle θ, vertical angle φ) are registered for each of the preset positions P1 to Pn included in the imaging target areas of the camera devices C1 to Cn.

  The user determines whether or not all preset position setting processes have been completed (S8). If all preset position setting processes have not been completed, the preset process returns to step S2 and the same process is repeated. .

  On the other hand, when the preset process is completed at all preset positions, the preset process before the start of operation is completed, and the operation (actual monitoring process) is started. The audio processing device 50 acquires the audio data from the microphone array device MA, and displays all preset positions extracted from the preset information included in the header of the packet PKT of the audio data on the display 58 (S9). FIG. 10 is a diagram showing a screen of the display 58 displayed after the preset process. On the audio map 65 displayed on the screen of the display 58, the sound source directions (direction directions) of the “registration R1”, “registration R2”, “magazine shelf”, and “beverage shelf” that are preset positions are determined, respectively. The sound source marks SD1, SD2, SD3, and SD4 are drawn in a circle. In particular, when it is not necessary to distinguish the sound source marks SD1 to SD4, they are simply referred to as a sound source mark SD. Camera information 67 is displayed at the lower right corner of the screen of the display 58. The camera information 67 includes preset positions and camera IP addresses corresponding to the camera devices C1 to Cn.

  When the preset position displayed on the display 58 is designated by the user via the operation unit 57 (S10), the audio processing device 50 sets the horizontal angle θ and the vertical angle φ representing the pointing direction of the designated preset position. Read (S11). The directivity forming unit 53 in the sound processing device 50 forms the directivity of the sound data in the directivity direction of the preset position specified by the read horizontal angle θ and vertical angle φ, and outputs the sound from the speaker 59 (S12). ).

  Further, when the user designates another position displayed on the display 58 during the reproduction of the sound collected during operation (that is, actual monitoring) (S13, YES), the sound processing device 50 has the designated position. The horizontal angle and the vertical angle are read from the preset information (S11), the directivity of the audio data is formed in the directivity direction, and the sound is output from the speaker 59 (S12).

  On the other hand, if there is no designation of a new designated position (S13, NO), the audio processing device 50 continues reproduction until the power is turned off (S14). The directivity formation may not be continued until the power is turned off by the user's instruction, but the directivity formation may be canceled by the user's support and the entire sound may be monitored until the next position is designated.

  FIG. 11 is a diagram illustrating the sound generation operation of the screen of the display 58 and the speaker 59 displayed during monitoring. During operation (monitoring), for example, when the user selects the sound source mark SD3 via the operation unit 57, the sound processing device 50 directs the direction of the sound source mark SD3 (θ3, φ3), that is, the direction of the magazine shelf (preset position P3). The directivity of the sound data is formed in the sound and the sound is collected, and this sound is output from the speaker 59. At this time, if the sound processing device 50 detects an abnormal sound (loud sound) whose volume exceeds the threshold value at or near the preset position P3, the sound processing mark SD3 blinks to notify the user. Here, the sound source mark is displayed so as to be identifiable by blinking, but the color, shape, size, etc. may be changed. Also, the audio processing device 50 changes the sound source mark color, changes the size or shape of the sound source mark, etc. according to the volume of the preset position, or changes only the corresponding camera information from the camera information. The color or background color may be changed.

  As described above, in the sound collection system 5 according to the first embodiment, when the microphone array device MA is attached later in the store 10 where the camera devices C1 to C4 are already installed, the preset processing includes the camera devices C1 to Cn. A sound source is placed at preset positions P1 to Pn in the optical axis direction, which is the center of imaging, and a sound is generated. When the microphone array device MA picks up the sound output from the sound source and sends the sound data to the sound processing device 50, the sound source direction detecting unit 52 indicates the directional direction (horizontal angle θ, vertical angle φ) on the display 58. The sound source marks SD to SD4 to be displayed are displayed to prompt the user to select the sound source marks SD to SD4 and to input camera information. The sound processing device 50 transmits the input camera information and the directivity direction to the microphone array device MA. The microphone array apparatus MA registers the camera information and the directing direction in the preset information table 90 and stores them in the storage unit 24. During operation, when the user selects any one of the sound source marks SD to SD4 on the sound map 65 displayed on the display 58, the directivity forming unit 53 for the sound data collected by the microphone array device MA, The directivity of the sound data is formed in the directivity direction (horizontal angle θ, vertical angle φ) corresponding to the sound source mark, and the sound processing device 50 causes the speaker 59 to output sound.

  As a result, the sound collection system 5 has a predetermined imaging position (that is, preset positions P1 to Pn) from the microphone array device MA even when the positional relationship between the camera devices C1 to Cn and the microphone array device MA is unknown. Directivity can be formed in the directivity direction toward, and the sound from the sound source that is pronounced there can be heard clearly. Therefore, in order to obtain the correspondence between the coordinate system of the camera device and the coordinate system of the microphone array device, the sound collection system 5 needs to measure the mounting position, direction, etc. on the site or perform geometric calculations. Therefore, the camera device and the microphone array device can be easily associated with each other. Further, the audio processing device 50 can obtain preset information using only the microphone array device MA.

  In operation (monitoring), the sound collection system 5 forms sound data directivity in the directivity direction (horizontal angle θ, vertical angle φ) associated with the preset position based on the preset information. The sound collected at the preset position can be output.

  The display 58 displays an audio map 65 drawn with concentric circles centered on the microphone array device MA, the central angle of which is a horizontal angle and the length of the radius is represented by a vertical angle. Since the sound source mark SDn is displayed on the top, the user can easily visually recognize the preset position Pn indicated by the sound source mark SDn.

  When the operation unit 57 receives the designation of the sound source mark SD displayed on the audio map 65, the directivity forming unit 53 causes the microphone array apparatus MA to move in the directivity direction associated with the designated sound source mark SD. Since the directivity of the voice data of the voice picked up at is formed, the user can listen to the voice uttered at the preset position Pn with a simple operation.

  Further, when the volume of the sound emitted at the preset position P exceeds the threshold value, the display 58 blinks and displays the sound source mark SD, so that a loud sound (abnormal sound) whose volume exceeds the threshold value is detected. This can be quickly notified to the user.

  Further, since the preset information is described in the header of the packet PKT of the audio data, it is possible to obtain the information on the direction of the preset position with only the audio data. In addition, since the preset information table 90 is stored in the storage unit 24 of the microphone array device MA, even when a plurality of microphone array devices are installed, it is not necessary to manage the correspondence between the microphone array device MA and the preset information. .

(Second Embodiment)
In the first embodiment, the audio processing device and the video processing device are operated separately. However, in the second embodiment, the audio processing device and the video processing device are integrated as a monitoring device, and are displayed on the display screen. The case where the image | video and audio | voice map imaged with the camera apparatus are displayed simultaneously is shown.

  FIG. 12 is a block diagram illustrating a configuration of a sound collection system 5A according to the second embodiment. The sound collection system of the second embodiment has almost the same configuration as that of the first embodiment. The same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  In the sound collection system 5 </ b> A, the monitoring device 100 is connected to the network 15. The monitoring apparatus 100 includes an audio processing unit 105, a video processing unit 107, an operation unit 117, a display 118, and a speaker 119.

  The sound processing unit 105 has the functions of a sound source direction detecting unit 52, a directivity forming unit 53, and an input / output control unit 54, like the signal processing unit 51 in the sound processing device 50 in the first embodiment. The video processing unit 107 displays video data captured by the fixed camera devices C1 to Cn on the display 118 in accordance with an operation instruction from the user.

  The operation unit 117 is arranged using, for example, a touch panel or a touch pad that is arranged corresponding to the screen of the display 118 and can be input with a user's finger or stylus pen. The operation unit 117 outputs, to the sound processing unit 105, data of coordinates of one or more designated locations for which enhancement (amplification) of the volume level of the sound data is desired in accordance with a user operation. Note that the operation unit 117 may be configured using a pointing device such as a mouse or a keyboard.

  The display 118 displays a video (image) based on video data captured by the camera devices C <b> 1 to Cn and transferred via the network 15 or recorded in the recorder device 40.

  The loudspeaker 119 is picked up by the microphone array device MA and transferred via the network 15, or recorded in the recorder device 40, or emphasized in a specific direction by the sound processing unit 105 based on the sound data. The processed voice data is output.

  The operation of the sound collection system 5A having the above configuration will be described. FIG. 13 is a flowchart showing a sound collection procedure during preset processing and monitoring. FIG. 14 is a flowchart showing a sound collection procedure during preset processing and monitoring following FIG. The same step number is assigned to the same step process as in the first embodiment, and the description thereof is omitted.

  In step S1, the voice processing unit 105 sets the IP address of the microphone array device MA to make the microphone array device MA communicable. Further, the sound processing unit 105 enters the preset mode and displays the sound map 65 on the display 118. The video processing unit 107 searches for the camera devices C1 to Cn connected to the network 15 by broadcasting to all the camera devices C1 to Cn connected to the network 15 and receiving the responses. Detect (S1A).

  The voice processing unit 105 stores the total number n of camera devices and each IP address obtained as a result of the search in a memory (not shown) in the voice processing unit 105 (S1B). The video processing unit 107 displays the video captured by the searched camera devices C1 to Cn on the screen of the display 118. FIG. 15 is a diagram showing a screen of the display 118 displayed during the preset process. On the left side of the screen of the display 118, thumbnails SZ1 to SZ4 of images captured by the camera devices C1 to Cn are displayed in a selectable manner. In particular, when there is no need to distinguish the thumbnails SZ1 to SZ4, they are simply referred to as thumbnails SZ. In addition, the thumbnail SZ is displayed by taking out a still image at regular time intervals from videos captured by the camera devices C1 to Cn. In addition, the sound processing unit 105 displays the sound map 65 on the right side from the center of the screen of the display 118.

  The audio processing unit 105 sets a variable i representing the camera device number to an initial value 0 (S1C). Then, the voice processing unit 105 increases the variable i by 1 (S1D). The audio processing unit 105 receives the thumbnail SZ selected by the user via the operation unit 117 (S1E). In this thumbnail selection, the user moves the cursor 123 displayed on the screen of the display 118 to select the thumbnail SZ. In FIG. 15, the thumbnail SZ3 is selected. The frame of the selected thumbnail SZ3 is highlighted in red or the like. Note that the sound source mark SD may be selected instead of the thumbnail SZ as in the first embodiment.

  The user installs a sound source in the imaging range captured by the camera devices C1 to Cn corresponding to the thumbnail SZ, and generates a sound for a predetermined time at a predetermined volume or more (S2). The position of the sound source does not have to be on the optical axis of the camera devices C1 to Cn, and may be within the imaging range. Further, as in the first embodiment, the user may utter while standing within the imaging range as a sound source.

  When the microphone array device MA picks up a sound of a predetermined volume or more emitted from the sound source and transmits the sound data to the sound processing unit 105, the sound processing unit 105 receives the sound data transmitted from the microphone array device MA. (S3).

  The sound processing unit 105 obtains a directivity direction (horizontal angle θ and vertical angle φ) from the microphone array apparatus MA toward the sound source based on the volume of the received sound data, and displays a sound source mark SD indicating the sound source position on the display 118. It is displayed on the displayed voice map 65 (S4). On the display 118, a sound source mark SD3 representing a new sound source position is drawn in a rectangular shape (see FIG. 15). Since the sound source marks SD1 and SD2 of “Register R1” and “Register R2” have already been determined, they are drawn in a round shape. Furthermore, the voice processing unit 105 displays an input field 129 for a camera name (for example, a place name) in the lower right corner of the screen of the display 118 to prompt input.

  The user selects the thumbnail SZ or the sound source mark SD, and inputs the camera information into the camera name input field 129 (S5A). Note that, instead of the user selecting the thumbnail SZ or the sound source mark SD, if the sound volume collected by the microphone array apparatus continues for a predetermined time period, the sound processing apparatus automatically selects the sound. A sound source mark may be displayed on the display by recognizing the generation direction, and the user may be prompted to input camera information.

  The sound processing unit 105 reads the horizontal angle θ and the vertical angle φ of the sound source mark SD (S6), the camera information (camera name and IP address) input in step S5A, and the directivity direction (horizontal angle) of the microphone array device MA. θ and vertical angle φ) are transmitted to the microphone array device MA (S7). The microphone array apparatus MA registers the preset information transmitted from the audio processing unit 105 in the preset information table 90 and stores it in the storage unit 24.

  The voice processing unit 105 determines whether or not the variable i has reached the total number n of searched camera devices (S8A). If the variable i has not reached the total number n of camera devices, the audio processing unit 105 returns to step S1D and repeats the same processing. On the other hand, when the variable i reaches the total number n of camera devices, the preset process is completed, and the process proceeds to the operation (monitoring) process.

  At the time of monitoring, the voice processing unit 105 acquires all the preset positions P1 to Pn from the microphone array device MA and displays them on the voice map 65 displayed on the display 118 (S9A). The video processing unit 107 reads the video captured by the camera devices C1 to Cn and displays it on the screen of the display 118 (S9B). FIG. 16 is a diagram showing the screen of the display 118 displayed during monitoring and the sound generation operation of the speaker 119. Here, a case where eight camera apparatuses C1 to C8 are installed is shown. Images GZ1 to GZ8 captured by the camera devices C1 to C8 are displayed on the left side of the screen of the display 118. Here, the images GZ1 to GZ8 are not thumbnails, and the camera devices C1 to C8 are “Register R1”, “Register R2”, “Register R3”, “Entry”, “Magazine shelf T2”, and “Magazine shelf T1”, respectively. , “Passage” and “Passport”.

  An audio map 65 and an operation panel 140 are displayed on the right side of the screen of the display 118. The sound map 65 displays sound source marks SD1 to SD8. In addition, the operation panel 140 includes any one of a luminance button 141 for adjusting the brightness of the images GZ1 to GZ8, a focus button 142 for adjusting the focus of an image captured by the camera devices C1 to C8, and the camera devices C1 to C8. A selection button 143 for selecting, a volume button 145 for adjusting the volume, and a preset button 146 for switching from directivity sound collection to a state of collecting the whole sound are provided.

  When outputting sound from the speaker 119, the sound processing unit 105 receives the sound source mark SD or the image GZ designated by the user (S10A). The user selects the sound source mark SD on the sound map 65 displayed on the screen of the display 118 by clicking with the cursor 123 or selects the images GZ1 to GZ8 displayed on the screen of the display 118 with the cursor 123. To do. Here, the image GZ5 or the sound source mark SD5 is selected, the frame of the image GZ5 is highlighted in red, and the sound source mark SD5 is surrounded by a rectangle having a red background color. The speaker 119 outputs audio data in which the “magazine shelf T2” is directed.

  Further, when the user designates another position displayed on the display 58 during the reproduction of the sound collected during operation (that is, actual monitoring) (S13, YES), the sound processing device 50 has the designated position. The horizontal angle and the vertical angle are read from the preset information (S11), the directivity of the audio data is formed in the directivity direction, and the sound is output from the speaker 59 (S12).

  On the other hand, if there is no designation of a new designated position (S13, NO), the audio processing device 50 continues reproduction until the power is turned off (S14). The user can click the preset button 146 to newly add, change, or delete the contents of the preset table, and the entire sound may be monitored until the next position is designated.

  As described above, the sound collection system 5A of the second embodiment can perform the preset processing for registering the sound directivity direction corresponding to the preset position while actually viewing the video imaged by the camera devices C1 to Cn. Workability at the time of preset processing is improved. For example, when the sound source is placed at a preset position (the same applies when the sounder stands at the preset position), it can be easily understood that the sound source may be placed at the center of the image captured by the camera device. Further, when switching the directivity direction of the microphone array device MA during monitoring, the user can determine the switching destination by looking at the images captured by the camera devices C1 to Cn.

(Third embodiment)
In the first and second embodiments, the preset information is stored in the microphone array device. In the third embodiment, a plurality of microphone array devices are installed, and the monitoring device centrally manages the preset information. .

  FIG. 17 is a block diagram illustrating a configuration of a sound collection system 5B according to the third embodiment. The sound collection system of the third embodiment has almost the same configuration as that of the first embodiment. About the same component as 1st Embodiment, the description is abbreviate | omitted by using the same code | symbol.

  A plurality of microphone array devices MA1 to MAm are connected to the network 15. Unlike the first and second embodiments, the plurality of microphone array devices MA to MAm do not have a storage unit for storing preset information. In the present embodiment, a case where two microphone array devices MA1 and MA2 are connected to the network 15 is shown. Three or more microphone array devices may be connected.

  The monitoring apparatus 100A includes a table memory 110 that stores a preset information table 130 in which preset information is registered. FIG. 18 is a diagram showing the registration contents of the preset information table 130 stored in the table memory 110.

  In the preset information table 130, a location, a preset value, and a camera IP address are registered. Also, the microphone array device number (Mic No.), directivity direction, and directivity control parameter are registered as preset values. The directivity control parameter is a coefficient of the directivity filter, and is determined by performing learning control in each directivity direction.

As preset information, for example, location: cash register R1, Mic No. : MA1, directivity direction: (θ ₁₁ , φ ₁₁ ), directivity control parameters (p ₁₁₁ ,..., P _11q ), camera IP address: “165.254.10.11” are registered. In the magazine shelf, preset information is registered in duplicate by the microphone array device MA1 and the microphone array device MA2. That is, as preset information, location: magazine shelf, Mic No. : MA1, directivity direction: (θ ₁₃ , φ ₁₃ ), directivity control parameter (p ₁₃₁ ,..., P _13q ), camera IP address: “165.254.10.13”, and place: magazine shelf, Mic No. : MA2, directivity direction: (θ ₂₃ , φ ₂₃ ), directivity control parameter (p ₂₃₁ ,..., P _23q ), and camera IP address: “165.254.10.13” are registered. When preset information is registered in duplicate for the same preset position, the preset information with the larger volume among the sounds collected by the two microphone array devices MA1 and MA2 is used preferentially. The sound collected by the microphone array apparatus MA corresponding to the preset information is output from the speaker 119.

  Operation | movement of the sound collection system 5B which has the said structure is shown. FIG. 19 is a flowchart showing a preset processing procedure. About the same step process as 1st, 2nd embodiment, the description is abbreviate | omitted by attaching | subjecting the same step number. Moreover, the case where there are two microphone array devices MA1 and MA2 is shown.

  The audio processing unit 105 in the monitoring device 100A sets an IP address for each of the microphone array devices MA1 and MA2 so that the microphone array devices MA1 and MA2 can communicate with each other (step S1Z). Further, the sound processing unit 105 enters the preset mode and displays two sound maps 65A and 65B on the display 118. The video processing unit 107 searches for the camera devices C1 to Cn connected to the network 15 by broadcasting to all the camera devices C1 to Cn connected to the network 15 and receiving a response thereof ( S1A).

  The voice processing unit 105 stores the total number n of camera devices and each IP address obtained as a result of the search in a memory (not shown) in the voice processing unit 105 (S1B). The video processing unit 107 displays the video captured by the searched camera devices C1 to Cn on the screen of the display 118. FIG. 20 is a diagram showing a screen of the display 118 displayed during the preset process. On the left side of the screen of the display 118, thumbnails SZ1 to SZ4 of images captured by the camera devices C1 to Cn are displayed in a selectable manner. In particular, when there is no need to distinguish the thumbnails SZ1 to SZ4, they are simply referred to as thumbnails SZ. In addition, the thumbnail SZ is displayed by taking out a still image at regular time intervals from videos captured by the camera devices C1 to Cn. In addition, the sound processing unit 105 displays two sound maps 65A and 65B on the right side from the center of the screen of the display 118.

  In step S2, when the user installs a sound source in the imaging range imaged by the camera devices C1 to Cn corresponding to the thumbnail SZ and generates sound for a predetermined time at a predetermined sound volume or higher, the microphone array devices MA1 to MAm respectively emit sound from the sound source. The sound of the predetermined volume or higher is collected, and each sound data is transmitted to the sound processing unit 105. The audio processing unit 105 receives the audio data transmitted from each of the microphone array devices MA1 and MA2 (S3A).

  The sound processing unit 105 obtains the directivity directions (horizontal angle θ and vertical angle φ) from the microphone array devices MA1 and MA2 toward the sound source based on the volume of the sound data received from the respective microphone array devices MA1 and MA2. The sound source mark SD (SD3A, SD3B) representing the sound source position is displayed on the voice maps 65A, 65B displayed on the display 118 (S4A). On the display 118, sound source marks SD3A and SD3B representing new sound source positions are drawn in a rectangular shape (see FIG. 20). Since the sound source marks SD1 and SD2 of “Register R1” and “Register R2” have already been determined, they are drawn in a round shape. Furthermore, the voice processing unit 105 displays an input field 129 for a camera name (for example, a place name) in the lower right corner of the screen of the display 118 to prompt input. Here, since the sound collected by the microphone array apparatus MA2 has a higher volume, the rectangular size of the sound source mark SD3B is larger than that of the sound source mark SD3A corresponding to the microphone array apparatus MA1.

  The audio processing unit 105 selects one of a plurality of microphone array devices (here, the microphone array devices MA1 and MA2) to be subjected to preset processing (S4B). When sound having a predetermined volume or higher is picked up by a plurality of microphone array devices, the microphone array devices MA1 and MA2 are selected by one of the following three methods. In the first method, the sound processing unit 105 selects a microphone array device having a larger volume among the microphone array devices MA1 and MA2. In the second method, the user selects one of the microphone array devices MA1 and MA2. In the third method, the sound volume is compared with a threshold value, and a microphone array apparatus that picks up sound with a sound volume equal to or higher than the threshold value is selected. In this case, a plurality of microphone array devices may be selected.

  The user selects the thumbnail SZ or the sound source mark SD for the selected microphone array device MA, and inputs the camera information to the camera name input field 129 (S5B). The subsequent steps S6 to S8A are the same as those in the second embodiment. When the variable i reaches the total number n of camera devices in step S8A, the sound processing unit 105 ends this operation.

  FIG. 21 is a flowchart showing a sound collection procedure during monitoring. The video processing unit 107 in the monitoring device 100A selects any one of the camera devices C1 to Cn, and displays the video imaged by the selected camera devices C1 to Cn on the display 118 (S21).

  FIG. 22 is a diagram illustrating the screen of the display 118 displayed during monitoring and the sound generation operation of the speaker 119. On the left side of the screen of the display 118, a pull-down menu 160 for various items is displayed. Here, the pull-down menu of the device tree is expanded and the camera device C2 is selected. A monitor screen 150 on which an image picked up by the selected camera device C2 is displayed is arranged at the upper part of the center of the display 118. An operation panel 140 </ b> A is arranged at the lower portion of the center of the display 118. The operation panel 140A includes a brightness button 141 for adjusting the brightness of the image, a focus button 142A for adjusting the focus of the image captured by the camera devices C1 to C8, and a selection button 143 for selecting any of the camera devices C1 to C8. A zoom button 147 for performing a zooming operation, and a preset input field 146A for inputting a preset position newly are provided.

  The audio processing unit 105 reads preset information corresponding to the selected camera information (S22). The audio processing unit 105 forms the directivity of the audio data in the directivity direction (horizontal angle θ, vertical angle φ) obtained from the preset information (S23). The audio processing unit 105 determines whether or not there are a plurality of preset microphone array devices MA (S24). When there are a plurality of microphone array devices MA, the audio processing unit 105 selects, for example, the microphone array device MA having the largest volume determined during the preset process (S25).

  The audio processing unit 105 outputs the audio data in which directivity is formed by the selected microphone array device MA from the speaker 119 (S26). In FIG. 22, “I welcome you” is output from the speaker 119. Thereafter, the sound processing unit 105 returns to step S21 and repeats the same operation.

  As described above, since the sound collection system 5B of the third embodiment includes the plurality of microphone array devices MA, sound collected in the store can be collected using the microphone array device that is easy for the user to hear. In addition, if multiple microphone array devices can pick up sound at a predetermined volume or higher at the preset position, the microphone array device that picks up the sound at the loudest volume will perform preset processing so that even small sounds will not be leaked. Can be heard. Furthermore, even when one microphone array device fails, the other microphone array device can be used to listen to the sound at the same preset position.

  Further, since the monitoring apparatus 100A centrally manages the preset information, each microphone array apparatus does not need to have a storage unit for storing the preset information, and the configuration can be simplified. In addition, the voice processing unit 105 does not need to transmit preset information to each microphone array apparatus MA, so that the processing load can be reduced and network traffic can be reduced.

(Modification 1)
FIG. 23 is a diagram illustrating a screen of the display 118 </ b> A displayed at the time of monitoring in the first modification of the third embodiment. A monitor screen divided into nine parts is arranged in a portion excluding the lower part of the screen of the display 118A. On the monitor screen, images GZ1A to GZ8A and GZ9 captured by the camera devices C1 to C9 are displayed slightly larger. A pull-down menu 160A is arranged on the lower left side of the screen. An operation panel 140B is arranged on the lower right side of the screen. The operation panel 140B is the same as the second embodiment except that a preset input field 146A is arranged instead of the preset button 146. In the first modification, the audio map is not displayed.

  In the monitoring apparatus according to the first modification, when the user selects a place to be heard from among a plurality of videos displayed on the screen of the display 118 </ b> A, the speaker 119 outputs the sound of the taken place. In addition, when the audio processing unit 105 receives audio data in which a loud sound is generated, the video processing unit 107 notifies the user of the generation location by changing the color of the frame of the video where the large sound is generated. Note that the sound switching is performed manually or automatically.

(Modification 2)
In the second modification of the third embodiment, the plurality of microphone array devices MA1 to MAn each have a storage unit, and the monitoring device integrates preset information received from each of the plurality of microphone array devices MA1 to MAn. Create an information table. The monitoring apparatus stores the created preset information table in the table memory. The integrated preset information stored as a preset information table in the table memory is transmitted to each of the microphone array devices MA1 to MAn.

  As a result, even when a new microphone array device is connected, the monitoring device does not need to perform a new preset process. New preset information is registered simply by acquiring and integrating preset information from the microphone array device. A preset information table can be obtained. In addition, even when another monitoring device is added, by transmitting preset information from the microphone array device to another monitoring device, by integrating the preset information transmitted from the plurality of microphone array devices by another monitoring device, A preset information table can be obtained. In this way, it is possible to easily construct a sound collection system in which a plurality of microphone array devices and a plurality of monitoring devices are combined.

(Modification 3)
In the third modification of the third embodiment, one of the plurality of camera devices C1 to Cn has a pan / tilt function, a zoom-in function, and a zoom-out function (hereinafter referred to as a PTZ function) that can be remotely operated from the monitoring device. ) Having a PTZ camera. The PTZ camera uses preset locations as preset values, and stores the pan / tilt angles and zoom values in a memory.

  When a plurality of preset positions are set in the PTZ camera, if a fixed camera is included in addition to the PTZ camera among the plurality of camera devices C1 to Cn, the number of times of performing the microphone preset is not the total number n of cameras, The preset number N takes into consideration the sum of the number of fixed cameras and the preset number of PTZ cameras.

  FIG. 24 is a table showing registration contents of the preset information table 130A in the third modification of the third embodiment. In the preset information table 130A, a location, a microphone preset value, a camera IP address, and a camera preset value are registered. The registration of the location, microphone preset value, and camera IP address is the same as in the preset information table 130 shown in FIG. As for the new camera preset value, since the imaging position does not change in the case of a fixed camera, the camera preset value is “Null”. On the other hand, in the case of a PTZ camera, the imaging position viewed from the PTZ camera (in other words, the direction of orientation from the microphone array device) changes between the magazine shelf T2 and the passage U1, so the camera preset values are “PT1” and “PT2”. is there.

  When a target area (location) to be imaged by the PTZ camera such as the magazine shelf T2 and the passage U1 is selected, the monitoring device reads the audio data in the direction of the microphone array device, and at the same time, sets the camera preset value to the PTZ camera. Send. The PTZ camera captures an image in the imaging direction corresponding to the preset value. By using the PTZ camera, the imaging area to be monitored can be easily changed. In Modification 3, a PTZ camera is used instead of a fixed camera, but an omnidirectional camera may be used.

  While various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  The present invention provides a sound collection system that appropriately forms directivity at a predetermined imaging position and clearly outputs sound at the predetermined imaging position even when the positional relationship between the camera and the microphone array is unknown. This is useful as a sound collection setting method.

5, 5A, 5B Sound collection system 10 Store 15 Network 24 Storage unit 25 Coding unit 26 Transmitting unit 40 Recorder device 50 Audio processing device 51 Signal processing unit
52 Sound source direction detection unit 53 Directivity forming unit 54 Input / output control unit 55 Memory 56 Communication unit 57, 117 Operation unit 58, 118 Display 59, 119 Speaker 65, 65A, 65B Audio map 65h, 65i, 65j Concentric circle 65m Line segment 67 Camera information 70 Video processing device 71 Camera monitor 81 Speaker 87, 123 Cursor 88 Input field 90, 130, 130A Preset information table 100 Monitoring device 105 Audio processing unit 107 Video processing unit 110 Table memory 129 Input field 140, 140A, 140B Operation Panel 141 Brightness button 142, 142A Focus button 143 Select button 145 Volume button 146 Preset button 146A Preset input field 147 Zoom button 150 Monitor screen 160, 160A Down menu A1-An A / D converter C1-Cn Camera device CR1-CRn Imaging range FLR Floor GZ1-GZn, GZ1A-GSnA Image M1-Mn Microphone MA, MA1-MAn Microphone array device O Center point P1-Pn Preset Position PA1 to PAn Amplifier (Amplifier)
PKT packet RF ceiling SD, SD1-SDn, SD3A, SD3B Sound source mark SZ1-SZ4 Thumbnail

Claims (10)

A sound collection unit having a plurality of sound collection elements, and collecting sound by the sound collection elements;
At least one imaging unit for imaging a predetermined position;
A sound source detection unit that detects a sound source direction from the sound collection unit, based on the sound data of the collected sound;
A display unit that displays a detected sound source direction from the sound pickup unit according to a predetermined sound output at the predetermined position;
An operation unit that receives input of information related to the imaging unit that images the predetermined position in accordance with the designation of the sound source direction from the displayed sound collection unit;
A storage unit that stores correspondence information in which the input information about the imaging unit and the sound source direction from the sound collection unit are associated with each other;
Sound collection system. The sound collection system according to claim 1,
Based on the correspondence information, a directivity forming unit that forms the directivity of the voice data of the sound collected by the sound collection unit in the sound source direction associated with the predetermined position;
An output unit that outputs voice data in which directivity is formed by the directivity forming unit;
Sound collection system. The sound collection system according to claim 2,
The display unit displays a marker indicating the sound source direction from the sound collection unit on a concentric sound map centering on the position of the sound collection unit, the center angle being a horizontal angle and the length of the radius representing a vertical angle. To
Sound collection system. The sound collection system according to claim 3,
The operation unit accepts designation of the marker displayed on the voice map,
The directivity forming unit forms the directivity of the voice data of the voice collected by the sound collecting unit in a sound source direction corresponding to the designated marker.
Sound collection system. The sound collection system according to claim 4,
The display unit identifies the marker corresponding to the sound source direction in which the directivity is formed on the audio map when the volume of the audio data in which directivity is formed by the directivity forming unit exceeds a threshold value. Display as possible,
Sound collection system. The sound collection system according to claim 2,
The sound collection unit includes the storage unit that stores the correspondence information, and transmits data in which the correspondence information is added to voice data of the collected voice to the directivity forming unit.
Sound collection system. The sound collection system according to claim 1,
A plurality of the imaging units are provided,
The display unit displays an image of each predetermined position imaged by each imaging unit,
The operation unit designates the sound source direction by a selection operation of the image at the predetermined position displayed on the display unit.
Sound collection system. The sound collection system according to claim 1,
A plurality of the sound collection unit and the imaging unit are provided,
The storage unit stores, for each of the plurality of sound collection units, the correspondence information in which information on any one of the imaging units and a sound source direction from the sound collection unit are associated with each other.
Sound collection system. The sound collection system according to claim 8,
The correspondence information includes the sound source directions by a plurality of the sound collection units at the overlapped default positions.
Sound collection system. A sound collection setting method in a sound collection system including at least one image pickup unit that picks up a predetermined position and a sound pickup unit,
Collecting a predetermined output sound of the sound source placed at the predetermined position by the sound collecting unit having a plurality of sound collecting elements;
Detecting the sound source direction from the sound collection unit based on the sound data of the sound collected by the sound collection unit;
Displaying the detected sound source direction from the sound collection unit on a display unit;
A sound source direction from the sound collection unit displayed on the display unit is designated;
In response to designation of the sound source direction, information regarding the imaging unit that images the predetermined position is input;
Storing correspondence information in which information about the input image capturing unit and a sound source direction from the sound collecting unit are associated with each other in a storage unit,
Sound pickup setting method.

Images