JP2019201377A - Imaging apparatus, imaging system, signal processing method, and program - Google Patents

Abstract

To make it possible to clearly collect a target sound even when a position of a subject to be collected is far from a front of a device.SOLUTION: The imaging apparatus includes: an imaging unit having an imaging element; a first microphone group including a plurality of microphones arranged in a direction substantially parallel to an optical axis of the imaging unit; and a second microphone group including a plurality of microphones arranged in a direction substantially perpendicular to the optical axis of the imaging unit. If a range of a sound collection target is wider than a predetermined angle of view, directivity processing is performed using sound signals obtained by the first microphone group. If a range of the sound collection target is narrower than the predetermined angle of view, directivity processing is performed using sound signals obtained by the second microphone group.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an imaging apparatus, an imaging system, a signal processing method, and a program.

  2. Description of the Related Art Conventionally, there is an image pickup apparatus that is equipped with a microphone and can collect voice of conversation and the like. In such an imaging apparatus, it is desired that the target sound can be clearly collected. In other words, it is desirable to be able to collect sound by removing as much as possible undesired sound (for example, in the case of collecting conversational sound, the driving sound of an air conditioner, etc.). For example, in Patent Document 1, by processing each sound signal collected by two mounted microphones, the voice of the responding operator (target sound) and other room sounds (not intended) An intercom device that separates sound) and makes it easy to hear the voice of the responding operator is disclosed.

JP 2017-34490 A

  However, although the technique disclosed in Patent Document 1 is suitable for a purpose of clearly collecting sound from the front direction of the apparatus (voice of the responding operator), It is not suitable for applications that clearly collect other room sounds. Therefore, an object of the present invention is to make it possible to clearly collect a target sound even when the position of a subject to be collected is away from the front direction of the apparatus.

  An imaging apparatus according to the present invention includes an imaging unit having an imaging element, a first microphone group including a plurality of microphones arranged in a direction substantially parallel to the optical axis of the imaging unit, and light of the imaging unit And a second microphone group including a plurality of microphones arranged apart from each other in a direction substantially perpendicular to the axis.

  According to the present invention, it is possible to clearly collect a target sound even when the position of a subject to be collected is away from the front direction of the apparatus.

It is a figure which shows the example of the hardware constitutions of the imaging system in this embodiment. It is a figure which shows the example of a function structure of the imaging system in this embodiment. It is a figure explaining the example of the imaging device in this embodiment. It is a figure which shows the structural example which concerns on the sound signal process in this embodiment. It is a figure explaining the sound signal process in this embodiment. It is a figure explaining the sound signal process in this embodiment. It is a flowchart which shows the example of the sound signal process in this embodiment. It is a figure which shows the example of a setting of the amplification amount of the amplifier in this embodiment. It is a figure explaining the other example of selection of the microphone group used in this embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating an example of a hardware configuration of an imaging system according to an embodiment of the present invention. The imaging system according to the present embodiment includes an imaging device 110 having an imaging unit and a plurality of microphones, and a client device 120 that can execute operations on the imaging device 110 and output images and sounds obtained by the imaging device 110. The imaging device 110 and the client device 120 are communicably connected via a network 130 such as an IP (Internet Protocol) network.

  The imaging device 110 includes a CPU 111, a ROM 112, a RAM 113, a communication interface (communication I / F) 114, an imaging unit 115, a microphone group 116, and a storage device 117. The CPU 111, the ROM 112, the RAM 113, the communication I / F 114, the imaging unit 115, the microphone group 116, and the storage device 117 are communicably connected via a transmission path 118 such as a system bus.

  A CPU (Central Processing Unit) 111 controls various devices of the imaging apparatus 110 connected via a transmission path 118 such as a system bus. A ROM (Read Only Memory) 112 stores a startup program of the imaging device 110 and the like. A RAM (Random Access Memory) 113 is used as a main storage device of the CPU 111. The communication I / F 114 connects the imaging apparatus 110 to the network 130 and controls information communication via the network 130.

  The imaging unit 115 has an imaging element, images a subject, and outputs an image signal of an image including the subject. The microphone group 116 includes a plurality of microphones, collects sounds around the imaging device 110, and outputs a sound signal of the collected sounds. The storage device 117 is, for example, a nonvolatile semiconductor storage device, and stores a control program related to the operation and processing of the imaging device 110. Note that the storage device 117 may store the image signal of the image output from the imaging unit 115 as necessary.

  In the imaging device 110 configured as described above, when the imaging device 110 is powered on, the CPU 111 reads a control program or the like from the ROM 112 or the storage device 117 into the RAM 113 in accordance with a startup program stored in the ROM 112. The CPU 111 realizes the function of the imaging device 110 by executing processing according to a control program or the like read into the RAM 113. That is, the functions and processing of the imaging device 110 are realized by the CPU 111 of the imaging device 110 executing processing based on a control program or the like.

  The client device 120 includes a CPU 121, ROM 122, RAM 123, communication interface (I / F) 124, output device 125, input device 126, and storage device 127. The CPU 121, ROM 122, RAM 123, communication I / F 124, output device 125, input device 126, and storage device 127 are communicably connected via a transmission path 128 such as a system bus.

  The CPU 121 controls various devices of the client apparatus 120 connected via a transmission path 128 such as a system bus. The ROM 122 stores a BIOS program and a boot program. The RAM 123 is used as a main storage device for the CPU 121. The communication I / F 124 connects the client device 120 to the network 130 and controls information communication via the network 130.

  The output device 125 outputs a processing result in the CPU 121 and the like. The output device 125 displays an image based on an image signal output from the imaging device 110 on an image display unit such as a display, or outputs a sound based on a sound signal output from the imaging device 110 to a sound output from a speaker or the like. Or output from the section. The input device 126 receives an input by the user. The storage device 127 stores an operating system (OS) program, various application programs operating on the OS, and the like. The storage device 127 is, for example, a hard disk drive (HDD) or a solid state drive (SSD).

  When power is turned on in the client device 120 configured as described above, the CPU 121 reads an OS program or the like from the storage device 127 or the like into the RAM 123 in accordance with a boot program stored in the ROM 122. The CPU 121 implements the function of the client device 120 by executing processing according to an OS program or the like read into the RAM 123. That is, the functions and processing of the client device 120 are realized by the CPU 121 of the client device 120 executing processing based on the program.

  FIG. 2 is a block diagram illustrating an example of a functional configuration of the imaging system according to the present embodiment. The imaging system in the present embodiment includes an imaging unit 210, a first microphone group 220, a second microphone group 230, an image processing unit 240, a sound signal processing unit 250, an output unit 260, a storage unit 270, a control unit 280, and An operation unit 290 is provided.

  The imaging unit 210 includes an optical system 211 and an imaging element 212, and performs imaging and outputs an image signal of an image including the captured subject. The optical system 211 is an optical system composed of a lens or the like, and includes a focus lens and its drive system. The imaging device 212 is an imaging device such as a CMOS image sensor, for example, and photoelectrically converts the optical image formed by the optical system 211 and outputs the obtained image signal.

  The first microphone group 220 and the second microphone group 230 each have a plurality of microphones 221 and 231 for collecting sound and output the obtained sound signals. The plurality of microphones 221 included in the first microphone group 220 are arranged apart from each other in a direction substantially parallel to the optical axis of the imaging unit 210 included in the imaging device. In addition, the plurality of microphones 231 included in the second microphone group 230 are arranged away from each other in a direction substantially perpendicular to the optical axis of the imaging unit 210 included in the imaging device.

  For example, as shown in FIG. 3A and FIG. 3B, for example, the distance D1 is approximately parallel to the optical axis 310 of the imaging unit included in the imaging apparatus 300, and the first Microphones 221A and 221B belonging to the microphone group 220 are arranged in order. Further, the microphones 231A and 231B belonging to the second microphone group 230 are sequentially arranged so as to have a distance D2 in a direction substantially perpendicular to the optical axis 310 of the imaging unit and to be substantially symmetric with respect to the optical axis 310 of the imaging unit. Has been placed. Here, FIG. 3A is an external view of the imaging device 300, and FIG. 3B is a cross-sectional view of the imaging device. The arrangement of the plurality of microphones belonging to the first microphone group 220 and the plurality of microphones belonging to the second microphone group 230 shown in FIGS. 3A and 3B is an example, and in this embodiment The arrangement of the microphone is not limited to this.

  The image processing unit 240 performs image processing related to the image signal obtained by the imaging unit 210. The sound signal processing unit 250 performs sound signal processing related to the sound signal obtained by the microphone 221 of the first microphone group 220 and the microphone 231 of the second microphone group 230. The sound signal processing performed by the sound signal processing unit 250 includes directivity processing related to the sound signal described later. The image signal subjected to image processing by the image processing unit 240 and the sound signal subjected to sound signal processing by the sound signal processing unit 250 are output to the output unit 260 and the storage unit 270. Note that the image processing unit 240 and the sound signal processing unit 250 may perform processing on the image signal and the sound signal stored in the storage unit 270 or the like.

  The output unit 260 includes an image display unit 261 that displays an image related to the image signal, and a sound output unit 262 that reproduces and outputs the sound related to the sound signal. The image display unit 261 displays an image related to the image signal based on the image signal output from the image processing unit 240 or the image signal stored in the storage unit 270. The sound output unit 262 outputs a sound related to the sound signal based on the sound signal output from the sound signal processing unit 250 or the sound signal stored in the storage unit 270.

  The storage unit 270 stores the image signal subjected to image processing by the image processing unit 240 and the sound signal subjected to sound signal processing by the sound signal processing unit 250. The control unit 280 controls each functional unit included in the imaging system. For example, the control unit 280 controls the imaging unit 210 or controls the image processing unit 240 and the sound signal processing unit 250 in accordance with an instruction from the operation unit 290. The operation unit 290 receives various user instruction operations for the imaging system and outputs them to the control unit 280 and the like.

  The functions of the image processing unit 240, the sound signal processing unit 250, and the control unit 280 are realized by the CPU 111 of the imaging apparatus 110 illustrated in FIG. 1 reading and executing the control program, for example. Note that the CPU 111 of the client device 120 may read out and execute the program, thereby realizing some of the functions of the image processing unit 240, the sound signal processing unit 250, and the control unit 280. For example, the function of the imaging unit 210 is realized by the imaging unit 115 of the imaging apparatus 110, and the functions of the first microphone group 220 and the second microphone group 230 are realized by the microphone group 116 of the imaging apparatus 110, for example. Further, for example, the function of the output unit 260 is realized by the output device 125 of the client device 120, and the function of the operation unit 290 is realized by the input device 126 of the client device 120, for example. The function of the storage unit 270 is realized by the storage device 117 of the imaging device 110 and the storage device 127 of the client device 120, for example.

  In the following, as illustrated in FIGS. 3A and 3B, the first microphone group 220 includes microphones 221A and 221B, and the second microphone group 230 includes microphones 231A and 231B. Explained as an example. FIG. 4 is a diagram illustrating a configuration example relating to sound signal processing (directivity processing) in the present embodiment. Sound signals obtained by the microphones 221A, 221B, 231A, and 231B are input to the sound signal processing unit 250.

  The selector 401 of the sound signal processing unit 250 receives the sound signals of the four microphones 221A, 221B, 231A, and 231B, and selects and outputs two of the sound signals based on the selection signal SEL from the CPU 403. Based on the selection signal SEL, the selector 401 outputs the sound signals of the microphones 221A and 221B included in the first microphone group 220 or the sound signals of the microphones 231A and 231B included in the second microphone group 230. The amplifiers 402A and 402B receive the sound signal output from the selector 401 and amplify the input sound signal by an amplification amount corresponding to the setting signals SGA and SGB from the CPU 403.

  A CPU (directivity processing unit) 403 receives the sound signals amplified by the amplifiers 402A and 402B, performs directivity processing on the sound signals, and outputs a sound signal SOUT after directivity processing. Here, directivity processing is signal processing that emphasizes sound from a target direction and suppresses sound from directions other than the target. Further, the CPU 403 receives a signal SIN indicating an instruction (directivity range designation) regarding a target direction from the operation unit 290, and outputs a selection signal SEL corresponding to the signal SIN.

  Next, sound signal processing (directivity processing) in the present embodiment will be described. In the following description, it is assumed that the distance between the microphone that collects the sound and the sound source is sufficiently larger than the distance between the microphones, and the direction (angle) of the sound source viewed from the microphone is the same angle.

FIG. 5A shows a state in which sound from the sound source reaches the microphone 231A and the microphone 232B belonging to the second microphone group 230 from the direction of the angle θ. The microphone 231A and the microphone 231B are arranged with a distance D2. In this case, the difference L between the distance from the sound source to the microphone 231A and the distance from the sound source to the microphone 231B is:
L = D2 × cos θ
It is represented by When the sound speed is V, the time T from when the sound from the sound source reaches the microphone 231A until the sound from the sound source reaches the microphone 231B is:
T = L / V = D2 × cos θ / V
It is represented by

  FIG. 5B shows distance difference L, time T value, and time T difference with respect to angle θ when D2 = 50 mm and V = 346.75 m / s. For example, when θ = 0 degrees, L = 50 mm and T = 144 μs, and when θ = 15 degrees, L = 48 mm and T = 139 μs. Therefore, the difference in time T between θ = 0 degrees and θ = 15 degrees is 5 μs.

  Here, in the directivity processing of the sound signal, calculation is performed based on the time T. For example, when the sound in the front direction (90 degrees) is to be emphasized (directivity is desired), the sound that reaches the microphone 231A and the microphone 231B at the same time (the sound of T = 0 μs) is emphasized, and the sound that reaches with a time difference ( (Sound of T ≠ 0 μs) is suppressed.

  Therefore, the greater the difference between the sound time T from the target direction and the sound time T from the direction other than the target direction, the more directivity is likely to be. In the example shown in FIG. 5B, the difference in time T is 37 μs at θ = 90 degrees and θ = 75 degrees. On the other hand, when θ = 0 degrees and θ = 15 degrees, the difference in time T is 5 μs. When these two are compared, in the microphones 231A and 232B belonging to the second microphone group 230 arranged in a direction substantially perpendicular to the optical axis of the imaging unit, θ = 90 degrees is more than θ = 0 degrees. It will have directivity.

  That is, when directivity is given to the sound from the front direction (θ = 90 degrees), the sound from θ = 75 degrees is suppressed well, and the sound from the front direction (θ = 90 degrees) is emphasized. hear. On the other hand, when directivity is given to the sound from θ = 0 degrees, the sound from θ = 15 degrees is the sound from θ = 75 degrees when the sound from θ = 90 degrees is given directivity. Since it is not so suppressed, sound from other than the target direction (θ = 0 degree) is leaked.

  Therefore, when directivity processing is performed using the microphone 231A and the microphone 231B arranged in a direction substantially perpendicular to the optical axis of the imaging unit, sound from the front direction of the apparatus (for example, θ = 90 degrees) is clearly displayed. It is suitable for the purpose of collecting sound. However, it can be understood that it is not suitable for the purpose of clearly collecting sounds from directions other than the front direction of the apparatus (for example, θ = 0 degree). Therefore, when the position of the subject to be collected is away from the front direction of the apparatus, there arises a problem that the target sound cannot be clearly collected.

Therefore, in the present embodiment, the above-described problems are solved by utilizing the microphones 221A and 2211B belonging to the first microphone group 220 arranged in a direction substantially parallel to the optical axis of the imaging unit. FIG. 6A shows a state in which sound from the sound source reaches the microphone 221A and the microphone 222B belonging to the first microphone group 220 from the direction of the angle θ. The microphone 221A and the microphone 221B are disposed with a distance D1 therebetween. In this case, the difference L between the distance from the sound source to the microphone 221A and the distance from the sound source to the microphone 221B is:
L = D1 × cos (90−θ) = D1 × sin θ
It is represented by When the sound speed is V, the time T from when the sound from the sound source reaches the microphone 221A until the sound from the sound source reaches the microphone 221B is:
T = L / V = D1 × sin θ / V
It is represented by

  FIG. 6B shows distance difference L, time T value, and time T difference with respect to angle θ when D1 = 50 mm and V = 346.75 m / s. Here, focusing on the difference in time T, unlike the case of using the microphone 231A and the microphone 231B (see FIG. 5B), the difference in the front direction is small and the difference in the direction other than the front direction is large. I understand that. As a matter of course, the arrangement direction of the microphones 221A and 221B belonging to the first microphone group 220 differs from the arrangement direction of the microphones 231A and 231B belonging to the second microphone group 231 by 90 degrees. It is because.

  In the sound signal processing (directivity processing) in the present embodiment, a sound signal obtained by a microphone group having a larger time T difference (suitable for directivity processing) is used. Comparing the examples shown in FIGS. 5B and 6B, the range 501 shown in FIG. 5B and the ranges 601 and 602 shown in FIG. 6B have a larger difference in time T than the other. . Therefore, in the range of the front direction (θ is 45 degrees to 135 degrees), directivity processing is performed using sound signals obtained by the microphones 231A and 232B belonging to the second microphone group 230. Further, in directions other than the front (θ is 0 ° to 45 °, 135 ° to 180 °), directivity is obtained using sound signals obtained by the microphones 221A and 221B belonging to the first microphone group 220. Perform the process. As described above, when the range of the sound collection target is on the side (wide angle side) wider than the predetermined angle of view (45 degrees to 135 degrees in this example), the sound signal obtained by the first microphone group 220 is used. To implement directivity processing. If the range of the sound collection target is on the side (telephoto side) narrower than a predetermined angle of view (45 degrees to 135 degrees in this example), the sound signal obtained by the second microphone group 230 is used for directing. Perform sex processing.

FIG. 7 is a flowchart showing an example of sound signal processing in the present embodiment.
First, when an instruction (designation of directivity range) regarding a target direction is given by the operation unit 290 in step S700, the CPU 403 of the sound signal processing unit 250 determines a microphone group to be used for directivity processing in step S701. To do. At this time, the CPU 403 determines to use the microphone group having the larger time T difference as described above. For example, when the directivity range is designated at θ = 75 degrees, the CPU 403 determines the second microphone group 230 (the microphone 231A and the microphone 231B).

  In step S702, the CPU 403 outputs a selection signal SEL to the selector 401 so as to select a sound signal obtained by the microphone group determined in step S701. For example, when the directivity range is designated at θ = 75 degrees, the selection signal SEL is output so that the sound signal from the second microphone group 230 (the microphone 231A and the microphone 231B) is selected.

  Subsequently, in step S703, the CPU 403 determines the amplification amounts of the amplifiers 402A and 402B. By this processing, the sensitivity difference for each microphone is corrected, and the sensitivity difference due to the distance difference between the subject and the microphone is corrected.

  For example, in the example shown in FIGS. 3A and 3B, the microphone 221B is disposed at a position farther from the subject by the distance D1 than the microphone 221A. Therefore, the volume from the sound source that reaches the microphone 221B is smaller than the volume from the sound source that reaches the microphone 221A. Therefore, it is necessary to set a larger amplification amount in the amplifier that amplifies the sound signal from the microphone 221B than in the amplifier that amplifies the sound signal from the microphone 221A.

  The CPU 105 determines the amplification amounts of the amplifiers 402A and 402B for the purpose of correcting the sensitivity difference due to the distance difference between the subject and the microphone in this way. An example of the amount of amplification set in the amplifiers 402A and 402B is shown in FIGS. 8A and 8B.

  When the first microphone group 220 is selected, for example, an amplification amount of 30.0 dB is set for the amplifier that amplifies the sound signal from the microphone 221A, and for the amplifier that amplifies the sound signal from the microphone 221B. Thus, an amplification amount of 30.8 dB is set. As described above, the difference between the setting values for the microphones 221A and 221B takes into account the sensitivity difference for each microphone and the sensitivity difference due to the distance difference between the subject and the microphone.

  When the second microphone group 230 is selected, for example, an amplification amount of 30.3 dB is set for the amplifier that amplifies the sound signal from the microphone 231A, and the amplifier that amplifies the sound signal from the microphone 231B. Is set to an amplification amount of 30.1 dB. The difference between the set values for the microphone 231A and the microphone 231B is only a value considering the sensitivity difference for each microphone. Since the microphone 231A and the microphone 231B of the second microphone group 230 are arranged so as to be substantially symmetric with respect to the optical axis of the imaging unit, the sensitivity difference due to the distance difference between the subject and the microphone is regarded as zero. .

  Comparing the microphones 221A, 221B, 231A, and 231B, the microphone arranged at the farthest distance from the subject is the microphone 221B. If the sensitivity difference for each microphone is ignored, it is necessary to set the largest amplification amount for the amplifier that amplifies the sound signal from the microphone 221B. In other words, if there is no sensitivity difference between microphones, the average value of the amplification amounts set for the microphones belonging to the first microphone group 220 is set for the microphones belonging to the second microphone group 230. It is necessary to make it larger than the average value of each amplification amount. Each amplification amount set for the microphones belonging to the first microphone group needs to be set higher as the distance from the subject side in the imaging apparatus increases.

  Next, in step S704, the CPU 403 outputs setting signals SGA and SGB to the amplifiers 402A and 402B based on the amplification amounts of the amplifiers 402A and 402B determined in step S703. Subsequently, in step S705, the CPU 403 performs arithmetic processing using the sound signals amplified by the amplifiers 402A and 402B so as to have directivity in the direction specified by the directivity range, and performs directivity processing. .

  As described above, according to the present embodiment, sounds in directions other than the front are clearly collected by switching the microphone used for directivity processing according to the direction in which directivity is given (angle θ). It becomes possible. Therefore, it is possible to collect the target sound even when the position of the subject (sound source) to be collected is away from the front direction of the apparatus.

The embodiments described above are examples, and the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the gist.
For example, the distance D1 between the microphone 221A and the microphone 221B and the distance D2 between the microphone 231A and the microphone 231B have been described as D1 = D2 = 50 mm in the above-described embodiment. However, the distance D1 and the distance D2 do not have to be equal. And may be different. As an example, FIG. 9 shows distance difference L, time T value, and time T difference with respect to angle θ when distance D1 = 100 mm and distance D2 = 50 mm. Comparing FIG. 6B and FIG. 9B showing the microphones 221A and 221B belonging to the first microphone group 220, the difference in the time T is a distance D1 = 100 mm than when the distance D1 = 50 mm. You can see that it is getting bigger. That is, it is clear that a larger distance between microphones is suitable for directivity processing.

  In a box-shaped imaging device (for example, a network camera) as shown in FIGS. 3A and 3B, the distance between the microphone 221A and the microphone 221B is larger than the distance between the microphone 231A and the microphone 231B. Can be larger. Therefore, it is more preferable for the directivity processing to have a relationship of D1> D2 than to set D1 = D2.

  For example, in the example shown in FIGS. 9A and 9B, the range 901 shown in FIG. 9A and the ranges 902 and 903 shown in FIG. large. Therefore, in the range of the front direction (θ is 60 degrees to 120 degrees), directivity processing is performed using sound signals obtained by the microphones 231 and 231B belonging to the second microphone group 230. Further, in directions other than the front (θ is 0 to 60 degrees, 120 to 180 degrees), directivity is obtained using sound signals obtained by the microphones 221A and 221B belonging to the first microphone group 220. Perform the process. From the above, it will be understood that the inter-microphone distance of the first microphone group and the inter-microphone distance of the second microphone group may be preferable as an embodiment even if they are not equidistant.

  In the above-described embodiment, the microphones belonging to the first microphone group 220 and the microphones belonging to the second microphone group 230 are all different, but at least one microphone is the first microphone group 220 and the first microphone group 220. You may make it belong to 2 microphone groups 230. For example, as shown in FIG. 3C, microphones belonging to the first microphone group 220 and microphones belonging to the second microphone group 230 are arranged, and one microphone (221A, 231A) is connected to the first microphone group 220. The second microphone group 230 may be shared. Even if the arrangement shown in FIG. 3C is used, the same effect as the arrangement shown in FIG. 3A can be obtained by appropriately selecting the sound signal used for the sound signal processing (directivity processing). It is done.

  As shown in FIG. 3D, the microphones belonging to the second microphone group 230 are substantially perpendicular to the optical axis of the imaging unit and substantially perpendicular to the direction connecting the microphones 231A and 231B. The microphone 231C may be further arranged in any direction. In this case, it is possible to provide directivity to an arbitrary position on a plane having the normal axis as the optical axis of the imaging unit.

  Note that the sound signal processing (directivity processing) described above may be performed in the imaging device 110 or may be performed in the client device 120. For example, directivity processing may be performed in the imaging device 110 and a sound signal after directivity processing may be output from the imaging device 110. In addition, for example, sound signals obtained from the first microphone group 220 and the second microphone group 220 are output from the imaging device 110, and a sound signal used in the client device 120 is selected to perform directivity processing. Also good. Further, for example, only the sound signals obtained from the microphone groups used in the first microphone group 220 and the second microphone group 220 are output from the imaging device 110 and the directivity processing is performed in the client device 120. Also good.

(Other embodiments of the present invention)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

110: Imaging device 111, 121: CPU 112, 122: ROM 113, 123: RAM 114, 124: Communication interface 115: Imaging unit 116: Microphone group 117, 127: Storage device 120: Client device 125: Output device 126: Input Device 130: Network 210: Imaging unit 211: Optical system 212: Imaging device 220, 230: Microphone group 221, 231: Microphone 240: Image processing unit 250: Sound signal processing unit 260: Output unit 261: Image display unit 262: Sound Output unit 270: storage unit 280: control unit 290: operation unit

Claims (11)

An imaging unit having an imaging element;
A first microphone group including a plurality of microphones arranged apart in a direction substantially parallel to the optical axis of the imaging unit;
An image pickup apparatus comprising: a second microphone group including a plurality of microphones arranged apart from each other in a direction substantially perpendicular to the optical axis of the image pickup unit. A sound signal processing unit that performs directivity processing using the input sound signal and outputs the sound signal subjected to the directivity processing,
The sound signal processing unit
When the sound collection target range is wider than a predetermined angle of view, the directivity processing is performed using a sound signal obtained from the first microphone group,
2. The directivity processing according to claim 1, wherein the directivity processing is performed using a sound signal obtained by the second microphone group when the range of the sound collection target is a side narrower than a predetermined angle of view. Imaging device.   The imaging apparatus according to claim 1, wherein at least one microphone belonging to the first microphone group belongs to the second microphone group.   The imaging apparatus according to claim 1, wherein a distance between microphones in the first microphone group is greater than a distance between microphones in the second microphone group.   4. The imaging apparatus according to claim 1, wherein a distance between microphones in the first microphone group is the same as a distance between microphones in the second microphone group. 5.   The imaging apparatus according to claim 1, wherein microphones belonging to the second microphone group are arranged substantially symmetrically with respect to an optical axis of the imaging unit. The sound signal processing unit has a plurality of amplifiers that amplify the input sound signal,
When the directivity processing is performed using the sound signal obtained from the first microphone group, the average value of the amplification amounts set in the plurality of amplifiers is the sound signal obtained from the second microphone group. The imaging apparatus according to claim 2, wherein the imaging apparatus is larger than an average value of amplification amounts set in the plurality of amplifiers when the directivity processing is used.   When the directivity processing is performed using sound signals obtained from the first microphone group, the amplification amounts set in the plurality of amplifiers are such that the positions of the microphones that obtain the sound signals are far from the subject side in the imaging apparatus. The imaging apparatus according to claim 7, wherein the imaging apparatus is set higher. An imaging unit having an imaging element; a first microphone group including a plurality of microphones arranged in a direction substantially parallel to an optical axis of the imaging unit; and a direction substantially perpendicular to the optical axis of the imaging unit. An imaging device having a second microphone group including a plurality of microphones arranged
A sound signal processing unit that performs directivity processing using the sound signal output from the imaging device and outputs the sound signal subjected to the directivity processing,
The sound signal processing unit
When the sound collection target range is wider than a predetermined angle of view, the directivity processing is performed using a sound signal obtained from the first microphone group,
When the range of the sound collection target is on a side narrower than a predetermined angle of view, the directivity process is performed using a sound signal obtained by the second microphone group. An imaging unit having an imaging element; a first microphone group including a plurality of microphones arranged in a direction substantially parallel to an optical axis of the imaging unit; and a direction substantially perpendicular to the optical axis of the imaging unit. A signal processing method of a sound signal obtained by an imaging device having a second microphone group including a plurality of microphones arranged
A determination step of determining a microphone group to be used from the first microphone group and the second microphone group according to a range of sound collection targets;
A sound signal processing step of performing directivity processing using a sound signal obtained by the microphone group determined in the determination step, and outputting the sound signal subjected to the directivity processing;
In the sound signal processing step,
When the range of the sound collection target is on the side wider than the predetermined angle of view, the microphone group to be used is determined as the first microphone group,
A signal processing method comprising: determining a microphone group to be used as the second microphone group when the range of the sound collection target is a side narrower than a predetermined angle of view. An imaging unit having an imaging element; a first microphone group including a plurality of microphones arranged in a direction substantially parallel to an optical axis of the imaging unit; and a direction substantially perpendicular to the optical axis of the imaging unit. A program for causing a computer to perform signal processing of a sound signal obtained by an imaging device having a second microphone group including a plurality of microphones arranged in a row,
A determination step of determining a microphone group to be used from among the first microphone group and the second microphone group according to a range of a sound collection target;
A sound signal processing step of performing directivity processing using a sound signal obtained by the microphone group determined in the determination step, and outputting the sound signal subjected to the directivity processing;
In the sound signal processing step,
When the range of the sound collection target is on the side wider than the predetermined angle of view, the microphone group to be used is determined as the first microphone group,
A program that determines a microphone group to be used as the second microphone group when the range of the sound collection target is narrower than a predetermined angle of view.

Images