WO2014125835A1 - 指向性制御システム、キャリブレーション方法、水平偏差角算出方法及び指向性制御方法 - Google Patents
指向性制御システム、キャリブレーション方法、水平偏差角算出方法及び指向性制御方法 Download PDFInfo
- Publication number
- WO2014125835A1 WO2014125835A1 PCT/JP2014/000775 JP2014000775W WO2014125835A1 WO 2014125835 A1 WO2014125835 A1 WO 2014125835A1 JP 2014000775 W JP2014000775 W JP 2014000775W WO 2014125835 A1 WO2014125835 A1 WO 2014125835A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- sound collection
- unit
- horizontal
- angle
- sound
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S3/00—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
- G01S3/78—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using electromagnetic waves other than radio waves
- G01S3/782—Systems for determining direction or deviation from predetermined direction
- G01S3/785—Systems for determining direction or deviation from predetermined direction using adjustment of orientation of directivity characteristics of a detector or detector system to give a desired condition of signal derived from that detector or detector system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/62—Control of parameters via user interfaces
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S3/00—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
- G01S3/78—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using electromagnetic waves other than radio waves
- G01S3/782—Systems for determining direction or deviation from predetermined direction
- G01S3/785—Systems for determining direction or deviation from predetermined direction using adjustment of orientation of directivity characteristics of a detector or detector system to give a desired condition of signal derived from that detector or detector system
- G01S3/786—Systems for determining direction or deviation from predetermined direction using adjustment of orientation of directivity characteristics of a detector or detector system to give a desired condition of signal derived from that detector or detector system the desired condition being maintained automatically
- G01S3/7864—T.V. type tracking systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
- H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
- H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/02—Casings; Cabinets ; Supports therefor; Mountings therein
- H04R1/04—Structural association of microphone with electric circuitry therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
- H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
- H04R1/406—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
- H04R2201/40—Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups
- H04R2201/401—2D or 3D arrays of transducers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
- H04R2430/20—Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic
- H04R2430/23—Direction finding using a sum-delay beam-former
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Definitions
- the present invention relates to a directivity control system that controls sound collection directivity of audio data, a calibration method, a horizontal deviation angle calculation method, and a directivity control method.
- the present invention is a directivity control method in a directivity control system including at least one image pickup unit for picking up an image and at least one sound pickup unit for picking up an audio, wherein the at least one image pickup unit Displaying the imaged video data on a screen, receiving an designation of an arbitrary position of the video data displayed on the screen, and corresponding to the position of the video data specified by the sound collection unit Calculating the sound collection direction toward the audio position, wherein the sound collection direction from the sound collection unit toward the audio position corresponding to the designated position of the video data is a predetermined sound collection direction calculation parameter;
- the directivity control method may be calculated using a direction from the at least one imaging unit toward an audio position corresponding to a designated position of the video data.
- the present invention is a horizontal deviation angle calculation method in a directivity control system including a first imaging unit, a second imaging unit, and a sound collecting unit, wherein the first imaging unit captures an image of a subject.
- the second imaging unit, the step of capturing the image of the subject, the step of collecting the voice of the subject in the sound collection unit, and the image data captured by the first imaging unit A step of displaying on a display unit, and a first imaging toward a sound position corresponding to a designated position of the video data from the first imaging unit in accordance with designation of an arbitrary position of the image data displayed on the display unit; A first horizontal deviation angle from the first reference direction of the horizontal angle of the direction and a second horizontal deviation angle from the second reference direction of the horizontal angle of the second imaging direction from the second imaging unit toward the voice position Step to calculate A horizontal deviation angle calculation method with.
- voice pickup directivity is formed toward a place or direction corresponding to a specified position from the video of the predetermined range captured by the camera device, and sound in the corresponding direction Data can be collected with high accuracy.
- FIG. 6A shows an outline of operation of the directivity control system of the present embodiment, (A) a state in which one camera device captures an image of an object appearing within a range of angle of view, and a person of the object whose microphone array device is present in the sound collecting direction. Of a user's finger designated from among the images displayed on the display device from the microphone array device when the user's speech is collected and the music output from the speaker device not present in the sound collection direction is collected.
- FIG. 36 (B) (A) Camera device, omnidirectional microphone array device, perspective view showing each position of reference point O and target sound source position A, (B) Horizontal plane viewed vertically from above in FIG. 37 (A) Fig. 37 (C) is a vertical sectional view taken along the line QQ 'of Fig.
- FIG. 87 A); (C) LL 'sectional view of FIG. 87 (B)
- A A plan view of a calibration floor marker MAK4 used in the thirteenth calibration method
- B An enlarged screen of points O, O 'and X by the focus function of the PTZ camera device 1
- A A diagram showing the positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2 and the calibration floor marker MAK 4 in the thirteenth calibration method
- FIG. 89 A).
- C KK 'cross section figure of FIG.
- the directional control system of this embodiment is, for example, a monitoring system (including a manned surveillance system and an unmanned surveillance system) installed in a factory, a public facility (eg, a library, an event site), or a store (eg, a retail store, a bank). Used.
- a monitoring system including a manned surveillance system and an unmanned surveillance system
- a public facility eg, a library, an event site
- a store eg, a retail store, a bank
- the present invention is expressed as a directivity control method having each operation (step) performed by each device (for example, a directivity control device described later) configuring the directivity control system or each device configuring the directivity control system. It is also possible.
- each of the sound collection directions ( ⁇ MAh , ⁇ MAv ) derived (hereinafter referred to as “calculation”) derived by the signal processing unit 33 in response to a directivity formation instruction from the signal processing unit 33 described later.
- the sound pickup directivity of the microphones 22 and 23 is formed.
- the directivity control device 3 is connected to the network NW1 or the network NW2, and may be, for example, a stationary PC (Personal Computer) installed in a monitoring system control room (not shown), or a portable telephone which can be carried by the user, a tablet It may be a terminal or a smartphone.
- a stationary PC Personal Computer
- a portable telephone which can be carried by the user
- a tablet It may be a terminal or a smartphone.
- the display device 36 as a display unit displays video data captured by the camera devices 11 to 1 n on the screen.
- the origin O C is the center of rotation of the respective pan direction and tilt direction of the PTZ camera apparatus 1
- the base parallel to the x-y plane of the PTZ camera device 1 is defined, this
- the xy plane is parallel to the wall surface.
- the x-axis is directed vertically upward.
- a reference point O is provided in the direction of the optical axis CX of the camera device 11.
- the input parameters in step ST2 shown in FIG. (1) Distance L CMh of horizontal component (direction) of distance L CM between camera device 11 and microphone array device 2; (2) The distance L CO from the camera device 11 to the reference point O and the depression angle ⁇ CO , (3) The distance L MO from the microphone array device 2 to the reference point O and the depression angle ⁇ MO (4) Camera device 11, microphone array device 2, heights H C , H M , H O from the horizontal plane of reference point O.
- FIG. 9 is a view showing the positional relationship between the camera device 11, the microphone array device 2, the reference point O, and the sound collection range central position A of the directivity control system 10 according to the first calculation method.
- FIG. 9A is a perspective view.
- FIG. 9B is a horizontal direction top view.
- FIG. 9C is a vertical direction sectional view taken along the line Q-Q 'of FIG. 9B.
- the signal processing unit 33 uses the calculation results of Equations (8) and (9) to follow the equation (11) according to Equation (11) according to the cosine theorem for .DELTA.COM shown in FIG. to the calculated cosine value cos [theta] MOH horizontal angle theta MOH depression angle theta MO.
- the signal processing unit 33 uses the calculation results of Equations (12) to (14) to follow the equation (15) according to Equation (15) according to the cosine theorem for .DELTA.COM shown in FIG. to the calculated cosine value cos [theta] CoV vertical angle theta CoV depression angle theta CO.
- the signal processing unit 33 can calculate the vertical angle ⁇ MAv of the depression angle ⁇ MA from the microphone array device 2 to the sound collection range central position A according to Expression (23).
- the directivity control device 3 is specified by the finger FG of the user in the image of the predetermined range to be captured as the monitoring target based on the position of the microphone array device 2 using these respective input parameters.
- the coordinate ( ⁇ MAh , ⁇ MAv ) indicating the direction toward the sound collection range center position A corresponding to the position A ′, that is, the direction from the microphone array device 2 to the sound collection range center position A is calculated as the sound collection direction .
- the length of the yarn STR is less than the height H M of the microphone array device 2 from the horizontal surface.
- the marker MAK is, for example, a ball of a color that is easy for the user to visually recognize when the camera device 11 captures an image.
- the directivity control device 3 provides, as the reference point O, the marker MAK suspended by the thread in the vertically downward direction of the microphone array device 2, and the distance between the camera device 11 and the marker MAK. and L CO and depression angle theta vertical angle of CO theta CoV, and the angle theta CMh between the direction of the optical axis CX of the camera apparatus 11 the direction from the camera device 11 to the microphone array system 2, the marker MAK from the microphone array device 2
- the distance LMO up to is used as each input parameter.
- FIG. 16 is a diagram showing the positional relationship between the camera device 11, the microphone array device 2, the sound source position (reference point O), and the sound collection range center position A of the directivity control system 10 of the third calculation method.
- FIG. 16A is a perspective view.
- FIG. 16B is a top view in the horizontal direction.
- FIG. 16C is a vertical direction sectional view taken along line R-R 'of FIG. 16B.
- the signal processing unit 33 uses the calculation results of Equations (36) and (38) and the heights H C and H O from the horizontal plane of the camera device 11 and the sound source position (reference point O).
- the distance L COv of the vertical component of the distance L CO from the camera device 11 to the sound source (reference point O) is calculated according to Equation (41).
- the signal processing unit 33 can calculate the vertical angle ⁇ MAv of the depression angle ⁇ MA from the 0 ° direction reference line of the microphone array device 2 to the sound collection range central position A according to Expression (53).
- the directivity control device 3 provides the sound source as the reference point O in the direction of the optical axis CX of the camera device 11, and the distance L between the camera device 11 and the sound source (reference point O) and CO, and the distance L CM of the horizontal angle theta MOH and vertical angle theta MOV depression angle theta MO towards the sound source (reference point O) from the microphone array system 2, a camera device 11 and the microphone array device 2, a camera device 11,
- the microphone array device 2 uses the heights H C , H M , and H O of the reference point O from the horizontal plane as the input parameters.
- the directivity control device 3 is specified by the finger FG of the user in the image of the predetermined range to be captured as the monitoring target based on the position of the microphone array device 2 using these respective input parameters.
- the coordinate ( ⁇ MAh , ⁇ MAv ) indicating the direction toward the sound collection range center position A corresponding to the position A ′, that is, the direction from the microphone array device 2 to the sound collection range center position A is calculated as the sound collection direction .
- the directivity control system 10 of the present embodiment is a case where the 0 degree direction reference line of the microphone array device 2 is not set in advance in the direction facing the camera device 11 Also, with reference to the position of the microphone array device 2, the sound collection directivity of the sound can be formed with high accuracy in the direction of the specified sound collection range center position A, and the sound data of the corresponding direction can be collected with high accuracy. Can.
- the vertical angle and the horizontal angle indicating the coordinates are approximately calculated. Although there is a location, it may be calculated more strictly using, for example, a geometrical positional relationship.
- the distance L CM between the camera device 11 and the microphone array device 2 is the dedicated jig It becomes a fixed value according to the parameter of.
- FIG. 18 shows the microphone array device 2 and the camera device 11 and the sound collection range central position A when the microphone array device 2 and the camera device 11 of the fourth calculation method are installed by connecting them using the dedicated jig 50. It is explanatory drawing which shows the positional relationship of.
- FIG. 18A is a perspective view.
- FIG. 18B is a horizontal direction top view.
- FIG. 18C is a vertical direction sectional view taken along the line Q-Q 'of FIG. 18B.
- the signal processing unit 33 by using the distance L CA and vertical angle theta CAv from the camera apparatus 11 to the sound pickup range center position A, in accordance with equation (54), the distance from the camera device 11 to the sound pickup range center position A L
- the distance L CAh of the horizontal component of CA is calculated.
- the directivity control system 10 of the present embodiment has a smaller number of input parameters compared to the first to third calculation methods, and the camera device 11 and the microphone array Since the 0-degree direction reference lines with the device 2 face each other, the coordinates ( ⁇ MAh , ⁇ MAv ) in the direction from the microphone array device 2 toward the sound collection range central position A can be calculated more easily. Furthermore, the directivity control system 10 can form the sound collection directivity of the voice in the direction of the designated sound collection range center position A with high accuracy with reference to the position of the microphone array device 2, and the voice data of the corresponding direction It can be picked up with high accuracy.
- the imaging direction of the camera apparatus is indicated.
- An example of a calibration method will be described in which each reference direction of the horizontal angle of the coordinate and the horizontal angle of the coordinate indicating the sound collection direction of the microphone array device is made to coincide.
- the calibration method according to each of the embodiments below is, for example, in a sound collection system in which an omnidirectional camera device and an omnidirectional microphone array device are integrally arranged coaxially, the horizontal direction of coordinates indicating the imaging direction of the omnidirectional camera device This is a method of matching the angle with the horizontal angle of the coordinate indicating the sound collection direction of the omnidirectional microphone array device.
- the sound collection system of each embodiment is installed, for example, on a predetermined installation surface (for example, a ceiling surface of an event hall) in a predetermined sound collection space.
- the omnidirectional microphone array 3AZ is not fitted in the inner peripheral space inside the opening 13 of the omnidirectional microphone array device 5A.
- the male screw portion inscribed in the outer periphery of the cover 3a of the omnidirectional camera device 3AZ is screwed into the female screw portion previously cut in the opening 13 of the device 5A, whereby the omnidirectional camera device 3AZ and the omnidirectional microphone array
- the device 5A may be integrated and united.
- an engagement hole 71 which engages with a fixing pin 33z provided on the bottom surface of the omnidirectional camera device 3BZ is formed in a substantially gourd shape whose diameter at one end is larger than that at the other end. It is done.
- the engagement hole 73 engaged with the fixing pin 35z provided on the bottom surface of the omnidirectional microphone array device 5B has a diameter of one end substantially larger than that of the other end. It is shaped like a gourd.
- the omnidirectional microphone array device 5C indicates, for example, the horizontal direction indicating the imaging direction of the omnidirectional camera device 3CZ using the horizontal angle shift amount ⁇ calculated by the omnidirectional camera device 3CZ.
- the coordinates (horizontal angle and vertical angle) of the sound collection direction in the omnidirectional microphone array device 5C can be adjusted by matching the angle with the horizontal angle indicating the sound collection direction of the omnidirectional microphone array device 5C.
- a locking member formed on an outer periphery of a casing of the camera device is fitted to a locking groove formed in a peripheral portion of the opening.
- the calibration method may be such that the reference direction of the horizontal angle of the microphone array device matches the reference direction of the horizontal angle of the camera device.
- the omnidirectional camera device 3BZ and the omnidirectional microphone array device 5B are integrally attached to the mounting member 7 attached and fixed to the ceiling surface 8, so the omnidirectional camera device 3BZ and the omnidirectional microphone array device 5B And the reference direction of each horizontal angle can be made to correspond.
- the omnidirectional camera device 11z has a function as a surveillance camera, for example, and has a housing in which an optical system (for example, a fisheye lens or a wide-angle lens) and an imaging system (for example, an image sensor) (not shown) are incorporated. It is installed on the surface (for example, the ceiling surface or the stand in the room of the event hall).
- the omnidirectional camera device 11z is connected to a host computer (not shown) in the central control room via a network (not shown), and operates pan direction, tilt direction, zoom operation according to remote control from the host computer.
- the image pickup operation, the distance measurement operation to the actual position corresponding to the position designated in the pickup image (for example, designated position A 'described later) and the angle measurement operation are performed.
- the omnidirectional camera device 11z picks up an object present in the first imaging direction CAX1 directed from, for example, the omnidirectional camera device 11z to an audio collection position (or audio position) A described later (see FIG. 27A).
- the omnidirectional camera device 11z is connected to the network NW, and input parameters (for example, the omnidirectional camera device 11z and calibration) for calculating the sound collecting direction ( ⁇ MAh , ⁇ MAv ) of the omnidirectional microphone array device 2 described later.
- the distance L CK between the omnidirectional camera device C1 is measured and acquired, and the measured input parameter and the imaged image data are transmitted to the directivity control device 3 or the recorder device 4 via the network NW. Send.
- the omnidirectional microphone array device 2 uses each of the microphone units 22 and 23 to pick up the voice in the sound collection direction in which the subject to be picked up is present, and the microphone units 22 and 23 pick up Audio data is transmitted to the directivity control device 3 or the recorder device 4 via the network NW.
- the appearance of the omnidirectional microphone array device 2 has been described above with reference to FIG.
- the microphone units 22 and 23 of the omnidirectional microphone array device 2 may be nondirectional microphones, or may be bidirectional microphones, unidirectional microphones, sharp directional microphones, superdirective microphones (for example, gun microphones) or A combination of these may be used.
- the communication unit 31 receives the image data or audio data transmitted by the omnidirectional camera device 11z, the omnidirectional camera device for calibration C1 or the omnidirectional microphone array device 2 via the network NW, and outputs the image data or the audio data to the signal processing unit 33 Do.
- the coordinate calculation unit 34 x proceeds from the omnidirectional microphone array device 2 to the sound collection position A based on the first horizontal deviation angle ⁇ Ch and the second horizontal deviation angle ⁇ Kh calculated by the horizontal deviation angle calculation unit 34 w.
- the horizontal angle ⁇ MAh and the vertical angle ⁇ MAv of the sound direction MIX are calculated as coordinates (sound collecting direction coordinates) indicating the sound collecting direction for collecting the voice of the subject on the omnidirectional microphone array device 2.
- the relationship between the coordinate axis of the omnidirectional camera device 11z and the coordinate axis of the omnidirectional microphone array device 2 is known from the first horizontal deviation angle ⁇ Ch , the second horizontal deviation angle ⁇ Kh and the mutual distance L ck
- the control device 3 calculates the sound collection direction ( ⁇ MAh , ⁇ MAv ) from the image data displayed on the display device 36 in accordance with the designation of the sound collection position.
- the sound collection position A is the position of the actual monitoring target site corresponding to the designated position A ′ designated by the user's finger FG or the stylus pen on the screen of the display unit 36 of the operation unit 32 (see FIG. 27 (A) and FIG. 29).
- the output control unit 34 c controls the operation of the display device 36 and the speaker device 37, causes the display device 36 to reproduce and output the video data transmitted from the omnidirectional camera device 11 z, and the audio transmitted from the omnidirectional microphone array device 2 The data is output to the speaker device 37 as voice. Further, the output control unit 34 c controls the operation of the omnidirectional microphone array device 2 and, for example, the sound in the sound collection direction MIX corresponding to the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) calculated by the coordinate calculation unit 34 x The sound collection directivity of data is formed in the omnidirectional microphone array device 2.
- the display device 36 as a display unit displays the image data captured by the omnidirectional camera device 11z or the omnidirectional camera device for calibration C1 on the screen.
- the memory 38 is configured using, for example, a RAM (Random Access Memory), and functions as a work memory when the respective units of the directivity control device 3 operate.
- RAM Random Access Memory
- FIG. 29 the direction from the omnidirectional microphone array device 2 to the sound collection position A corresponding to the designated position A ′ designated by the user's finger from among the images displayed on the display device 36 is collected as the sound collection direction It is a figure which shows a mode that the audio
- FIG. 29 shows a mode that the audio
- the omnidirectional camera device 11z captures an object (for example, two persons) shown in FIG.
- the omnidirectional microphone array device 2 picks up surrounding voice including conversational voice of the subject.
- FIG. 27A two persons of the subject are in conversation.
- image data captured by the omnidirectional camera device 11z is displayed on the display device 36 of the directivity control device 3 (see FIG. 29).
- the omnidirectional microphone array device 2 does not have the volume level of the conversation (Hello) of two persons present in the direction in which the sound collection directivity is formed, and does not exist in the direction in which the sound collection directivity is formed. It can be higher than the volume level emitted from the object.
- a fixture (not shown, for example, a metal fitting, a ceramic fitting, or a synthetic resin (for example, plastic or elastomer) fitting) is attached to a predetermined installation surface (for example, a stand). And fixed.
- both of the calibration omnidirectional camera device C1 and the omnidirectional microphone array device 2 are attached to the fixture.
- the calibration omnidirectional camera device C1 and the omnidirectional microphone array device 2 attached to the fixture are formed at the center of the casing of the omnidirectional microphone array device 2 with the calibration omnidirectional camera device C1. It is integrally formed by being fitted into the inner circumferential space of the opening 21a. Further, the calibration omnidirectional camera device C1 and the omnidirectional microphone array device 2 are integrally formed so that the front direction (0 ° direction) of each horizontal angle is common.
- the horizontal deviation angle calculation unit 34w After the omnidirectional camera device 11z, the calibration omnidirectional camera device C1, and the omnidirectional microphone array device 2 are initially installed, the horizontal deviation angle calculation unit 34w generates the first horizontal deviation angle ⁇ Ch and the second horizontal deviation angle ⁇ Kh
- the measurement of the input parameter for example, the distance L CK ) required to calculate H.sub.2 is performed (ST12).
- the distance L CK represents the distance between the omnidirectional camera device 11 z and the omnidirectional camera device for calibration C 1.
- the directivity control device 3 receives specification of an arbitrary designated position A 'from among the image data displayed on the screen of the display device 36 via the operation unit 32 (ST13).
- the directivity control device 3 transmits that the designation of the designated position A 'of the image data displayed on the screen of the display device 36 has been received to the omnidirectional camera device 11z and the omnidirectional camera device C1 for calibration.
- step ST13 starts from the installation position of the calibration omnidirectional camera device C1.
- horizontal angle and vertical angle obtained in step ST14 ( ⁇ KAh, ⁇ KAv) of the coordinate data of the horizontal angle theta Kah, state or second front direction is any direction has not yet found Data of
- FIG. 32 is a view showing the positional relationship between the omnidirectional camera device 11z, the omnidirectional camera device for calibration C1, and the sound collection position A in this embodiment.
- FIG. 32 (A) is a perspective view.
- FIG. 32 (B) is a plan view when looking vertically downward from the upper side in FIG. 32 (A).
- FIG. 32C is a vertical direction sectional view taken along the line K-K 'in FIG.
- H K, H M is the omnidirectional camera device 11z
- calibration It is a fixed value determined at each initial installation of the omnidirectional camera device C1 and the omnidirectional microphone array device 2.
- the heights H C , H K , and H M from the horizontal plane of the omnidirectional camera device 11z, the omnidirectional camera device for calibration C1, and the omnidirectional microphone array device 2 are the same.
- the horizontal deviation angle calculation unit 34w in the illustrated ⁇ CAS FIG 31 (C), the height H C of the horizontal plane of the omnidirectional camera device 11z, the height H A of the horizontal plane of the audio sound pickup position A, the 1 based on the vertical angle theta CAv imaging direction CAX1, the distance L CAh of the horizontal component of the distance from the omnidirectional camera device 11z to speech sound pickup position a is calculated according to equation (64).
- the horizontal deviation angle calculation unit 34w calculates the height H K from the horizontal plane of the omnidirectional camera device for calibration C1 and the height H A from the horizontal plane of the sound collection position A in ⁇ KAS shown in FIG. If, based on the vertical angle theta kav second imaging direction CAX2, calculates the distance L Kah horizontal component of the distance from the calibration omnidirectional camera system C1 to the speech sound pickup position a according to equation (65).
- the horizontal deviation angle calculation unit 34 w uses the distance L CK and the distances L CAh and L KAh calculated by the equation (64) and the equation (65) to calculate the cosine theorem in ⁇ KCA shown in FIG.
- a first imaging unit for capturing an image of a subject
- a second imaging unit for capturing an image of the subject
- a sound collection unit for capturing a voice of the subject
- the first imaging A display unit for displaying image data captured by the unit, and an audio pickup position corresponding to a designated position of the image data from the first imaging unit according to designation of an arbitrary position of the displayed image data;
- a first horizontal deviation angle from a first reference horizontal angle of a horizontal angle of a first imaging direction to head, and a second reference direction of a horizontal angle of a second imaging direction from the second imaging unit to the sound collecting position
- a deviation control unit configured to calculate a second horizontal deviation angle.
- the directivity control device 3 surrounds the casing periphery of the calibration omnidirectional camera device C1 from the imaging direction coordinate from the omnidirectional camera device 11z toward the sound collection position A.
- Omnidirectional microphone array device 2 mounted at the position of the omnidirectional camera device C1 for calibration to coincide with the reference direction of the horizontal angle of the omnidirectional camera device C1 for calibration or the omnidirectional camera device C1 for calibration It is possible to calculate each deviation amount formed by the front direction (for example, 0 ° direction) of each horizontal angle of the second sound collection direction coordinate and the reference direction of each other. Therefore, in the directivity control system 10, the directivity control device 3 corrects the sound collecting direction coordinate from the omnidirectional microphone array device 2 to an arbitrary sound collecting position of the image data captured by the omnidirectional camera device 11z. Can be calculated.
- a horizontal angle and a vertical angle in a direction from the sound collecting unit toward the sound collection position are determined based on the first horizontal deviation angle and the second horizontal deviation angle.
- the coordinate control unit further includes: a coordinate calculation unit configured to calculate coordinates of a sound collection direction in which the sound of the sound collection unit is collected by the sound collection unit.
- a cylindrical opening is formed at the center of the housing of the sound collection unit, and the second image pickup unit is the opening of the sound collection unit and the second imaging unit.
- the directional control system is integrally formed by being fitted into the inner circumferential space of
- one embodiment of the present invention is a method of calculating a horizontal deviation angle in a directivity control system including a first imaging unit, a second imaging unit, and a sound collecting unit, wherein the first imaging unit is a subject.
- Capturing an image of the subject capturing an image of the subject in the second imaging unit, capturing an audio of the subject in the sound collecting unit, and capturing by the first imaging unit
- the second horizontal deviation angle A calculating and a horizontal deviation angle calculation method.
- Patent Document 1 for example, a usage mode in a video conference system is assumed, and it is premised that a camera device, a microphone array device, and an object (for example, a person) exist on the same plane.
- the camera device, the microphone array device, and the object for example, a person
- the object for example, a person
- the directional control system of each embodiment is used as, for example, a monitoring system (including a manned surveillance system and an unmanned surveillance system) installed in a factory, a company, a public facility (for example, an event hall), or a store (for example, a retail store).
- a monitoring system including a manned surveillance system and an unmanned surveillance system
- a public facility for example, an event hall
- a store for example, a retail store.
- the installation location is not particularly limited.
- the directivity control system of each embodiment is described as being installed in, for example, a store.
- the camera devices 11 to 1 n as at least one imaging unit have a housing in which an optical system (for example, a wide angle lens) (not shown) and an imaging system (for example, an image sensor) are embedded. Fixed to a stand (see FIG. 34A) and has a function as a surveillance camera.
- the camera devices 11 to 1 n are connected to the directivity control device 3 of the central control room (not shown) connected via the network NW, and pan direction operation, tilt according to the remote control from the directivity control device 3 Direction operation, zoom operation, image pickup operation, distance measurement operation and angle measurement operation up to the actual target sound source position A corresponding to the position designated in the captured image (for example, designated position A ′ shown in FIG. 34 (B) described later) Etc.
- the omnidirectional microphone array device 2 as a sound collection unit has, for example, a doughnut-shaped or ring-shaped (annular) housing 21C (see FIG. 2D) in which an opening 21a is formed at the center of the housing. For example, it is fixedly installed on a ceiling surface in a store or a predetermined stand (see FIG. 34A).
- the omnidirectional microphone array device 2 forms sound collection directivity for collecting voice with high accuracy in the sound collection directivity direction from the installation position M of the omnidirectional microphone array device 2 to the target sound source position A described later.
- the speech sound for example, “Hello” of the object (two persons) present in the sound collection orientation direction is collected with high accuracy.
- the casing shape of the omnidirectional microphone array device 2 is not limited to the donut shape or the ring shape (annular shape) shape, and the description has been omitted because it has been described with reference to FIG.
- the omnidirectional microphone array device 2 is connected to the network NW, and includes microphone units 22 and 23 (see FIGS. 2A to 2E) including microphones provided at equal intervals, and each microphone unit 22, And at least a control unit (not shown) for controlling each of the operations of.
- the omnidirectional microphone array device 2 uses each of the microphone units 22 and 23 to pick up voice in a sound collection directional direction in which an object (sound source) to be a voice collection target is present,
- the voice data collected by the voice processing unit 23 is subjected to predetermined voice processing and transmitted to the directivity control device 3 or the recorder device 4 via the network NW.
- omnidirectional microphone array device 2 can relatively increase the volume level of the sound collected from the sound collection directional coordinates ( ⁇ MAh , ⁇ MAv ) in which the sound collection directivity is formed, and the sound collection directivity is improved. It is possible to relatively reduce the volume level of the sound collected from the direction not formed.
- the method of calculating the sound collection directional coordinates ( ⁇ MAh , ⁇ MAv ) will be described later.
- the directivity control device 3 is configured to include at least a communication unit 31, an operation unit 32, a signal processing unit 33, a display device 36, a speaker device 37, and a memory 38.
- the signal processing unit 33 at least includes a sound source height determination unit 34 a, a sound collection directivity direction calculation unit 34 b, and an output control unit 34 c.
- the operation unit 32 is a user interface (UI: User Interface) for notifying the signal processing unit 33 of the content of the user's input operation, and is, for example, a pointing device such as a mouse or a keyboard.
- UI User Interface
- the operation unit 32 may be disposed, for example, in correspondence with the screen of the display device 36, and may be configured using a touch panel or a touch pad that allows an input operation with the user's finger FG or a stylus pen.
- the sound source height determination unit 34a as the height determination unit specifies a designated position A 'of the video data captured by the camera device 11 using the finger FG or stylus pen of the user as the video data displayed on the display device 36, for example Then, the height HA from the floor surface BL of the target sound source position A corresponding to the designated designated position A 'is determined.
- the reference plane is the floor plane BL in the store unless otherwise specified.
- ⁇ MAh represents the horizontal angle in the sound collection directional direction from the omnidirectional microphone array device 2 to the target sound source position A
- ⁇ MAv from the omnidirectional microphone array device 2 This represents the vertical angle of the sound collection pointing direction toward the target sound source position A.
- the target sound source position A is the position of the actual monitoring target site corresponding to the designated position A ′ designated by the user's finger FG or the stylus pen in the video data displayed on the display device 36 by the operation unit 32. is there.
- the display device 36 as a display unit displays video data captured by the camera devices 11 to 1 n on the screen.
- the speaker device 37 as the sound output unit is sound collection directivity direction of sound collection directional coordinates ( ⁇ MAh , ⁇ MAv ) calculated by sound data collected by the omnidirectional microphone array device 2 or sound collection directional direction calculation unit 34 b.
- the omnidirectional microphone array device 2 outputs the voice data picked up after the pickup directivity is formed.
- the display device 36 and the speaker device 27 may be configured separately from the directivity control device 3.
- the memory 38 as a storage unit is configured by using, for example, a RAM (Random Access Memory), and functions as a program memory, a data memory, and a work memory when the respective units of the directivity control device 3 operate.
- the memory 38 also stores a setting file CF1 shown in FIG.
- the setting file CF1 includes, for example, data of height H C from the floor surface BL of the camera device 11, data of height H M from the floor surface BL of the omnidirectional microphone array device 2, and, for example, the camera device 11
- the data (first setting data) of the height HA from the floor surface BL of the target sound source position A corresponding to the predetermined designated position A ′ of the video data being processed is included at least.
- the recorder device 4 associates the video data captured by the camera devices 11 to 1 n with the audio data collected by the omnidirectional microphone array device 2 and records them.
- 34B shows the sound collection directivity direction from the video data displayed on the display device 36 from the omnidirectional microphone array device 2 to the target sound source position A corresponding to the designated position A 'designated by the user's finger FG. It is a figure which shows a mode that the sound volume level of the collection
- the omnidirectional microphone array device 2 uses the coordinate data of the sound collection directional coordinates ( ⁇ MAh , ⁇ MAv ) calculated by the directivity control device 3 to set the target sound source position A from the installation position M of the omnidirectional microphone array device 2 Sound pickup directivity is formed in the direction toward the head.
- Step ST2 includes the case where the user performs measurement using a measuring instrument (for example, a laser range finder) or the case where the camera device 11 performs measurement and acquisition using the function of the known technique of the camera device 11 itself.
- a measuring instrument for example, a laser range finder
- Each input parameter in step ST2 differs depending on the method of calculating the sound collection directional coordinates, and will be described in detail with reference to FIGS. 36 to 38.
- the signal processing unit 33 generates a setting file CF1 including the input parameters acquired in step ST3 and stores the setting file CF1 in the memory 38 (ST4). Thus, the operation of the initial setting of the directivity control system 10 is completed.
- FIG. 35 (B) is a flow chart for explaining the operation procedure after the initial setting of the directivity control system 10 of the seventh embodiment.
- the camera device 11 uses the installation position C of the camera device 11 as a starting point, and coordinate data of horizontal angle and vertical angle ( ⁇ CAh , ⁇ CAv ) up to the target sound source position A corresponding to the designated position A ′ designated in step ST11. Are transmitted to the directivity control device 3.
- the sound collection orientation direction calculation unit 34 b calculates the calculation result of Equation (64), the horizontal angle ⁇ CAh from the camera device 11 to the target sound source position A, and the distance L between the camera device 11 and the omnidirectional microphone array device 2 Using CM , the distance L MAh of the horizontal component of the distance from the omnidirectional microphone array device 2 to the target sound source position A is calculated according to the equation (72) according to the cosine theorem for ⁇ CAM shown in FIG. .
- the sound collection directivity direction calculation unit 34 b can calculate the vertical angle ⁇ MAv of the depression angle ⁇ MA from the omnidirectional microphone array device 2 to the target sound source position A according to Expression (76).
- the directivity control device 3 sets the height from the floor surface BL of the target sound source position A of the object present in the imaging direction in which the camera device 11 is imaging in the sound collecting space K.
- H A can be uniquely determined, and based on the height H A from the floor surface BL of the target sound source position A, the sound collection directivity direction from the omnidirectional microphone array device 2 toward the target sound source position A can be accurately calculated Can.
- the directivity control device 3 can form the sound collection directivity of the sound in the omnidirectional microphone array device 2 in the calculated sound collection directivity direction.
- FIG. 38A is a block diagram showing a system configuration of a directivity control system 10A of the eighth embodiment.
- Figure 38 (B) is a diagram showing displayed on the display device 36, and a selection screen WD2 for selecting the height H A from the floor surface BL of the display screen WD1 and the target sound source position A of the video data .
- the directivity control system 10 A shown in FIG. 38A includes at least one camera device 11 to 1 n, an omnidirectional microphone array device 2, a directivity control device 3 A, and a recorder device 4.
- the sound collection directivity direction calculation unit 34b of the signal processing unit 33A of the directivity control device 3A calculates the coordinate data of the horizontal angle and the vertical angle ( ⁇ CAh , ⁇ CAv ) from the camera device 11 to the target sound source position A in step ST15.
- Sound collecting directional coordinates from the installation position M of the omnidirectional microphone array device 2 to the target sound source position A using the height HA from the floor surface BL of the target sound source position A determined according to the selected option ( ⁇ MAh , ⁇ MAv ) is calculated (ST13A).
- the calculation content of step ST13A and the operation content of step ST14 are the same as the calculation content and operation content in the above-described seventh embodiment, and thus the description thereof will be omitted. Thus, the operation after the initial setting of the directivity control system 10A is completed.
- the memory 38A of the directivity control device 3A has a plurality of heights HA-A and H as heights of the target sound source position A from the floor surface BL. a-B, H a-D , the configuration file CF2 containing ... of height data (second setting data) is stored.
- the output control unit 34cA of the directivity control device 3A causes the display device 36 to display, as the height from the floor surface BL of the target sound source position A, a plurality of choices of height data of the second setting data of the setting file CF2. .
- FIG. 40A is a block diagram showing a system configuration of a directivity control system 10B of the ninth embodiment.
- FIG. 40B is a diagram showing a display screen WD1 of video data and an input form screen WD3 for inputting the height of the target sound source position A from the floor surface BL displayed on the display device 36.
- a directivity control system 10 B shown in FIG. 40A includes at least one camera device 11 to 1 n, an omnidirectional microphone array device 2, a directivity control device 3 B, and a recorder device 4.
- the output control unit 34cB of the signal processing unit 33B is an input form screen for allowing the user to directly input the height from the floor surface BL of the target sound source position A corresponding to the designated position A ′ designated by the user's finger FG.
- the WD 3 is displayed on the display unit 36.
- the sound source height determination unit 34aB of the signal processing unit 33B determines the numerical value of the height from the floor surface BL of the target sound source position A input to the input form screen WD3 from the floor surface BL of the target sound source position A desired by the user. It is judged as the height HA of
- FIG. 41A is a flowchart for describing an operation procedure of initial setting of the directivity control system 10B of the ninth embodiment.
- steps having the same contents as the flowchart shown in FIG. 35A will be assigned the same step numbers to omit or simplify the description, and different contents will be described.
- each input parameter measured in step ST2 is input from the camera device 11 to the signal processing unit 33B of the directivity control device 3B or from the operation unit 32. Is input to (ST3B).
- step ST3B as opposed to a step ST3A shown in step ST3, or 40 shown in FIG. 35 (A) (A), the input parameter is not included in the height H A from the floor surface BL of the target sound source position A .
- the signal processing unit 33B generates a setting file CF3 including the input parameters acquired in step ST3B and stores the setting file CF3 in the memory 38A (ST4). Thus, the operation of the initial setting of the directivity control system 10B is completed.
- FIG. 41 (B) is a flow chart for explaining the operation procedure after the initial setting of the directivity control system 10B of the ninth embodiment.
- the steps having the same contents as the flowchart shown in FIG. 35B will be assigned the same step numbers to omit or simplify the description, and different contents will be described.
- the output control unit 34cB directly gives the user the height from the floor surface BL of the target sound source position A corresponding to the designated position A ′ designated by the user's finger FG.
- An input form screen WD3 for inputting is displayed on the display device 36 (see FIG. 40 (B)).
- a numerical value that is the height from the floor surface BL of the target sound source position A is directly input by the user's input operation (for example, an input with a finger FG or an input from a keyboard (not shown)). It is assumed that (ST16).
- the sound source height determination unit 34aB determines the numerical value of the height from the floor surface BL of the target sound source position A input to the input form screen WD3 to the height H A from the floor surface BL of the target sound source position A desired by the user. Determined as
- the sound collection directivity direction calculation unit 34b of the signal processing unit 33B of the directivity control device 3B calculates the coordinate data of the horizontal angle and the vertical angle ( ⁇ CAh , ⁇ CAv ) from the camera device 11 to the target sound source position A and step ST16. Using the height HA from the floor surface BL of the target sound source position A determined according to the input to the input form screen WD3, the collection from the installation position M of the omnidirectional microphone array device 2 to the target sound source position A Sound directional coordinates ( ⁇ MAh , ⁇ MAv ) are calculated (ST13B).
- the calculation content of step ST13B and the operation content of step ST14 are the same as the calculation content and operation content in the above-described seventh embodiment, and thus the description thereof will be omitted. Thus, the operation after the initial setting of the directivity control system 10B is completed.
- the output control unit 34cB of the directivity control device 3B directly inputs the height of the target sound source position A from the floor surface BL to the input form screen WD3. It is displayed on the display unit 36.
- the sound source height determination unit 34aB of the directivity control device 3B responds to the input of the height from the floor surface BL of the target sound source position A on the displayed input form screen WD3 to the target sound source position desired by the user.
- the height from the floor surface BL of A can be determined accurately.
- the sound collection directivity direction calculation unit 34b of the directivity control device 3B receives the sound collection directivity from the omnidirectional microphone array device 2 toward the target sound source position A corresponding to the specified designated position A ′. Coordinates can be calculated accurately.
- the directivity control device 3C selects, for example, the first designated position A1 around the object (for example, a person) by the finger FG of the user.
- the second designated position A2 'on the floor surface BL in the vertically downward direction (directly below) of the second designated position A1' is designated, the floor surface of the target sound source position A1 corresponding to the first designated position A1 ' Calculate the height from BL.
- Directivity control apparatus 3C determines height calculated, as the height H A1 from the floor surface BL of the target sound source position A1.
- FIG. 42A is a block diagram showing a system configuration of a directivity control system 10C of the tenth embodiment.
- FIG. 42B is a diagram showing how the first designated position A1 'and the second designated position A2' are designated on the display screen WD4 of the video data displayed on the display device 36.
- a directivity control system 10C shown in FIG. 42A includes at least one camera device 11 to 1n, an omnidirectional microphone array device 2, a directivity control device 3B, and a recorder device 4.
- the operation procedure (see FIG. 43A) of the initial setting of the directivity control system 10C of this embodiment is the operation of the initial setting of the directivity control system 10B of the tenth embodiment shown in FIG. Since the procedure is the same as the procedure, the description is omitted.
- FIG. 43A is a flowchart for describing an operation procedure of initial setting of the directivity control system 10C of the tenth embodiment.
- the directivity control device 3C displays the first designated position A1 ′ and the second designated position A2 ′ from the display screen WD4 of the video data displayed on the display device 36 shown in FIG. 42 (B).
- the designation is received via the operation unit 32 (ST11C).
- the directivity control device 3C transmits to the camera device 11 that the designation of the first designated position A1 'and the second designated position A2' is received in the display screen WD4 of the video data displayed on the display device 36.
- step ST11C when the camera device 11 receives from the directivity control device 3C that the designation of the first designated position A1 ′ and the second designated position A2 ′ has been received, the installation position C of the camera device 11 is used as a starting point , Coordinate data of the horizontal angle and the vertical angle ( ⁇ CA1h , ⁇ CA1v ) to the target sound source position A1 corresponding to the first specified position A1 ′ specified in step ST11C, and the second specified position A2 specified in step ST11C Coordinate data of the horizontal angle and the vertical angle ( ⁇ CA2h , ⁇ CA2v ) up to the position A2 corresponding to 'are acquired (ST12 C).
- the camera device 11 takes the installation position C of the camera device 11 as a starting point and sets the horizontal angle and the vertical angle ( ⁇ CA1h , ⁇ CA1v ) to the target sound source position A1 corresponding to the first designated position A1 ′ designated in step ST11C.
- the coordinate data and the coordinate data of the horizontal angle and the vertical angle ( ⁇ CA2h , ⁇ CA2 v ) to the position A2 corresponding to the second designated position A2 ′ are transmitted to the directivity control device 3C.
- Height determining portion 34aC of the signal processing section 33C of the directional control device 3C is transmitted from the camera apparatus 11 ( ⁇ CA1h, ⁇ CA1v) coordinate data and ( ⁇ CA2h, ⁇ CA2v) of using the coordinate data of, calculating the height H A1 from the floor surface BL of the target sound source position A1 corresponding to the first designated position A1 'specified by the user's finger FG (ST17).
- the details of step ST17 will be described later with reference to FIGS. 44 and 45.
- the sound source height determination unit 34 a of the directivity control device 3 specifies the designated position of the designated video data in accordance with the designation of the designated position A ′ by the user in the video data displayed on the display unit 36.
- the height HA from the floor surface BL of the target sound source position A corresponding to A ′ is determined.
- the sound collection directivity direction calculation unit 34b of the directivity control device 3 corresponds to the designated position A 'designated by the omnidirectional microphone array device 2 based on the height HA from the floor surface BL of the target sound source position A.
- the sound collection directivity direction toward the target sound source position A to be calculated is calculated.
- the output control unit 34c of the directivity control device 3 causes the omnidirectional microphone array device 2 to form the sound collection directivity of the voice in the calculated sound collection directivity direction.
- the directivity control device 3 sets the height from the floor surface BL of the target sound source position A of the object present in the imaging direction in which the camera device 11 is imaging in the sound collecting space K.
- H A can be uniquely determined, and based on the height H A from the floor surface BL of the target sound source position A, the sound collection directivity direction from the omnidirectional microphone array device 2 toward the target sound source position A can be accurately calculated Can.
- the directivity control device 3 can form the sound collection directivity of the sound in the omnidirectional microphone array device 2 in the calculated sound collection directivity direction.
- control unit causes the display unit to display an input form of the height from the reference plane of the target sound source position, and the height determination unit displays the displayed input form It is a directivity control system which determines the height from the standard plane of the object sound source position corresponding to the specified above-mentioned designated position according to the input of the height from the standard plane of the above-mentioned object sound source position to it.
- the directivity control device 3C makes it unnecessary to create a setting file in which the target sound source position corresponding to the specified position of the video data and the height from the floor surface BL of the target sound source position are associated in advance. From the floor surface BL of the target sound source position A1 desired by the user according to a simple designation operation of designation of two designated positions without displaying the choice of height from the surface BL or the input form on the display device 36 The height can be calculated accurately.
- the directivity control device 3 sets the height from the floor surface BL of the target sound source position A of the object present in the imaging direction in which the camera device 11 is imaging in the sound collecting space K.
- H A can be uniquely determined, and based on the height H A from the floor surface BL of the target sound source position A, the sound collection directivity direction from the omnidirectional microphone array device 2 toward the target sound source position A can be accurately calculated Can.
- the directivity control device 3 can form the sound collection directivity of the sound in the omnidirectional microphone array device 2 in the calculated sound collection directivity direction.
- Patent Document 1 assumes that the camera device, the microphone array device, and the subject (for example, a speaker) exist on the same plane on the assumption that, for example, a use mode in a video conference system. However, in the monitoring system described above, it is rare that all of the camera device, the microphone array device, and the subject (for example, a store clerk or a visitor person) are actually present on the same plane.
- the camera device and the microphone array device are often installed above the subject (for example, the ceiling surface of a store), the camera device, the microphone array device, and the subject should exist on two-dimensional planar coordinates. Rather, they often exist on three-dimensional three-dimensional coordinates.
- the present invention can also be expressed as a sound collection control method having an operation (step) performed by a device (for example, a directivity control device described later) that configures a sound collection system or a device that configures a sound collection system. It is.
- a device for example, a directivity control device described later
- the PTZ camera device 1 as an example of an imaging unit is fixedly installed, for example, on a ceiling or a stand (see FIG. 48A) in a store.
- the PTZ camera device 1 has a function as a surveillance camera in a surveillance system, for example, and is remotely controlled from a surveillance control room (not shown) connected to the network NW to mount in the known pan or tilt direction.
- An object for example, a person
- a predetermined sound collection area for example, a predetermined area in a store
- the PTZ camera device 1 transmits captured image data obtained by imaging to the directivity control device 3 or the recorder device 4 via the network NW.
- the PTZ camera device 1 when an arbitrary position (for example, designated position A ′) is designated by the finger FG of the user among the captured image data displayed on the display device 36 described later, the PTZ camera device 1 in the captured image data Coordinate data of the designated position A ′ is received from the directivity control device 3.
- the PTZ camera device 1 calculates parameters of the distance and direction from the installation position of the PTZ camera device 1 to the actual site position (target sound source position A) corresponding to the designated position A ′.
- the PTZ camera device 1 is a distance from the installation position of the PTZ camera device 1 to at least one of the calibration markers MAK and MAK3, the calibration floor marker MAK2, the omnidirectional microphone array device 2 and the target sound source position A. , Parameters of direction are transmitted to the directivity control device 3.
- the omnidirectional microphone array device 2 receives the sound collection direction coordinates ( ⁇ MAh , ⁇ MAv ) included in the directivity formation instruction according to the directivity formation instruction (see below) transmitted from the directivity control device 3. Sound collection directivity of each of the microphone units 22 and 23 is formed in the sound direction.
- the omnidirectional microphone array device 2 performs predetermined voice processing on voice data collected by each of the microphone units 22 and 23, and directivity control of voice data obtained by the predetermined voice processing via the network NW. Transmit to device 3 or recorder device 4.
- omnidirectional microphone array device 2 can relatively increase the volume level of the sound data in the sound collection direction in which the sound collection directivity is formed, and the sound volume level of sound data in the direction in which the sound collection directivity is not formed. It can be reduced relatively.
- the method of calculating the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) will be described later.
- the directivity control device 3 may be, for example, a stationary PC (Personal Computer) installed in a monitoring control room (not shown), or a portable telephone which can be carried by the user, a PDA (Personal Digital Assistant), a tablet terminal, a smartphone Or the like may be used.
- a stationary PC Personal Computer
- PDA Personal Digital Assistant
- the directivity control device 3 is configured to include at least a communication unit 31, an operation unit 32, a signal processing unit 33, a display device 36, a speaker device 37, and a memory 38.
- the signal processing unit 33 at least includes a sound collection direction calculation unit 34 and an output control unit 35.
- the communication unit 31 receives the captured image data transmitted by the PTZ camera device 1 or the audio data transmitted by the omnidirectional microphone array device 2, and outputs the data to the signal processing unit 33.
- the operation unit 32 is a user interface (UI: User Interface) for notifying the signal processing unit 33 of the content of the user's input operation, and is, for example, a pointing device such as a mouse or a keyboard.
- UI User Interface
- the operation unit 32 may be disposed, for example, in correspondence with the screen of the display device 36, and may be configured using a touch panel or a touch pad that allows an input operation with the user's finger FG or a stylus pen.
- the operation unit 32 is in a range where the user desires to increase the volume level, ie, FIG.
- the coordinate data of the designated position A ′ designated by the user's finger FG in the sound collection range B shown is output to the signal processing unit 33.
- the operation unit 32 includes calibration markers MAK and MAK3, calibration floor marker MAK2 (see below), and omnidirectional microphone array
- calibration markers MAK and MAK3 are designated by the user's finger FG
- calibration floor marker MAK2 see below
- omnidirectional microphone array When at least one of the devices 2 is designated by the user's finger FG, coordinate data of the designated position is output to the signal processing unit 33.
- the signal processing unit 33 is configured using, for example, a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP), and generally controls the operation of each unit of the directivity control device 3. Control processing, data input / output processing with other units, data calculation (calculation) processing, and data storage processing are performed.
- CPU central processing unit
- MPU micro processing unit
- DSP digital signal processor
- the communication unit 31 causes the PTZ camera device 1 to transmit the coordinate data.
- the sound collecting direction calculating unit 34 is a parameter of the distance and direction from the installation position of the PTZ camera device 1 to at least one of the calibration markers MAK and MAK3, the calibration floor marker MAK2, and the omnidirectional microphone array device 2 Is acquired from the communication unit 31.
- the sound collection direction calculation unit 34 acquires coordinate data of the designated position A ′ from the operation unit 32.
- the coordinate data is transmitted to the camera device 1.
- the sound collection direction calculation unit 34 acquires from the communication unit 31 parameters of the distance and direction from the installation position of the PTZ camera device 1 to the target sound source position A.
- the output control unit 35 controls the operation of the display device 36 and the speaker device 37 to display the captured image data transmitted from the PTZ camera device 1 on the display device 36, and the audio data transmitted from the omnidirectional microphone array device 2 Are output to the speaker device 37 as voice. Further, the output control unit 35 controls the operation of the omnidirectional microphone array device 2 and, for example, in the sound collection direction MIX indicated by the sound collection direction coordinates ( ⁇ MAh , ⁇ MAv ) calculated by the sound collection direction calculation unit 34.
- the sound collection directivity of the sound collected by the omnidirectional microphone array device 2 is formed in the omnidirectional microphone array device 2, or the sound collection directivity of the sound collected by the omnidirectional microphone array device 2 is output control unit 35 (see FIG. 48A).
- the display apparatus 36 as an example of a display part is comprised using LCD (Liquid Crystal Display) or organic EL (Electroluminescence), for example, and displays the captured image data of PTZ camera apparatus 1 under control of the output control part 35. Do.
- LCD Liquid Crystal Display
- organic EL Electrode
- the speaker device 37 as an example of the sound output unit is sound-oriented in the sound collection direction indicated by the sound data of the sound collected by the omnidirectional microphone array device 2 or the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) The voice data of the voice picked up after the sex is formed is output.
- the display device 36 and the speaker device 37 may be configured separately from the directivity control device 3.
- the recorder device 4 associates and stores captured image data of the PTZ camera device 1 and audio data of audio collected by the omnidirectional microphone array device 2.
- the directivity The control device 3 calculates the sound collecting direction coordinate ( ⁇ MAh , ⁇ MAv ) in the sound collecting direction from the installation position of the omnidirectional microphone array device 2 toward the target sound source position A using the coordinate data representing the designated position A ′.
- the omnidirectional microphone array device 2 uses the coordinate data of the sound collection direction coordinates ( ⁇ MAh , ⁇ MAv ) calculated by the directivity control device 3 to collect sound from the omnidirectional microphone array device 2 toward the target sound source position A Form the sound collection directivity (sound collection direction MIX) of the voice.
- the omnidirectional microphone array device 2 collects the sound volume level of the conversation (Hello) of two persons existing in the direction in which the sound collection directivity (sound collection direction MIX) is formed, the sound collection directivity (sound collection direction The volume level of the music ( ⁇ -) of the speaker device SP which does not exist in the direction in which MIX) is formed can be increased.
- the directivity control device 3 picks up the volume level of the conversation (Hello) of the two persons existing in the direction in which the sound collection directivity (sound collection direction MIX) is formed in the speaker device 37. It is possible to cause voice output to be larger than the volume level of the music ( ⁇ to) of the speaker device SP which does not exist in the direction in which the character (collection direction MIX) is formed (see FIG. 48B).
- FIG. 49A is a flowchart for describing the overall operation procedure of the sound collection system 10 of the present embodiment.
- FIG. 49 (B) is a flowchart for explaining in detail the operation procedure of calibration of the sound collection system 10 of the present embodiment.
- calibration is necessary for the directivity control device 3 to calculate the sound collection direction coordinates ( ⁇ MAh , ⁇ MAv ), and calculates different predetermined calibration parameters for each calibration method. Or defined as an operation to be acquired.
- the PTZ camera device 1 and the omnidirectional microphone array device 2 constituting the sound collection system 10 of this embodiment are fixed to different predetermined positions (for example, a ceiling surface or a stand in a room of an event hall). Is initially installed (ST11).
- FIG. 49 the processing content of the calibration will be described with reference to FIG. 49 (B).
- a plurality of calibration methods will be described with reference to FIG. 50 and subsequent drawings, but in FIG. 49B, contents common to the respective calibration methods will be described.
- at least one calibration marker MAK, MAK3 or calibration floor marker MAK2 is used, but in FIG. 49B, for example, the case where the calibration marker MAK is used explain.
- a calibration marker MAK is set (ST12-1).
- the calibration marker MAK is a solid or circular object such as a marker (for example, a ball or a paper) used in each calibration method, and is installed so as to be included within the imaging viewing angle of the PTZ camera device 1 Therefore, the display unit 36 displays the
- step ST12-1 when the calibration marker MAK displayed on the display device 36 is designated by the user's finger FG, the operation unit 32 acquires coordinate data of the designated position and outputs the coordinate data to the signal processing unit 33. To do (ST12-2).
- the sound collection direction calculation unit 34 acquires coordinate data of the calibration marker MAK displayed on the display device 36 from the operation unit 32, the collection direction calculation unit 34 causes the communication unit 31 to transmit the coordinate data to the PTZ camera device 1.
- the PTZ camera device 1 calculates parameters of the distance and direction from the installation position of the PTZ camera device 1 to the calibration marker MAK, and transmits it to the directivity control device 3.
- the communication unit 31 receives parameters of the distance and direction from the installation position of the PTZ camera device 1 to the calibration marker MAK from the PTZ camera device 1, and outputs the parameters to the signal processing unit 33.
- the sound collection direction calculation unit 34 acquires from the communication unit 31 parameters of the distance and direction from the installation position of the PTZ camera device 1 to the calibration marker MAK.
- the sound collection direction calculation unit 34 calculates different calibration parameters for each calibration method (see below) using parameters of the distance from the installation position of the PTZ camera device 1 to the calibration marker MAK and the direction (ST12) -3). After step ST12-3, the calibration marker MAK set in step ST12-1 is removed (ST12-4). The sound collection direction calculation unit 34 temporarily stores the calibration parameters calculated in step ST12-3 in the memory 38. Thus, the calibration process shown in step ST12 ends.
- the operation unit 32 is a range where the user desires to increase the volume level in the captured image data (see FIG. 48B) of the PTZ camera device 1 displayed on the display device 36, That is, coordinate data of the designated position A ′ corresponding to the target sound source position A is acquired and output to the signal processing unit 33 (ST13).
- the sound collection direction calculation unit 34 uses the parameters of the distance and direction from the installation position of the PTZ camera device 1 to the target sound source position A, and the calibration parameters calculated at the time of calibration in ST12, to obtain the omnidirectional microphone array device 2
- the sound collecting direction coordinate ( ⁇ MAh , ⁇ MAv ) in the sound collecting direction from the installation position of to the target sound source position A is calculated (ST14).
- the output control unit 35 generates a directivity formation instruction for forming the directivity of the voice in the sound collection direction indicated by the coordinate data of the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) calculated in step ST14.
- a directivity formation instruction including coordinate data of sound direction coordinates ( ⁇ MAh , ⁇ MAv ) is transmitted from the communication unit 31 to the omnidirectional microphone array device 2.
- the omnidirectional microphone array device 2 in accordance with the directivity formation instruction transmitted from the directivity control device 3, in the sound collection direction MIX indicated by the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) included in the directivity formation instruction
- the sound pickup directivity of each of the microphone units 22 and 23 is formed (ST15).
- step ST15 In the process of forming sound collection directivity shown in step ST15, it has been described that the omnidirectional microphone array apparatus 2 performs according to the directivity formation instruction transmitted from the directivity control apparatus 3. However, the directivity control apparatus 3 performs step S15.
- the processing for forming the sound collection directivity shown in FIG. Specifically, the output control unit 35 calculates the coordinate data of the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) calculated in step ST 14 and the sound data of the sound collected by the omnidirectional microphone array device 2. Using this, the directivity forming process described with reference to FIG. 3 is performed. As a result, the directivity control device 3 can easily obtain voice data in which sound collection directivity is formed with high accuracy in the sound collection direction MIX indicated by the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ).
- FIG. 50 to FIG. 79 show the method of calculating the calibration parameter and the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) which are different for each calibration method, in the sound collection direction calculation unit 34 of the directivity control device 3. It will be described in detail by reference.
- a total of ten calibration methods that is, ten calibration parameter calculation methods will be described, and a total of four sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) calculation methods will be described. Note that one of the total of ten calibration parameter calculation methods corresponds to one of the total of four sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) calculation methods.
- FIG. 50 is an explanatory diagram of a first calibration method in the eleventh embodiment.
- FIG. 51A is a diagram showing the positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2 and the calibration marker MAK in the first calibration method.
- 51 (B) is a horizontal direction top view of FIG. 51 (A).
- FIG. 51 (C) is a vertical direction sectional view taken along the line K-K 'of FIG. 51 (B).
- the sound collection direction calculation unit 34 uses the omnidirectional microphone array device 2 for calibration.
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 is calculated using the distance L MO to the marker MAK.
- the method of calculating the distance L CMh in the horizontal direction in the sound collection direction calculation unit 34 will be specifically described with reference to FIGS. 51 (B) and (C).
- the sound collection direction calculation unit 34 measures the vertical angle ⁇ COv from the PTZ camera device 1 to the calibration marker MAK and the omnidirectional microphone array device 2 to the calibration marker MAK at ⁇ COM shown in FIG. 51C .
- the sound collection direction calculation unit 34 (1) A horizontal angle ⁇ CMh from the PTZ camera device 1 to the omnidirectional microphone array device 2; (2) The horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2; (3) Horizontal angle ⁇ MCh from omnidirectional microphone array device 2 to PTZ camera device 1 (4) Using the horizontal angle ⁇ CAh and vertical angle ⁇ CAv from the PTZ camera device 1 to the target sound source position A, the sound collecting direction coordinate in the sound collecting direction from the omnidirectional microphone array device 2 to the target sound source position A Calculate ( ⁇ MAh , ⁇ MAv ). Referring to FIGS. 52 (B) and (C) and FIGS. 53 (B) and (C), the method of calculating the first sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) in the sound collection direction calculation unit 34 is specifically described. Explain it.
- the horizontal angle ⁇ CAh and the vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A are determined by the PTZ camera device 1 according to the designation of the designated position A 'in the captured image data displayed on the display device 36. The calculated value is used.
- the sound collection direction calculation unit 34 calculates each of the calculation results of Equations (78) and (79), the horizontal angle ⁇ CAh from the PTZ camera device 1 toward the target sound source position A, and the omnidirectional microphone array device 2 from the PTZ camera device 1 From the omnidirectional microphone array device 2 according to the equation (86) according to the cosine theorem in ⁇ CAM shown in FIG. 52 (B) using the angle ( ⁇ CAh - ⁇ CMh ) corresponding to the difference between the heading horizontal angle ⁇ CMh A horizontal distance L MAh of the distance L MA to the target sound source position A is calculated.
- the sound collection direction calculation unit 34 uses the calculation results of the equations (84) to (86) and the omnidirectional microphone array device 2 according to the equation (87) according to the cosine theorem in ⁇ CAM shown in FIG. calculating a cosine value cos [theta] MAh horizontal angle theta MAh toward the target sound source position a from. As a result, the sound collection direction calculation unit 34 can calculate the horizontal angle ⁇ MAh from the omnidirectional microphone array device 2 toward the target sound source position A according to equation (88).
- the sound collection direction calculation unit 34 uses the calculation result of Expression (86) and the distance L MO corresponding to the difference between the height from the floor of the omnidirectional microphone array device 2 and the target sound source position A.
- the loss tangent at ⁇ MAS "shown in FIG. 53 (C) calculates the tangent value tan .theta MAv vertical angle theta MAv toward the target sound source position a from the omnidirectional microphone array apparatus 2.
- the sound direction calculation unit 34 can calculate the vertical angle ⁇ MAv from the omnidirectional microphone array device 2 toward the target sound source position A according to Formula (90).
- the sound collection direction calculation unit 34 displays the calibration marker MAK displayed on the display device 36.
- the center point O is designated by the user's finger FG, for example, the PTZ camera device 1 is located directly below the omnidirectional microphone array device 2 because an obstacle is present on the floor directly below the omnidirectional microphone array device 2. Even when the floor surface can not be imaged, the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 can be easily calculated as a calibration parameter.
- FIGS. 54 and 55A to 55 C This will be described with reference to FIGS. 56 (A) to 56 (C) and FIGS. 57 (A) to (C).
- FIG. 56A is a diagram showing the positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2, and the target sound source position A in the second calibration method.
- FIG. 56 (B) is a horizontal direction top view of FIG. 56 (A).
- FIG. 56 (C) is a vertical direction sectional view taken along the line Q-Q 'of FIG. 56 (B).
- FIG. 57A is a diagram showing the positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2, and the target sound source position A in the second calibration method.
- FIG. 57 (B) is a horizontal direction top view of FIG. 57 (A).
- FIG. 57 (C) is a vertical direction sectional view taken along the line R-R 'in FIG. 57 (B).
- each height from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 is the same, and a predetermined position of the omnidirectional microphone array device 2 is directly below the vertical direction, for example
- the calibration marker MAK which is a solid object, is suspended using an STR or a string (see FIG. 55A).
- the distance L MO from the omnidirectional microphone array device 2 to the calibration marker MAK is constant.
- the calibration parameter in the second calibration method is the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2.
- the sound collection direction calculation unit 34 uses the PTZ camera device 1 to specify the calibration marker MAK.
- the horizontal angle ⁇ COh and the vertical angle ⁇ COv are obtained.
- the data of the horizontal angle ⁇ COh and the vertical angle ⁇ COv is calculated by the PTZ camera device 1 and then transmitted from the PTZ camera device 1 to the directivity control device 3.
- the sound collection direction calculation unit 34 (1) A horizontal angle ⁇ CMh from the PTZ camera device 1 to the omnidirectional microphone array device 2; (2) The horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2; (3) Horizontal angle ⁇ MCh from omnidirectional microphone array device 2 to PTZ camera device 1 (4) Each height H C and H M from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2, (5) Horizontal angle ⁇ CAh and vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A, (6) is input by the user, the height H A from the floor of the target sound source position A, The sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) in the sound collection direction from the omnidirectional microphone array device 2 toward the target sound source position A is calculated using Referring to FIGS. 56 (B) and (C) and FIGS. 57 (B) and (C), the method of calculating the second sound collection direction coordinate ( ⁇ MAh , ⁇ MAv )
- the sound collection direction calculation unit 34 calculates a distance (H C -H A ) corresponding to the difference between the height from the floor of the PTZ camera device 1 and the target sound source position A in ⁇ CAS ′ shown in FIG. , calculated using a vertical angle theta CAv directed from the PTZ camera device 1 to the target sound source position a, in accordance with equation (92), the horizontal distance L CAh of the distance L CA from PTZ camera device 1 to the target sound source position a Do.
- the sound collection direction calculation unit 34 calculates each of the calculation results of the equations (91) and (92), the horizontal angle ⁇ CAh from the PTZ camera device 1 to the target sound source position A, and the omnidirectional microphone array device 2 from the PTZ camera device 1 Using the angle ( ⁇ CAh - ⁇ CMh ) corresponding to the difference between the heading horizontal angle ⁇ CMh and the cosine theorem in ⁇ CAM shown in FIG. A horizontal distance L MAh of the distance L MA to the target sound source position A is calculated.
- the sound collection direction calculation unit 34 uses the calculation results of the equations (91) to (93) and the omnidirectional microphone array device 2 according to the equation (94) according to the cosine theorem in ⁇ CAM shown in FIG. calculating a cosine value cos [theta] MAh horizontal angle theta MAh toward the target sound source position a from.
- the sound collection direction calculation unit 34 can calculate the horizontal angle ⁇ MAh from the omnidirectional microphone array device 2 toward the target sound source position A according to Formula (95).
- the sound collection direction calculating unit 34 formulas and the calculation result of (93), the distance corresponding to the difference between the height from the floor of the omnidirectional microphone array apparatus 2 and the target sound source position A (H M -H A) And the tangent value tan ⁇ MAv of the vertical angle ⁇ MAv from the omnidirectional microphone array device 2 to the target sound source position A according to the equation (96) by the tangent in ⁇ MAS shown in FIG. 57C .
- the sound collection direction calculation unit 34 can calculate the vertical angle ⁇ MAv from the omnidirectional microphone array device 2 toward the target sound source position A according to Expression (97).
- the sound collection direction calculation unit 34 displays the calibration marker MAK displayed on the display device 36.
- the center point O is designated by the user's finger FG, for example, the PTZ camera device 1 is located directly below the omnidirectional microphone array device 2 because an obstacle is present on the floor directly below the omnidirectional microphone array device 2. Even when the floor surface can not be imaged, the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 can be easily calculated as a calibration parameter.
- the sound collection direction calculation unit 34 uses the height H M from the floor of the omnidirectional microphone array device 2 which is relatively easy to measure at the time of initial installation of the omnidirectional microphone array device 2. Therefore, when the calibration marker MAK is suspended from the omnidirectional microphone array device 2, adjustment of the suspension distance can be simplified. Furthermore, the height H A from the floor of the target sound source position A, since the desired value of the user becomes available, thereby improving the degree of freedom of selection of the height H A from the floor of the target sound source position A.
- each height from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 is the same, and a position on the floor directly below the predetermined position of the omnidirectional microphone array device 2 in the vertical direction
- the calibration parameters in the third calibration method are the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 and the height H M from the floor of the omnidirectional microphone array device 2 is there.
- the sound collection direction calculation unit 34 uses the PTZ camera device 1 for calibration. Horizontal distance between PTZ camera device 1 and omnidirectional microphone array device 2 using distance L CO to floor marker MAK 2 and vertical angle ⁇ COv from PTZ camera device 1 to calibration floor marker MAK 2 The distance L CMh in the direction and the height H M from the floor of the omnidirectional microphone array device 2 are calculated. 59 (B) and (C), calculation of distance L CMh in the horizontal direction in sound collection direction calculation unit 34 and height H M from the horizontal surface (eg floor) of omnidirectional microphone array device 2 The method will be specifically described.
- the sound collection direction calculation unit 34 uses the PTZ camera device 1 for calibration.
- the distance L CO to the floor marker MAK 2 and the horizontal angle ⁇ COh and the vertical angle ⁇ COv toward the floor marker MAK 2 for calibration from the PTZ camera device 1 are acquired.
- the data of the distance L CO , the horizontal angle ⁇ COh and the vertical angle ⁇ COv is calculated by the PTZ camera device 1 and then transmitted from the PTZ camera device 1 to the directivity control device 3.
- the sound collection direction calculation unit 34 displays the calibration floor marker displayed on the display device 36.
- the center point O (see FIG. 59) of MAK 2 is designated by the user's finger FG
- the distance L CO from PTZ camera device 1 to calibration floor marker MAK 2 using the focus function of PTZ camera device 1 can be easily calculated, and further, the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 can be easily calculated as a calibration parameter.
- FIGS. 60 and 61A to 61C Method of calculating second sound collection direction coordinate according to the fourth calibration method
- the calibration is performed.
- the floor marker MAK2 is located at the center of the screen of the display unit 36 (see FIG. 60).
- each height from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 is the same, and a position on the floor directly below the predetermined position of the omnidirectional microphone array device 2 in the vertical direction
- the calibration parameters in the fourth calibration method are the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 and the height H M from the floor of the omnidirectional microphone array device 2 is there.
- the output control unit 35 sets the guide line GUD for indicating the place to be specified next. Display.
- the output control unit 35 can allow the user to easily grasp the position of the end point O 'to be specified next to the center point of the calibration floor marker MAK2 displayed on the display device 36.
- the sound collection direction calculation unit 34 calculates the vertical angle ⁇ COv from the PTZ camera device 1 toward the central point O of the calibration floor marker MAK 2 at ⁇ COM shown in FIG. 61C and the omnidirectional microphone array device 2.
- a relational expression between the height H M from the floor and the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 is calculated according to the equation (100).
- the sound collection direction calculation unit 34 sets a vertical angle ⁇ CO′v from the PTZ camera device 1 to the end point O ′ of the floor marker MAK2 for calibration at ⁇ CO′T shown in FIG. 61C .
- the sound collection direction calculation unit 34 calculates the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 using the equations (100) and (101). Calculate according to 102).
- the sound collection direction calculation unit 34 calculates the height H M from the floor of the omnidirectional microphone array device 2 according to the equation (103) using the calculation results of the equations (100) and (102). In addition, the sound collection direction calculation unit 34 can also calculate the height H C from the floor of the PTZ camera device 1 according to Expression (103).
- the method of calculating the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) in the sound collection direction calculation unit 34 after the calibration parameter is calculated by the fourth calibration method is the second sound collection direction coordinate described above. The description is omitted because it is the same as the calculation method of ( ⁇ MAh , ⁇ MAv ).
- FIGS. 62 and 63A to 63C Method of calculating the first sound collecting direction coordinate according to the fifth calibration method
- FIG. 62 is an explanatory diagram of a fifth calibration method in the eleventh embodiment.
- FIG. 63A shows a positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2 and the calibration marker MAK3 in the fifth calibration method.
- FIG. 63 (B) is a horizontal direction top view of FIG. 63 (A).
- FIG. 63 (C) is a vertical direction sectional view taken along the line K-K 'of FIG. 63 (B).
- the calibration marker MAK 3 is located at the center point of the screen of display device 36 (see FIG. 62).
- each height from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 is the same, and the floor of the omnidirectional microphone array device 2 is directly below the predetermined position from the predetermined position.
- a circular calibration marker MAK3 is placed at a position where the height from the center of the circle is constant (see FIG. 63A). Therefore, the height H 2 O from the floor of the calibration marker MAK 3 is a predetermined value.
- the sound collection direction calculation unit 34 causes the PTZ camera device 1 to specify the calibration marker MAK3.
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 using the distance L COv up to and the vertical angle ⁇ COv from the PTZ camera device 1 toward the calibration marker MAK 3 The distance L MOv between the omnidirectional microphone array device 2 and the calibration marker MAK 3 is calculated. Referring to FIG. 63 (B) and (C), the horizontal distance L CMh in sound collection direction calculation unit 34, and the distance L MOV between the omnidirectional microphone array apparatus 2 and the calibration marker MAK3 The calculation method will be specifically described.
- the sound collection direction calculation unit 34 causes the PTZ camera device 1 to specify the calibration marker MAK3.
- the data of the distance L CO , the horizontal angle ⁇ COh and the vertical angle ⁇ COv is calculated by the PTZ camera device 1 and then transmitted from the PTZ camera device 1 to the directivity control device 3.
- the calibration marker MAK3 is installed at a position at a fixed height from the floor just below the predetermined position of the omnidirectional microphone array device 2 from the PTZ camera device 1 toward the calibration marker MAK3.
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 is calculated according to Formula (104) using the vertical angle ⁇ COv toward the center point O of the calibration marker MAK 3 .
- the distance L MOv between the omnidirectional microphone array device 2 and the calibration marker MAK 3 is calculated according to the equation (105) using the vertical angle ⁇ COv directed to the marker MAK 3.
- the method of calculating the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) in the sound collection direction calculation unit 34 after the calibration parameter has been calculated by the fifth calibration method is the first sound collection direction coordinate described above. The description is omitted because it is the same as the calculation method of ( ⁇ MAh , ⁇ MAv ).
- FIGS. 64 and 65A to 65C Method of calculating third sound collection direction coordinates after calculation of calibration parameters by the sixth calibration method and the sixth calibration method. This will be described with reference to FIGS. 66 (A) to 66 (C) and FIGS. 67 (A) to (C).
- FIG. 66A is a diagram showing the positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2, and the target sound source position A in the sixth calibration method.
- FIG. 66 (B) is a horizontal direction top view of FIG. 66 (A).
- FIG. 66 (C) is a vertical direction sectional view taken along the line Q-Q 'of FIG. 66 (B).
- FIG. 67A shows a positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2 and the target sound source position A in the sixth calibration method.
- FIG. 67 (B) is a horizontal direction top view of FIG. 67 (A).
- FIG. 67 (C) is a vertical direction sectional view taken along the line R-R 'of FIG. 67 (B).
- each height from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 is different, and the omnidirectional microphone array device 2 exists within the imaging view angle of the PTZ camera device 1 To be installed. For this reason, after the PTZ camera device 1 is driven in the pan direction and the tilt direction, both the omnidirectional microphone array device 2 and the calibration marker MAK are focused and zoomed, so that the omnidirectional microphone array device 2 and the calibration are used.
- the marker MAK is located at the center point of the screen of the display device 36 (see FIG. 64).
- the height H M from the floor of the omnidirectional microphone array device 2 is higher than the height H C of the PTZ camera device 1 from the floor (H C ⁇ H M ), and the omnidirectional microphone
- the calibration marker MAK which is a solid object, is suspended from the predetermined position of the array device 2 directly under the vertical direction using, for example, a yarn STR or a string (see FIG. 65A).
- the distance L MO from the omnidirectional microphone array device 2 to the calibration marker MAK is constant.
- the calibration parameters in the sixth calibration method are the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 and the vertical distance between the PTZ camera device 1 and the calibration marker MAK. It is the distance L MOv ' of the direction.
- the sound collection direction calculation unit 34 designates two places of the center point O of the calibration marker MAK displayed on the display device 36 and the omnidirectional microphone array device 2 (see FIG. 64) by the user's finger FG, PTZ camera device 1 and omnidirectional microphone array device using vertical angle ⁇ COv from PTZ camera device 1 toward calibration marker MAK and vertical angle ⁇ CMv from PTZ camera device 1 to omnidirectional microphone array device 2
- the horizontal distance L CMh between 2 and 2 and the vertical distance L MOv ' between the PTZ camera device 1 and the calibration marker MAK are calculated.
- a method of calculating the distance L CMh in the horizontal direction and the distance L MOv 'in the vertical direction in the sound collection direction calculation unit 34 will be specifically described with reference to FIGS. 65 (B) and 65 (C).
- the sound collection direction calculation unit 34 designates two places of the center point O of the calibration marker MAK displayed on the display device 36 and the omnidirectional microphone array device 2 (see FIG. 64) by the user's finger FG, horizontal angle theta COH directed from the PTZ camera apparatus 1 in the marker MAK calibration, to obtain the vertical angle theta CoV, horizontal angle theta CMh directed from the PTZ camera device 1 to the omnidirectional microphone array apparatus 2, and a vertical angle theta CMv.
- the data of the horizontal angle ⁇ COh , the vertical angle ⁇ COv , the horizontal angle ⁇ CMh and the vertical angle ⁇ CMv is calculated by the PTZ camera device 1 and transmitted from the PTZ camera device 1 to the directivity control device 3 .
- the sound collection direction calculation unit 34 calculates the vertical angle ⁇ COv from the PTZ camera device 1 to the central point O of the calibration marker MAK at ⁇ COP shown in FIG. 65C, the PTZ camera device 1 and the omnidirectional microphone array
- the relationship between the horizontal distance L CMh between the device 2 and the vertical distance L MOv ' between the PTZ camera device 1 and the calibration marker MAK is calculated according to equation (106).
- the vertical distance L MOV between the omnidirectional microphone array apparatus 2 and the calibration marker MAK includes a distance L MOV 'in the vertical direction between the PTZ camera device 1 and the calibration marker MAK, PTZ Since this is the sum of the distance L MOv "in the vertical direction between the camera device 1 and the omnidirectional microphone array device 2, equation (108) is established.
- the sound collection direction calculation unit 34 calculates the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 using the equations (106) to (108). And the vertical distance L.sub.MOv ' between the PTZ camera device 1 and the calibration marker MAK is calculated according to equation (110), and the distance between the PTZ camera device 1 and the omnidirectional microphone array device 2 is calculated.
- the vertical distance L MOv " is calculated according to equation (111).
- the sound collection direction calculation unit 34 (1) Horizontal angle ⁇ CMh and vertical angle ⁇ CMv from the PTZ camera device 1 to the omnidirectional microphone array device 2 (2) The horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2; (3) Horizontal angle ⁇ MCh from omnidirectional microphone array device 2 to PTZ camera device 1 (4) The height L MOv ′ from the target sound source position A of the PTZ camera device 1 and the height L MO from the target sound source position A of the omnidirectional microphone array device 2 (5) Using the horizontal angle ⁇ CAh and vertical angle ⁇ CAv from the PTZ camera device 1 to the target sound source position A, the sound collecting direction coordinate in the sound collecting direction from the omnidirectional microphone array device 2 to the target sound source position A Calculate ( ⁇ MAh , ⁇ MAv ).
- the height L MOv ′ from the target sound source position A of the PTZ camera device 1 is a calibration parameter in the sixth calibration method (see Equation (110)), and the target sound source of the omnidirectional microphone array device 2
- the height LMO from the position A is a default value.
- the horizontal angle ⁇ CAh and the vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A are determined by the PTZ camera device 1 according to the designation of the designated position A ′ in the captured image data displayed on the display device 36 The calculated value is used.
- the sound collection direction calculation unit 34 calculates the vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A and the PTZ camera device 1 that is the calculation result of Formula (110) in ⁇ CAS ′ shown in FIG. And the vertical distance L MOv ' between the calibration marker MAK and the horizontal distance L CAh of the distance L CA from the PTZ camera device 1 to the target sound source position A according to formula (112) calculate.
- the sound collection direction calculation unit 34 calculates the calculation results of Equations (109) and (112), the horizontal angle ⁇ CAh from the PTZ camera device 1 to the target sound source position A, the omnidirectional microphone array device 2 from the PTZ camera device 1
- the horizontal direction of the distance L MA from the omnidirectional microphone array device 2 to the target sound source position A according to the formula (113) according to Expression (113) according to the cosine theorem in ⁇ CAM shown in FIG. Calculate the distance L MAh of
- the sound collecting direction calculating unit 34 uses the calculation results of the equations (109), (112), and (113) to obtain all directions according to the equation (114) according to the cosine theorem in ⁇ CAM shown in FIG. calculating a cosine value cos [theta] MAh horizontal angle theta MAh directed from the microphone array device 2 to the target sound source position a.
- the sound collection direction calculation unit 34 can calculate the horizontal angle ⁇ MAh from the omnidirectional microphone array device 2 toward the target sound source position A in accordance with Expression (115).
- the sound collection direction calculation unit 34 uses the calculation results of the equations (108) and (113) to calculate the omnidirectional microphone array according to the equation (116) by the tangent at .DELTA.MAS shown in FIG. calculated from the device 2 the tangent value tan .theta MAv vertical angle theta MAv toward the target sound source position a.
- the sound collection direction calculating unit 34, the omnidirectional microphone array apparatus vertical angle theta MAv going from 2 to target sound source position a Can be calculated according to equation (117).
- the sound collection direction calculation unit 34 generates the calibration marker MAK displayed on the display device 36.
- two points of center point O and omnidirectional microphone array device 2 are designated by the user's finger FG, for example, an obstacle is present on the floor directly below omnidirectional microphone array device 2
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 can be used as a calibration parameter. It can be calculated to
- the sound collection direction calculating unit 34 even if the height from the floor PTZ camera device 1 and the omnidirectional microphone array apparatus 2 are different, thereby inputting the height H A from the floor of the target sound source position A to the user Without, it is possible to easily calculate the coordinate in the sound collection direction from the omnidirectional microphone array device 2 toward the target sound source position A.
- FIGS. 68 and 69A to 69C This will be described with reference to FIGS. 70 (A) to 70 (C) and FIGS. 71 (A) to (C).
- FIG. 68 is an explanatory diagram of a seventh calibration method in the eleventh embodiment.
- FIG. 69A shows a positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2 and the floor marker MAK2 for calibration in the seventh calibration method.
- FIG. 69 (B) is a horizontal direction top view of FIG. 69 (A).
- FIG. 69 (C) is a vertical direction sectional view taken along the line K-K 'in FIG. 69 (B).
- FIG. 70A is a diagram showing the positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2, and the target sound source position A in the seventh calibration method.
- FIG. 70 (B) is a horizontal direction top view of FIG. 70 (A).
- FIG. 70 (C) is a vertical direction sectional view taken along the line Q-Q 'of FIG. 70 (B).
- FIG. 71A shows a positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2, and the target sound source position A in the seventh calibration method.
- FIG. 71 (B) is a horizontal direction top view of FIG. 71 (A).
- FIG. 71 (C) is a vertical direction sectional view taken along line R-R 'of FIG. 71 (B).
- each height from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 is different, and the omnidirectional microphone array device 2 exists within the imaging view angle of the PTZ camera device 1 To be installed. For this reason, after the PTZ camera device 1 is driven in the pan direction and the tilt direction, both the omnidirectional microphone array device 2 and the calibration marker MAK are focused and zoomed, so that the omnidirectional microphone array device 2 and the calibration are used.
- the marker MAK is located at the center point of the screen of the display device 36 (see FIG. 68).
- two points of the center point O of the calibration floor marker MAK2 displayed on the display device 36 and the omnidirectional microphone array device 2 are designated by the user's finger FG
- a vertical angle theta CoV directed from the PTZ camera device 1 to the calibration floor standing marker MAK2 a vertical angle theta CMv directed from the PTZ camera device 1 to the omnidirectional microphone array apparatus 2
- floor calibration from PTZ camera device 1 Place by using the distance L CoV to marker MAK2
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array apparatus 2 the height H C from the floor PTZ camera apparatus 1
- the height H M from the floor of the omnidirectional microphone array device 2 is calculated.
- a method of calculating the distance L CMh in the horizontal direction and the heights H M and H C in the sound collection direction calculation unit 34 will be specifically described with reference to FIGS. 69 (B) and 69 (C).
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 is calculated according to equation (118) using the vertical angle ⁇ COv toward the center point O of the floor marker MAK 2
- the height H C from the floor of the PTZ camera device 1 is calculated according to equation (119).
- the sound collection direction calculation unit 34 uses the calculation result of equation (118) and the vertical angle ⁇ CMv directed from the PTZ camera device 1 to the omnidirectional microphone array device 2 in ⁇ CMP shown in FIG. 69C .
- the distance L MOv corresponding to the difference (H M ⁇ H C ) between the height H M from the floor of the omnidirectional microphone array device 2 and the height H C from the floor of the PTZ camera device 1 according to the formula (120)
- the sound collection direction calculation unit 34 uses the calculation results of the equations (119) and (120) to calculate the height H M from the floor of the omnidirectional microphone array device 2 according to the equation (121). calculate.
- the height of the target sound source position A from the floor is an input value input by the user, and the third sound collection direction coordinate It differs from the calculation method of ( ⁇ MAh , ⁇ MAv ). Further, the heights H C and H M from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 are different.
- the sound collection direction calculation unit 34 (1) A horizontal angle ⁇ CMh from the PTZ camera device 1 to the omnidirectional microphone array device 2; (2) The horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2; (3) Horizontal angle ⁇ MCh from omnidirectional microphone array device 2 to PTZ camera device 1 (4) Each height H C and H M from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2, (5) Horizontal angle ⁇ CAh and vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A, (6) is input by the user, the height H A from the floor of the target sound source position A, The sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) in the sound collection direction from the omnidirectional microphone array device 2 toward the target sound source position A is calculated using Referring to FIGS. 70 (B) and (C) and FIGS. 71 (B) and (C), the method of calculating the fourth sound collection direction coordinate ( ⁇ MAh , ⁇ MAv )
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 is a calibration parameter in the seventh calibration method (see equation (112)).
- the horizontal angle ⁇ MCh from the omnidirectional microphone array device 2 to the PTZ camera device 1 is a predetermined value (zero) given by the positional relationship between the PTZ camera device 1 and the omnidirectional microphone array device 2 in the seventh calibration method. ).
- the heights H C and H M from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 are calibration parameters in the seventh calibration method (see Equations (119) and (121). ).
- the horizontal angle ⁇ CAh and the vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A are determined by the PTZ camera device 1 according to the designation of the designated position A ′ in the captured image data displayed on the display device 36 The calculated value is used.
- the height H A of the horizontal plane of the target sound source position A e.g. the floor
- a value entered by the input operation of the user a value entered by the input operation of the user.
- the sound collection direction calculation unit 34 calculates a distance corresponding to the difference in height from the floor (H C -H A ) between the PTZ camera device 1 and the target sound source position A in ⁇ CAS ′ shown in FIG. 70 (C). , calculated using a vertical angle theta CAv directed from the PTZ camera device 1 to the target sound source position a, according to equation (122), the horizontal distance L CAh of the distance L CA from PTZ camera device 1 to the target sound source position a Do.
- the sound collection direction calculation unit 34 calculates the calculation results of Equations (118) and (122), the horizontal angle ⁇ CAh from the PTZ camera device 1 to the target sound source position A, the omnidirectional microphone array device 2 from the PTZ camera device 1
- the horizontal direction of the distance L MA from the omnidirectional microphone array device 2 to the target sound source position A according to the equation (123) according to the cosine theorem in ⁇ CAM shown in FIG. 70 (B) using the horizontal angle ⁇ CMh Calculate the distance L MAh of
- the sound collection direction calculation unit 34 uses the calculation results of the equations (118), (122), and (123) to obtain all directions according to the equation (124) according to the cosine theorem in ⁇ CAM shown in FIG. calculating a cosine value cos [theta] MAh horizontal angle theta MAh directed from the microphone array device 2 to the target sound source position a. As a result, the sound collection direction calculation unit 34 can calculate the horizontal angle ⁇ MAh from the omnidirectional microphone array device 2 toward the target sound source position A in accordance with Expression (125).
- the sound collection direction calculation unit 34 71C uses the distance corresponding to the difference (H M -H A ) with the height H A and the tangent in ⁇ MAS ′ ′ shown in FIG. 71C according to the equation (126), the omnidirectional microphone array device 2 calculating a tangent value tan .theta MAv vertical angle theta MAv toward the target sound source position a from.
- the sound collection direction calculating unit 34, the vertical angle theta MAv toward the target sound source position a from the omnidirectional microphone array apparatus 2 It can be calculated according to equation (127).
- the sound collection direction calculation unit 34 displays the calibration floor marker displayed on the display device 36.
- the height H C of the horizontal plane of the PTZ camera device 1 and the omnidirectional microphone array apparatus 2 , H M can be easily calculated, and further, the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 can be easily calculated as a calibration parameter.
- the sound collection direction calculation unit 34 calculates the horizontal angle ⁇ CMh calculated at the time of calibration, the horizontal distance L CMh , and the heights H C and H from the horizontal plane of the PTZ camera device 1 and the omnidirectional microphone array device 2.
- M the predetermined horizontal angle ⁇ MCh , the horizontal angle ⁇ CAh and the vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A, and the height H A of the target sound source position A input by the user from the horizontal plane
- the sound pickup direction coordinate in the sound pickup direction from the omnidirectional microphone array device 2 to the target sound source position A can be easily calculated using the input values of The height H A from the floor of the target sound source position A, since the desired value of the user becomes available, thereby improving the degree of freedom of selection of the height H A from the floor of the target sound source position A.
- FIG. 72 is an explanatory diagram of an eighth calibration method in the eleventh embodiment.
- FIG. 73A is a diagram showing the positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2, the calibration marker MAK, and the calibration floor marker MAK2 in the eighth calibration method.
- FIG. 73 (B) is a horizontal direction top view of FIG. 73 (A).
- FIG. 73 (C) is a vertical direction sectional view taken along line K-K 'in FIG. 73 (B).
- the heights H C and H M from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 are different, and the omnidirectional microphone array device 2 has an imaging field of view of the PTZ camera device 1 It is installed to exist outside the corner. For this reason, after the PTZ camera device 1 is driven in the pan direction and in the tilt direction, both the calibration marker MAK and the calibration floor marker MAK2 are focused and zoomed, so that the calibration marker MAK and the calibration are used.
- the floor marker MAK2 is located at the center point of the screen of the display device 36 (see FIG. 72).
- the height H M from the floor of the omnidirectional microphone array device 2 is higher than the height H C of the PTZ camera device 1 from the floor (H C ⁇ H M ), and the omnidirectional microphone
- the calibration marker MAK which is a solid object, is suspended from a predetermined position of the array device 2 vertically below the predetermined position of the array device 2, and further, from the predetermined position of the omnidirectional microphone array device 2 in the vertical direction
- a circular calibration floor marker MAK2 is installed at a position on the floor directly below (see FIG. 73 (A)).
- the calibration parameters in the eighth calibration method are the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2, the height H C of the PTZ camera device 1 from the floor, and the total It is height H M from the floor of the azimuth microphone array device 2.
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 is expressed according to the equation (128) using the vertical angle ⁇ CO ′ ′ v from the device 1 to the calibration floor setting marker MAK 2 calculated, to calculate the height H C from the floor PTZ camera apparatus 1 according to equation (129).
- the calibration floor marker MAK2 is installed on the floor directly below the predetermined position of the omnidirectional microphone array device 2 in the vertical direction, so that it is between the PTZ camera device 1 and the calibration floor marker MAK2.
- the sound collection direction calculation unit 34 uses the calculation result of Expression (130) to obtain all directions.
- the height H M of the microphone array device 2 from the floor is calculated according to the equation (131).
- the method of calculating the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) in the sound collection direction calculation unit 34 after the calibration parameter is calculated by the eighth calibration method is the fourth sound collection direction coordinate described above. The description is omitted because it is the same as the calculation method of ( ⁇ MAh , ⁇ MAv ).
- the sound collection direction calculation unit 34 displays the calibration marker MAK and the calibration marker MAK displayed on the display device 36.
- the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 is selected.
- the heights H C and H M can be easily calculated, and further, the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 can be easily calculated as a calibration parameter.
- FIGS. 74 and 75A to 75C Method of calculating fourth sound collection direction coordinate according to ninth calibration method
- FIG. 74 is an explanatory diagram of a ninth calibration method in the eleventh embodiment.
- FIG. 75A is a diagram showing the positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2 and the floor marker MAK2 for calibration in the ninth calibration method.
- FIG. 75 (B) is a horizontal direction top view of FIG. 75 (A).
- FIG. 75 (C) is a vertical direction sectional view taken along the line K-K 'of FIG. 75 (B).
- the heights H C and H M from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 are different, and the omnidirectional microphone array device 2 has an imaging field of view of the PTZ camera device 1 It is installed to exist outside the corner. Therefore, after the PTZ camera device 1 is driven in the pan direction and in the tilt direction, the calibration floor marker MAK2 is focused and zoomed, so that the calibration floor marker MAK2 is at the center point of the screen of the display device 36. Position (see FIG. 74).
- the height H M from the floor of the omnidirectional microphone array device 2 is higher than the height H C of the PTZ camera device 1 from the floor (H C ⁇ H M ), and the omnidirectional microphone A circular calibration floor marker MAK2 is installed at a position on the floor directly below the predetermined position of the array device 2 in the vertical direction (see FIG. 75A).
- the calibration parameters in the ninth calibration method are the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 and the height H C of the PTZ camera device 1 from the floor.
- the sound collection direction calculation unit 34 uses the PTZ camera device 1 for calibration. Horizontal angle between the PTZ camera device 1 and the omnidirectional microphone array device 2 using the vertical angle ⁇ COv directed to the floor setting marker MAK 2 and the distance L COv from the PTZ camera device 1 to the floor setting marker MAK 2 for calibration The distance L CMh in the direction and the height H C from the floor of the PTZ camera device 1 are calculated. A method of calculating the distance L CMh in the horizontal direction and the height H C in the sound collection direction calculation unit 34 will be specifically described with reference to FIGS. 75 (B) and 75 (C).
- the sound collection direction calculation unit 34 uses the PTZ camera device 1 for calibration.
- the distance L COv to the floor marker MAK 2 and the horizontal angle ⁇ COh and the vertical angle ⁇ COv toward the floor marker MAK 2 for calibration from the PTZ camera device 1 are acquired.
- the data of the distance L COv , the horizontal angle ⁇ COh , and the vertical angle ⁇ COv is calculated by the PTZ camera device 1 and then transmitted from the PTZ camera device 1 to the directivity control device 3.
- the method of calculating the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) in the sound collection direction calculation unit 34 after the calibration parameter is calculated by the ninth calibration method is the fourth sound collection direction coordinate described above. The description is omitted because it is the same as the calculation method of ( ⁇ MAh , ⁇ MAv ).
- the sound collection direction calculation unit 34 sets the center point O of the calibration floor marker MAK2 displayed on the display device 36 (see FIG. 74 Referring) is specified by the finger FG of the user, it can be calculated height H C from the floor PTZ camera apparatus 1 in a simple, further, the horizontal between the PTZ camera device 1 and the omnidirectional microphone array apparatus 2
- the distance L CMh in the direction can be easily calculated as a calibration parameter.
- FIGS. 76 and 77 (A) to (C) This will be described with reference to FIGS. 78 (A) to (C) and FIGS. 79 (A) to (C).
- FIG. 76 is an explanatory diagram of a tenth calibration method in the eleventh embodiment.
- FIG. 77A shows a positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2 and the calibration marker MAK in the tenth calibration method.
- FIG. 77 (B) is a horizontal direction top view of FIG. 77 (A).
- FIG. 77 (C) is a vertical direction sectional view taken along the line K-K 'of FIG. 77 (B).
- each height from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 is different, and the omnidirectional microphone array device 2 exists within the imaging view angle of the PTZ camera device 1 To be installed. For this reason, after the PTZ camera device 1 is driven in the pan direction and the tilt direction, both the omnidirectional microphone array device 2 and the calibration marker MAK are focused and zoomed, so that the omnidirectional microphone array device 2 and the calibration are used.
- the marker MAK is located at the center point of the screen of the display device 36 (see FIG. 76).
- the height H C from the floor of the PTZ camera device 1 is higher than the height H M from the floor of the omnidirectional microphone array device 2 (H M ⁇ H C ), and the omnidirectional microphone A circular calibration floor marker MAK2 is installed at a position on the floor directly below the predetermined position of the array device 2 in the vertical direction (see FIG. 77A).
- the calibration parameters in the tenth calibration method are the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2, the height H C of the PTZ camera device 1 from the floor, and the total It is height H M from the floor of the azimuth microphone array device 2.
- two points of the central point O of the calibration floor marker MAK2 displayed on the display device 36 and the omnidirectional microphone array device 2 are designated by the user's finger FG
- a vertical angle theta CoV directed from the PTZ camera device 1 to the calibration floor standing marker MAK2 a vertical angle theta CMv directed from the PTZ camera device 1 to the omnidirectional microphone array apparatus 2
- floor calibration from PTZ camera device 1 Place by using the distance L CoV to marker MAK2
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array apparatus 2 the height H C from the floor PTZ camera apparatus 1
- the height H M from the floor of the omnidirectional microphone array device 2 is calculated.
- FIGS. 77 (B) and (C) the method of calculating the distance L CMh in the horizontal direction and the heights H M and H C in the sound collection direction calculation unit 34 will be specifically described.
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 is calculated according to equation (134) using the vertical angle ⁇ COv toward the center point O of the floor marker MAK 2
- the height H C from the floor of the PTZ camera device 1 is calculated according to equation (135).
- the sound collection direction calculation unit 34 calculates the vertical axis ⁇ COv from the PTZ camera device 1 toward the center point O of the calibration floor marker MAK2 in the ⁇ CMP shown in FIG. 77C.
- Height from the floor of the PTZ camera device 1 H C and the omnidirectional microphone array using the difference between the vertical angle ⁇ CMv from the PTZ camera device 1 to the omnidirectional microphone array device 2 ( ⁇ COv - ⁇ CMv ) a distance difference between the height H M from the device 2 bed (H C -H M) is calculated according to equation (136), formulas the height H M from the omnidirectional microphone array apparatus 2 floor (137) Calculate according to
- the sound collection direction calculation unit 34 (1) A horizontal angle ⁇ CMh from the PTZ camera device 1 to the omnidirectional microphone array device 2; (2) The horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2; (3) Horizontal angle ⁇ MCh from omnidirectional microphone array device 2 to PTZ camera device 1 (4) Each height H C and H M from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2, (5) Horizontal angle ⁇ CAh and vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A, (6) is input by the user, the height H A from the floor of the target sound source position A, The sound collection direction coordinate ( ⁇ MAh , ⁇ MAv ) in the sound collection direction from the omnidirectional microphone array device 2 toward the target sound source position A is calculated using Referring to FIGS. 78 (B) and (C) and FIGS. 79 (B) and (C), the method of calculating the sound collection direction coordinate ( ⁇ MAh , ⁇ MAv )
- the heights H C and H M from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 are calibration parameters in the tenth calibration method (see Equations (135) and (137). ).
- the horizontal angle ⁇ CAh and the vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A are determined by the PTZ camera device 1 according to the designation of the designated position A ′ in the captured image data displayed on the display device 36 The calculated value is used.
- the height H A from the floor of the target sound source position A which is the value entered by the input operation of the user.
- the sound collection direction calculation unit 34 calculates the distance corresponding to the difference in height from the floor (H C -H A ) between the PTZ camera device 1 and the target sound source position A in ⁇ CAS ′ shown in FIG. 78 (C). , calculated using a vertical angle theta CAv directed from the PTZ camera device 1 to the target sound source position a, according to equation (138), the horizontal distance L CAh of the distance L CA from PTZ camera device 1 to the target sound source position a Do.
- the sound collection direction calculation unit 34 calculates the calculation results of Equations (134) and (138), the horizontal angle ⁇ CAh from the PTZ camera device 1 to the target sound source position A, and the omnidirectional microphone array device 2 from the PTZ camera device 1.
- the sound collection direction calculation unit 34 uses the calculation results of the equations (134), (138), and (139) to obtain all directions according to the equation (140) according to the cosine theorem in ⁇ CAM shown in FIG. calculating a cosine value cos [theta] MAh horizontal angle theta MAh directed from the microphone array device 2 to the target sound source position a.
- the sound collection direction calculation unit 34 can calculate the horizontal angle ⁇ MAh from the omnidirectional microphone array device 2 toward the target sound source position A according to equation (141).
- the sound collection direction calculation unit 34 uses the distance corresponding to the difference (H M -H A ) with the height HA of H, and the tangent at ⁇ MAS shown in FIG. 79C according to the equation (142), the omnidirectional microphone array device 2 calculating a tangent value tan .theta MAv vertical angle theta MAv toward the target sound source position a from.
- the sound collection direction calculation unit 34 displays the calibration floor marker displayed on the display device 36.
- H C from horizontal plane of PTZ camera device 1 and omnidirectional microphone array device 2
- H M can be easily calculated
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 can be easily calculated as a calibration parameter.
- the third sound collection direction coordinate according to the sixth calibration method similarly applies the fourth sound collection direction coordinate calculation method according to the calculation method or the eighth and ninth calibration methods, so that the sound collection direction calculation unit can generate the target sound source from the omnidirectional microphone array device 2
- the sound collecting direction coordinate in the sound collecting direction toward the position A can be calculated.
- an image pickup unit that picks up an object in a predetermined sound pickup area, a sound pickup unit that picks up sound in the sound pickup area, and an image of the object picked up by the image pickup unit
- a display unit for displaying data and designation of an arbitrary position in the image data of the object
- the sound collection direction from the sound collection unit toward the target sound source position corresponding to the specified position of the image data is calculated.
- a sound collection direction calculation unit, and a sound collection control unit that forms a sound collection directivity of the sound collected by the sound collection unit in the sound collection direction calculated by the sound collection direction calculation unit; Control system.
- the PTZ camera device 1 captures an object (for example, a person) present in a predetermined sound collection area using a well-known chassis drive function and zoom function in the pan direction or tilt direction. .
- the omnidirectional microphone array device 2 picks up the sound of an object present in the imaging direction of the PTZ camera device 1.
- sound collection direction calculation unit 34 of directivity control device 3 corresponds to designated position A 'from PTZ camera device 1 Sound collection direction coordinates ( ⁇ MAh , ⁇ MAv ) in the sound collection direction toward the target sound source position A to be calculated.
- the output control unit 35 of the directivity control device 3 collects the sound collection directivity of the sound collected by the omnidirectional microphone array device 2 in the sound collection direction indicated by the calculated sound collection direction coordinate ( ⁇ MAh , ⁇ MAv )
- the designated position A ′ corresponding to the target sound source position A is designated on the display device 36 on which the captured image data of the PTZ camera device 1 is displayed.
- input parameters for example, distance, direction
- the directivity control device 3 collects the sound in the sound collecting direction toward the target sound source position A corresponding to the designated position A ′ in the captured image data of the PTZ camera device 1
- the directivity can be easily formed, and the voice data of the voice in the sound collection direction collected with high accuracy can be easily obtained.
- the heights of the image pickup unit and the sound collection unit from the horizontal plane are the same, and the sound collection direction calculation unit is a horizontal angle from the image pickup unit to the sound collection unit A horizontal distance between the image pickup unit and the sound collection unit, a horizontal angle from the sound collection unit toward the image pickup unit, and a horizontal angle and a vertical angle from the image pickup unit to the target sound source position
- the directionality control system calculates the horizontal angle and the vertical angle in the sound collection direction from the sound collection unit toward the target sound source position using.
- the sound collection direction calculation unit 34 of the directivity control device 3 calculates the horizontal angle ⁇ CMh calculated during calibration, the horizontal distance L CMh , the predetermined horizontal angle ⁇ MCh, and the target sound source from the PTZ camera device 1
- the sound collection direction coordinate from the omnidirectional microphone array device 2 to the target sound source position A can be easily calculated using the horizontal angle ⁇ CAh and the vertical angle ⁇ CAv toward the position A.
- a first marker having a constant distance from the sound collection unit is suspended immediately below the sound collection unit, and the height of the first marker from the horizontal surface, and the object
- the height of the sound source position is the same as the height from the horizontal plane
- the sound collection direction calculation unit is configured to set the sound collection unit and the first marker in accordance with the designation of the first marker displayed on the display unit.
- the directivity control system calculates the horizontal distance between the imaging unit and the sound collection unit using the distance between the two.
- the calibration marker MAK having a constant distance (L MOv ) from the omnidirectional microphone array device 2 is suspended directly below the omnidirectional microphone array device 2, and the calibration marker MAK and the target sound source position Each height H O and H A from the floor of A is identical. Therefore, when the calibration marker MAK displayed on the display device 36 is designated by the user's finger FG, the sound collection direction calculation unit 34 of the directivity control device 3 may, for example, set the floor directly below the omnidirectional microphone array device 2 Even if the PTZ camera device 1 can not image the floor directly below the omnidirectional microphone array device 2 because of the presence of obstacles in the horizontal direction between the PTZ camera device 1 and the omnidirectional microphone array device 2 the distance L CMh can be calculated easily as the calibration parameters.
- the heights of the image pickup unit and the sound collection unit from the horizontal plane are the same, and the sound collection direction calculation unit is a horizontal angle from the image pickup unit to the sound collection unit A horizontal distance between the image pickup unit and the sound collecting unit, a horizontal angle from the sound collecting unit toward the image pickup unit, and heights of the image pickup unit and the sound collecting unit from a horizontal plane, A sound collection direction from the sound collection unit to the target sound source position using a horizontal angle and a vertical angle from the imaging unit to the target sound source position and an input value of the height of the target sound source position from the horizontal plane Directivity control system that calculates the horizontal angle and the vertical angle of
- the sound collection direction calculation unit 34 of the directivity control device 3 calculates the horizontal angle ⁇ CMh calculated at the time of calibration, the horizontal distance L CMh , the predetermined horizontal angle ⁇ MCh, and the predetermined PTZ camera device 1 and The heights H C and H M from the horizontal surface of the omnidirectional microphone array device 2, the horizontal angle ⁇ CAh and the vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A, and the target sound source position input by the user by using the input value of the height H a from the floor of a, omnidirectional microphone array system the sound collection direction of the collected sound direction coordinate going from 2 to target sound source position a can be calculated easily with.
- the sound collection direction calculation unit 34 of the directivity control device 3 uses the height H M from the horizontal surface of the omnidirectional microphone array device 2 which is relatively easy to measure at the time of initial installation of the omnidirectional microphone array device 2. Therefore, when the calibration marker MAK is suspended from the omnidirectional microphone array device 2, adjustment of the suspension distance can be simplified. Furthermore, the height H A from the floor of the target sound source position A, since the desired value of the user becomes available, thereby improving the degree of freedom of selection of the height H A from the floor of the target sound source position A.
- a first marker having a constant distance from the sound collection unit is suspended immediately below the sound collection unit, and the height of the sound collection unit from the horizontal plane is a predetermined value.
- the sound collection direction calculation unit uses the distance between the sound collection unit and the first marker according to the specification of the first marker displayed on the display unit. It is a directivity control system which calculates the distance of the horizontal direction between the sound collection parts.
- the calibration marker MAK having a constant distance (L MOv ) from the omnidirectional microphone array device 2 is suspended directly below the omnidirectional microphone array device 2, and from the horizontal surface of the omnidirectional microphone array device 2
- the height H M of is known. Therefore, when the calibration marker MAK displayed on the display device 36 is designated by the user's finger FG, the sound collection direction calculation unit 34 of the directivity control device 3 may, for example, set the floor directly below the omnidirectional microphone array device 2 Even if the PTZ camera device 1 can not image the floor directly below the omnidirectional microphone array device 2 because of the presence of obstacles in the horizontal direction between the PTZ camera device 1 and the omnidirectional microphone array device 2 the distance L CMh can be calculated easily as the calibration parameters.
- a circular second marker is placed at a position on the horizontal surface directly below the sound collection unit, and the sound collection direction calculation unit is displayed on the display unit.
- the imaging unit and the sound collection unit using the distance from the imaging unit to the second marker and the vertical angle from the imaging unit to the second marker according to the specification of the center point of the second marker It is a directivity control system which calculates the distance of the horizontal direction between parts, and the height from the horizontal surface of the sound collection part.
- the circular calibration floor marker MAK2 is installed at a position on the horizontal surface directly below the omnidirectional microphone array device 2. Therefore, when the center point O of the calibration floor marker MAK2 displayed on the display device 36 is designated by the user's finger FG, the sound collection direction calculation unit 34 of the directivity control device 3 may use, for example, the PTZ camera device 1
- the distance L CO from the PTZ camera device 1 to the calibration floor marker MAK 2 can be easily calculated using the focus function of the above-mentioned, and furthermore, the horizontal direction between the PTZ camera device 1 and the omnidirectional microphone array device 2
- the distance L CMh can be easily calculated as a calibration parameter.
- a circular second marker having a predetermined radius is installed at a position on the horizontal surface directly below the sound collection unit, and the sound collection direction calculation unit includes the display unit. And the vertical angle from the imaging unit to the center point of the second marker and the circle of the second marker from the imaging unit according to the designation of the center point and the end point on the circumference of the second marker displayed on the screen.
- a horizontal distance between the imaging unit and the sound collecting unit and a height from the horizontal surface of the sound collecting unit using the vertical angle toward the end point on the circumference and the radius of the second marker It is a directivity control system that calculates
- a circular calibration floor marker MAK2 having a predetermined radius is installed at a position on the floor directly below the omnidirectional microphone array device 2. Therefore, even if, for example, the distance from the omnidirectional microphone array device 2 to the calibration floor marker MAK 2 can not be measured, the sound collection direction calculation unit 34 of the directivity control device 3 displays the calibration displayed on the display device 36.
- the sound collection direction calculation unit 34 of the directivity control device 3 displays the calibration displayed on the display device 36.
- the height H M of the omnidirectional microphone array device 2 from the floor can be easily calculated, and further, the horizontal distance L between the PTZ camera device 1 and the omnidirectional microphone array device 2 CMh can be easily calculated as a calibration parameter.
- the calibration marker MAK3 is installed directly below the omnidirectional microphone array device 2 at a position where the height H O from the horizontal surface is constant, and the calibration marker MAK3 and the target sound source position A from the horizontal surface
- the heights H O and H A are identical.
- the sound collection direction calculation unit 34 of the directivity control device 3 can detect, for example, the omnidirectional microphone array device 2 Even if the PTZ camera device 1 can not capture an image of the floor surface directly below the omnidirectional microphone array device 2 because there is an obstacle on the floor directly below, the PTZ camera device 1 and the omnidirectional microphone array device 2 The distance L CMh in the horizontal direction between them and the height L MOv from the calibration marker MAK 3 of the omnidirectional microphone array device 2 can be easily calculated as a calibration parameter.
- the sound collection direction calculation unit 34 of the directivity control device 3 calculates the horizontal angle ⁇ CMh and the vertical angle ⁇ CMv calculated at the time of calibration, the horizontal distance L CMh , the PTZ camera device 1 and the omnidirectional microphone array device 2 Using each height L MOv ' , L MO from the target sound source position A, the predetermined horizontal angle ⁇ MCh, and the horizontal angle ⁇ CAh and the vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A The sound collecting direction coordinate from the omnidirectional microphone array device 2 to the target sound source position A can be easily calculated.
- the sound collection direction calculation unit 34 of the directivity control apparatus 3 even if the height from the floor PTZ camera device 1 and the omnidirectional microphone array apparatus 2 are different, the height H A from the floor of the target sound source position A
- the coordinates of the sound collecting direction from the omnidirectional microphone array device 2 to the target sound source position A can be easily calculated without the user inputting.
- a first marker having a constant distance from the sound collection unit is suspended immediately below the sound collection unit, and the height of the first marker from the horizontal surface, and the object
- the height of the sound source position is the same as the height from the horizontal plane
- the sound collecting direction calculation unit is configured to receive the sound from the imaging unit according to the specification of the sound collecting unit and the first marker displayed on the display unit.
- the horizontal distance between the image pickup unit and the sound collecting unit using the vertical angle toward the sound unit and the vertical angle from the image pickup unit to the first marker, and the purpose of the image pickup unit It is a directivity control system which calculates the height from the sound source position.
- the calibration marker MAK having a constant distance (L MOv ) from the omnidirectional microphone array device 2 is suspended directly below the omnidirectional microphone array device 2, and the calibration marker MAK and the target sound source position Each height H O and H A from the floor of A is identical.
- the sound collection direction calculation unit 34 of the directivity control device 3 may, for example, Even if PTZ camera device 1 can not image the floor surface directly below omnidirectional microphone array device 2 because an obstacle is present on the floor directly below omnidirectional microphone array device 2, omnidirectional direction with PTZ camera device 1
- the horizontal distance L CMh with the microphone array device 2 can be easily calculated as a calibration parameter.
- the sound collection direction calculation unit 34 of the directivity control device 3 calculates the horizontal angle ⁇ CMh calculated at the time of calibration, the horizontal distance L CMh , the PTZ camera device 1 and the floor surface of the omnidirectional microphone array device 2
- the heights H C and H M , the predetermined horizontal angle ⁇ MCh , the horizontal angle ⁇ CAh and the vertical angle ⁇ CAv from the PTZ camera device 1 toward the target sound source position A, and the target sound source position A input by the user
- the sound collecting direction coordinate in the sound collecting direction from the omnidirectional microphone array device 2 to the target sound source position A can be easily calculated using the input value of the height HA from the floor.
- a circular calibration floor marker MAK2 is installed at a position on the floor directly below the omnidirectional microphone array device 2. Therefore, in the sound collection direction calculation unit 34 of the directivity control device 3, two points of the center point O of the floor marker MAK2 for calibration displayed on the display device 36 and the omnidirectional microphone array device 2 are the fingers FG of the user.
- the heights H C and H M from the floor of the PTZ camera device 1 and the omnidirectional microphone array device 2 can be easily calculated, and further, between the PTZ camera device 1 and the omnidirectional microphone array device 2
- the horizontal distance L CMh can be easily calculated as a calibration parameter.
- the heights of the image pickup unit and the sound collection unit from the horizontal plane are different, and the sound collection unit is installed outside the imaging view angle of the image pickup unit.
- the sound collection direction calculation unit is a horizontal angle from the image pickup unit to the sound collection unit, a horizontal distance between the image pickup unit and the sound collection unit, and a distance from the sound collection unit to the image pickup unit Horizontal angle, height of the image pickup unit from the horizontal surface, height of the sound collection unit from the horizontal surface, horizontal angle and vertical angle from the image pickup unit to the target sound source position, horizontal surface of the target sound source position
- a directional control system for calculating a horizontal angle and a vertical angle in a sound collecting direction from the sound collecting unit toward the target sound source position using an input value of a height from the to.
- the heights of the PTZ camera device 1 and the omnidirectional microphone array device 2 from the floor are different, and the omnidirectional microphone array device 2 exists outside the imaging view angle of the PTZ camera device 1, ie, the omnidirectional microphone
- the array device 2 is installed so as not to be displayed on the display device 36.
- the height H A from the floor of the target sound source position A since the desired value of the user becomes available, thereby improving the degree of freedom of selection of the height H A from the floor of the target sound source position A.
- a first marker having a constant distance from the sound collection unit is suspended immediately below the sound collection unit, and the first marker is positioned at the horizontal plane directly below the sound collection unit.
- a second marker having a circular shape is installed, and the sound collecting direction calculation unit is configured to set the first marker from the imaging unit according to the designation of the center point of the first marker and the second marker displayed on the display unit.
- the sound collecting direction calculation unit is configured to set the first marker from the imaging unit according to the designation of the center point of the first marker and the second marker displayed on the display unit.
- a circular second marker is placed at a position on the horizontal surface directly below the sound collection unit, and the sound collection direction calculation unit is displayed on the display unit.
- the imaging unit and the sound collection unit using the vertical angle from the imaging unit to the second marker and the distance from the imaging unit to the second marker according to the specification of the center point of the second marker It is a directivity control system which calculates the distance of the horizontal direction between parts, and the height from the horizontal surface of the imaging part.
- One embodiment of the present invention is a directivity control method in a directivity control system including an imaging unit that images an object in a predetermined sound collection area, and a sound collection unit that collects sound of the sound collection area.
- a step of displaying image data of the object picked up by the imaging unit; a step of accepting designation of an arbitrary position in the image data of the displayed object; and an arbitrary step in image data of the object Calculating the sound collecting direction from the sound collecting unit toward the target sound source position corresponding to the designated position of the image data according to the specification of the position; and the sound collecting unit in the calculated sound collecting direction Forming the sound collection directivity of the collected sound.
- the designated position A ′ corresponding to the target sound source position A is designated on the display device 36 on which the captured image data of the PTZ camera device 1 is displayed.
- input parameters for example, distance, direction
- the directivity control device 3 collects the sound in the sound collecting direction toward the target sound source position A corresponding to the designated position A ′ in the captured image data of the PTZ camera device 1
- the directivity can be easily formed, and the voice data of the voice in the sound collection direction collected with high accuracy can be easily obtained.
- this modified example a modified example of the calibration method in the sound collection system 10 of the eleventh embodiment (hereinafter, referred to as “this modified example”) will be described with reference to FIGS. 82 to 91.
- this modified example unlike the method of calculating the sound collection direction coordinate described above, the front direction (0 degree direction) of each horizontal angle in the PTZ camera device 1 and the omnidirectional microphone array device 2 does not face each other, and further different directions Point to
- the horizontal angle ⁇ MCh in the direction in which the PTZ camera device 1 is viewed from the omnidirectional microphone array device 2 is zero.
- the horizontal angle ⁇ MCh indicating the shift amount in the X-axis direction (horizontal angle 0 ° direction, see FIG. 82) of the omnidirectional microphone array device 2 is 0, in the present modification below, this horizontal The case where the angle ⁇ MCh is not zero will be described.
- a method of calculating a calibration parameter will be described, and a method of calculating a sound collection direction coordinate using the calibration parameter is the same as that in the above-described eleventh embodiment, and thus the description thereof will be omitted.
- the calibration parameters in the eleventh calibration method are the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2, the height H C of the PTZ camera device 1 from the floor, and the total
- the horizontal angle ⁇ MCh indicates the height H M of the azimuth microphone array device 2 from the floor and the displacement of the omnidirectional microphone array device 2 in the X-axis direction.
- FIG. 82A is a plan view of a calibration floor marker MAK4 used in the eleventh calibration method.
- FIG. 82B is an enlarged screen of the point O and the point X by the focusing function of the PTZ camera device 1.
- the calibration floor marker MAK4 shown in FIG. 82A is configured using, for example, a sheet material.
- the center point O of the sheet material is set to correspond to the position directly below the center position of the omnidirectional microphone array device 2 in the vertical direction.
- the point X is provided to coincide with the front direction (0 degree direction) of the horizontal angle of the omnidirectional microphone array device 2.
- the distance from the center point O of the calibration floor marker MAK 4 to the point X (circle radius R 0 ) is a known (constant) value.
- the PTZ camera device 1 focuses on the point O of the calibration floor marker MAK 4 by the focus function of the PTZ camera device 1 itself according to the user's input operation (for example, the touch operation of the finger FG on the display device 36). Then, the distance L CO from the PTZ camera device 1 to the point O, the horizontal angle ⁇ COh and the vertical angle ⁇ COv are calculated. Further, the PTZ camera device 1 focuses on the point X of the calibration floor marker MAK 4 by the focus function of the PTZ camera device 1 itself according to the user's input operation (for example, the touch operation of the finger FG on the display device 36). By doing this, the distance L CX from the PTZ camera device 1 to the point X, the horizontal angle ⁇ CXh and the vertical angle ⁇ CXv are calculated. The calculation result of the PTZ camera device 1 is transmitted to the directivity control device 3.
- FIG. 83A shows a positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2 and the floor marker MAK4 for calibration in the eleventh calibration method.
- FIG. 83 (B) is a horizontal direction top view of FIG. 83 (A).
- FIG. 83 (C) is a cross-sectional view taken along the line K-K 'of FIG. 83 (B).
- FIG. 84A shows a positional relationship between the PTZ camera device 1, the omnidirectional microphone array device 2 and the floor marker MAK4 for calibration in the eleventh calibration method.
- FIG. 84 (B) is a horizontal direction top view of FIG. 84 (A).
- FIG. 84C is a cross-sectional view taken along line L-L 'in FIG.
- the sound collection direction calculation unit 34 calculates the distance L CO from the PTZ camera device 1 to the central point O of the calibration floor marker MAK 4 at ⁇ COM shown in FIG.
- the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 is calculated according to the equation (144) using the vertical angle ⁇ COv toward the center point O of the floor setting marker MAK 4 according to equation (144).
- the height H C of the camera device 1 from the floor is calculated according to the equation (145).
- the calibration floor marker MAK4 is installed on the floor directly below the predetermined position of the omnidirectional microphone array device 2 in the vertical direction, so that it is between the PTZ camera device 1 and the calibration floor marker MAK4.
- the sound collection direction calculating unit 34 the ⁇ CXM shown in FIG. 84 (C), the distance L CX from PTZ camera device 1 to the point X calibration floor standing marker MAK4, calibration from PTZ camera device 1
- the horizontal distance L CXh from the PTZ camera device 1 to the point X of the calibration floor marker MAK 4 is calculated according to Formula (146) using the vertical angle ⁇ CXv toward the point X of the floor marker MAK 4 for floor use .
- the sound collection direction calculation unit 34 is an omnidirectional microphone array device using calculation results of Equations (144) to (146) according to the cosine theorem in ⁇ CXM shown in FIG. 83 (B) or FIG. 84 (B)
- a horizontal angle ⁇ MCh indicating the amount of deviation in the X-axis direction of 2 is calculated according to equation (147).
- the directivity control device 3 installs a plurality of PTZ camera devices 1 for one omnidirectional microphone array device 2 Also in the case of the system configuration of the sound collection system, calibration parameters between each PTZ camera device 1 and the omnidirectional microphone array device 2 can be similarly calculated. Therefore, the omnidirectional microphone array apparatus 2 is installed toward the PTZ camera apparatus 1, and the installation restriction of the omnidirectional microphone array apparatus 2 is eliminated, and the installation of the omnidirectional microphone array apparatus 2 is facilitated.
- the heights H C and H M from the horizontal surface (floor surface) of the PTZ camera device 1 and the omnidirectional microphone array device 2 are equal, and H C and H M are known.
- the calibration parameters in the twelfth calibration method indicate the horizontal distance L CMh between the PTZ camera device 1 and the omnidirectional microphone array device 2 and the amount of deviation of the omnidirectional microphone array device 2 in the X-axis direction. It is a horizontal angle ⁇ MCh .
- the height H C from the floor of the PTZ camera device 1 is known, it may be specifically calculated by the twelfth calibration method. In the description of the twelfth calibration method, the description of the same contents as the description of the eleventh calibration method will be omitted or simplified, and different contents will be described.
Abstract
Description
本発明によれば、カメラ装置の撮像方向座標とマイクアレイ装置の収音方向座標との各水平角における0°方向と両者を結ぶ互いの基準方向との間の角度を示す水平偏差角を算出し、マイクアレイ装置に、カメラ装置の撮像方向に存在している被写体の会話音声を適正に収音させることができる。
(指向性制御システムのシステム構成)
図1(A)及び図1(B)は、本実施形態の指向性制御システム10,10Aのシステム構成を示すブロック図である。図1(A)に示す指向性制御システム10は、少なくとも1つのカメラ装置11,…,1nと、マイクアレイ装置(又は全方位マイクアレイ装置)2と、指向性制御装置3とを含む構成である。以下、nはカメラ装置の台数を表し、1以上の整数である。カメラ装置11~1nと、マイクアレイ装置2と、指向性制御装置3とは、ネットワークNW1を介して相互に接続されている。
またこれまでの説明では、図6に示すステップST12の動作がカメラ装置11で行われているが、カメラ装置11は、パン方向の角度情報とチルト方向の角度とズーム情報とを指向性制御装置3へ送り、指向性制御装置3が、カメラ装置11から収音範囲中心位置Aまでの距離、水平角及び垂直角(LCA,θCAh,θCAv)を算出しても良く、以下の各実施形態においても同様である。
図80は、部屋の壁面に取り付けられたカメラCXと、部屋の天井面に取り付けられた全方位マイクアレイ装置MXと、音源位置Pとの位置関係を示す説明図である。図81は、PTZカメラ装置1のカメラ座標系から変換した全方位マイクアレイ装置2のマイク座標系を用いて算出される、全方位マイクアレイ装置2から音源Pの位置に向かう水平角及び垂直角の説明図である。
また、PTZカメラ装置1が直交座標に変換後のデータを送るように説明したが、カメラ座標系のデータを指向性制御装置3に送信して、指向性制御装置3で座標変換しても良い。
次に、指向性制御装置3の信号処理部33がマイクアレイ装置2の収音方向を示す座標(θMAh,θMAv)を算出する方法について、図7~図18を参照して詳細に説明する。ここでは合計4種類の算出方法を説明する。
第1の算出方法では、カメラ装置11の光軸CXの方向に基準点Oを設ける。
(1)カメラ装置11とマイクアレイ装置2との間の距離LCMの水平成分(方向)の距離LCMhと、
(2)カメラ装置11から基準点Oまでの距離LCO及び俯角θCOと、
(3)マイクアレイ装置2から基準点Oとの距離LMO及び俯角θMOと、
(4)カメラ装置11、マイクアレイ装置2、基準点Oの水平面からの各高さHC,HM,HOと、
(5)カメラ装置11から収音範囲中心位置Aまでの水平角θCAh及び垂直角θCAvと、
(6)収音範囲中心位置Aの水平面からの高さHAと、を基に、
マイクアレイ装置2の収音方向を示す座標(θMAh,θMAv)を算出する。
(1)カメラ装置11とマイクアレイ装置2との間の距離LCMの水平成分(方向)の距離LCMhと、
(2)カメラ装置11から基準点Oまでの距離LCO及び俯角θCOと、
(3)マイクアレイ装置2から基準点Oとの距離LMO及び俯角θMOと、
(4)カメラ装置11、マイクアレイ装置2、基準点Oの水平面からの各高さHC,HM,HOである。
(2)カメラ装置11から基準点Oまでの距離LCO及び俯角θCOは、例えばユーザが基準点Oの位置においてレーザ距離計をカメラ装置11に向けることで、簡易に測定できる。
(3)マイクアレイ装置2から基準点Oとの距離LMO及び俯角θMOは、例えばユーザが基準点Oの位置においてレーザ距離計をマイクアレイ装置2に向けることで、簡易に測定できる。
(4)カメラ装置11、マイクアレイ装置2、基準点Oの水平面からの各高さHC,HM,HOは、カメラ装置11、マイクアレイ装置2の各初期設定時に決まる固定値であり、更に、基準点Oの位置が決められた時に定まる固定値である。
(5)カメラ装置11から収音範囲中心位置Aまでの水平角θCAh及び垂直角θCAv、が用いられ、これらはカメラ装置11における公知技術の機能によって取得される。
(6)収音範囲中心位置Aの水平面からの高さHAは、予め決められた固定値であり、例えばユーザの指FGが位置A’を指定する時に、収音範囲中心位置Aの周囲に人物がいる場合には人物の大きさをHAとして選択された値又は入力された値である。又は、ユーザの指FGが位置A’を指定した時に、指向性制御装置3が指定された位置に人物(例えば大人又は子供)がいることを判定した場合に、既定の値(例えば1.5m又は0.8m)が用いられても良い。
第2の算出方法では、カメラ装置11の光軸CXの方向ではなく、マイクアレイ装置2から鉛直下方向に糸STRによってつり下げられたマーカMAKの位置を基準点Oとして設ける。
(1)カメラ装置11からマイクアレイ装置2の鉛直下方向につり下げられたマーカMAKまでの距離LCOと、
(2)マイクアレイ装置2からマーカMAKまでの距離LMOと、
(3)カメラ装置11の光軸CXの方向とカメラ装置11からマーカMAKに向かう方向との間の水平角θCMh(=θCOh)及び垂直角θCOvと、
(4)カメラ装置11から収音範囲中心位置Aまでの距離LCA、水平角θCAh及び垂直角θCAvと、を基に、マイクアレイ装置2の収音方向を示す座標(θMAh,θMAv)を算出する。
(1)カメラ装置11からマイクアレイ装置2の鉛直下方向につり下げられたマーカMAKまでの距離LCOと、
(2)マイクアレイ装置2からマーカMAKまでの距離LMOと、
(3)カメラ装置11の光軸CXの方向とカメラ装置11からマーカMAKに向かう方向との間の水平角θCMh及び垂直角θCOvである。
(2)マイクアレイ装置2からマーカMAKまでの距離LMOは、糸STRの長さに等しい。
(3)カメラ装置11の光軸CXの方向とカメラ装置11からマーカMAKに向かう方向との間の水平角θCMh及び垂直角θCOvは、カメラ装置11の公知技術の機能によって取得される。
(4)カメラ装置11から収音範囲中心位置Aまでの距離LCA、水平角θCAh及び垂直角θCAvは、図6に示すステップST12においてカメラ装置11の公知技術の機能によって取得される。
第3の算出方法では、カメラ装置11の光軸CXの方向に基準点Oとしての音源を設ける。
(1)カメラ装置11とマイクアレイ装置2との間の距離LCMと、
(2)カメラ装置11から光軸方向の所定の音源位置(基準点O)までの距離LCOと、
(3)マイクアレイ装置2から音源位置(基準点O)までの水平角θMOh及び垂直角θMOvと、
(4)カメラ装置11、マイクアレイ装置2及び音源位置(基準点O)の水平面からの各高さHC,HM,HOと、
(5)カメラ装置11から収音範囲中心位置Aまでの距離LCA、水平角θCAh及び垂直角θCAvと、を基に、マイクアレイ装置2の収音方向を示す座標(θMAh,θMAv)を算出する。
(1)カメラ装置11とマイクアレイ装置2との間の距離LCMと、
(2)カメラ装置11から光軸方向の所定の音源位置(基準点O)までの距離LCOと、
(3)マイクアレイ装置2から音源位置(基準点O)までの水平角θMOh及び垂直角θMOvと、
(4)カメラ装置11、マイクアレイ装置2及び音源位置(基準点O)の水平面からの各高さHC,HM,HOである。
(2)カメラ装置11から光軸方向の所定の音源位置(基準点O)までの距離LCOは、例えばユーザが基準点Oの位置においてレーザ距離計をカメラ装置11に向けることで、簡易に測定できる。
(3)マイクアレイ装置2から音源位置(基準点O)までの水平角θMOh及び垂直角θMOvは、マイクアレイ装置2の公知技術の機能(例えば音源検出機能)を用いて測定できる。
(4)カメラ装置11、マイクアレイ装置2、基準点Oの水平面からの各高さHC,HM,HOは、カメラ装置11、マイクアレイ装置2の各初期設定時に決まる固定値であり、更に、基準点Oの位置が決められた時に定まる固定値である。
(5)カメラ装置11から収音範囲中心位置Aまでの距離LCA、水平角θCAh及び垂直角θCAvは、図6に示すステップST12においてカメラ装置11の公知技術の機能によって取得される。
第4の算出方法では、カメラ装置11とマイクアレイ装置2とが専用冶具50を用いて連結して固定され、例えば室内の天井面に設置される(図17又は図18参照)。
(1)専用冶具50の長さと、
(2)カメラ装置11から収音範囲中心位置(点A)までの距離LCA、水平角θCAh及び垂直角θCAvと、を基に、マイクアレイ装置2の収音方向を示す座標(θMAh,θMAv)を算出する。
(1)専用冶具50の長さ、
即ち、どの種類の専用冶具50が用いられるかという情報である。例えば専用冶具50の長さが5mであれば、カメラ装置11とマイクアレイ装置2との間の距離LCMも5mとなり、専用冶具50の長さが10mであれば、カメラ装置11とマイクアレイ装置2との間の距離LCMも10mとなる。
(2)カメラ装置11から収音範囲中心位置Aまでの距離LCA、水平角θCAh及び垂直角θCAvは、図6に示すステップST12においてカメラ装置11の公知技術の機能によって取得される。
第2の実施形態のキャリブレーション方法について、図20(A)、図20(B)及び図20(C)を参照して説明する。図20(A)は、第2の実施形態の収音システム1Zにおけるキャリブレーション方法を表す模式図である。図20(B)は、全方位カメラ装置3zを鉛直方向の下側から見た平面図である。図20(C)は、全方位マイクアレイ装置5を鉛直方向の下側から見た平面図である。
次に、第3の実施形態のキャリブレーション方法について、図21(A)、図21(B)及び図21(C)を参照して説明する。図21(A)は、第3の実施形態の収音システム1Aにおけるキャリブレーション方法を表す模式図である。図21(B)は、全方位カメラ装置3AZを鉛直方向の下側から見た平面図である。図21(C)は、全方位マイクアレイ装置5Aを鉛直方向の下側から見た平面図である。
次に、第4の実施形態のキャリブレーション方法について、図22、図23(A)及び図23(B)を参照して説明する。図22は、第4の実施形態の収音システム1Bにおけるキャリブレーション方法を表す模式図である。図23(A)は、取付部材7に全方位カメラ装置と全方位マイクアレイ装置とが取り付けられた状態を示す平面図である。図23(B)は、図23(A)のE-E断面図である。
次に、第5の実施形態のキャリブレーション方法について、図25(A)、図25(B)、図25(C)及び図26を参照して説明する。図25(A)は、第5の実施形態のキャリブレーション方法において冶具61を全方位マイクアレイ装置5Cに取り付ける様子を示す側面図である。図25(B)は、冶具61の全方位マイクアレイ装置5Cへの取り付けが完了した様子を示す側面図である。図25(C)は、冶具61の全方位マイクアレイ装置5Cへの取り付けが完了した収音システム1Cの外観斜視図である。図26は、全方位カメラ装置3CZの撮像画像80zに冶具61が映った様子を示す図である。
(指向性制御システムのシステム構成)
図27(A)は、キャリブレーション用全方位カメラ装置C1と全方位マイクアレイ装置2とが一体的に取り付けられる場合の本実施形態の指向性制御システム10のシステム概要図である。図27(B)は、全方位マイクアレイ装置2の収音方向の水平角の基準方向とキャリブレーション用全方位カメラ装置C1の撮像方向の水平角の基準方向とが一致するように全方位マイクアレイ装置2が取り付けられる場合の本実施形態の指向性制御システム10Aのシステム概要図である。
次に、指向性制御装置3の水平偏差角算出部34wが第1水平偏差角εCh及び第2水平偏差角εKhを算出する方法について、図31及び図32を参照して詳細に説明する。
(1)ステップST12により測定された全方位カメラ装置11zとキャリブレーション用全方位カメラ装置C1との間の距離LCKと、
(2)ステップST14により測定された第1撮像方向CAX1の水平角及び垂直角(θCAh,θCAv)の座標と、
(3)ステップST14により測定された第2撮像方向CAX2の水平角及び垂直角(θKAh,θKAv)の座標と、
(4)ステップST11の初期設置時において測定された全方位カメラ装置11z、キャリブレーション用全方位カメラ装置C1の水平面からの各高さHC,HKと、
(5)音声収音位置Aの水平面からの高さHAと、を基に、
第1水平偏差角εCh及び第2水平偏差角εKhを算出する。
(指向性制御システムのシステム構成)
図33は、第7の実施形態の指向性制御システム10のシステム構成を示すブロック図である。図33に示す指向性制御システム10は、少なくとも1つのカメラ装置11~1nと、全方位マイクアレイ装置2と、指向性制御装置3と、レコーダ装置4とを含む。nはカメラ装置の台数を表し、1以上の整数である。カメラ装置11~1nと、全方位マイクアレイ装置2と、指向性制御装置3と、レコーダ装置4とは、ネットワークNWを介して互いに接続されている。ネットワークNWは、有線ネットワーク(例えばイントラネット、インターネット)でも良いし、無線ネットワーク(例えば無線LAN(Local Area Network))でも良く、以下の各実施形態においても同様である。
ここで、指向性制御装置3の信号処理部33の収音指向方向算出部34bが全方位マイクアレイ装置2の収音指向座標(θMAh,θMAv)を算出する方法について、図36及び図37を参照して詳細に説明する。
(1)カメラ装置11と全方位マイクアレイ装置2との間の距離LCMと、
(2)カメラ装置11から対象音源位置Aまでの水平角θCAh及び垂直角θCAvと、
(3)カメラ装置11,全方位マイクアレイ装置2の床面BLからの各高さHC,HM(HC=HM)と、
(4)対象音源位置Aの床面BLからの高さHAと、を基に、
全方位マイクアレイ装置2の収音指向座標(θMAh,θMAv)を算出する。
(1)カメラ装置11と全方位マイクアレイ装置2との間の距離LCMと、
(3)カメラ装置11、全方位マイクアレイ装置2の床面BLからの各高さHC,HM(HC=HM)である。
(1)カメラ装置11と全方位マイクアレイ装置2との間の距離LCMは、例えば図35(A)に示すステップST1の初期設置時に測定された固定値である。
(3)カメラ装置11、全方位マイクアレイ装置2の床面BLからの各高さHC,HM(HC=HM)は、例えば図35(A)に示すステップST1の初期設置時に測定された固定値である。また、説明を簡単にするために、カメラ装置11、全方位マイクアレイ装置2の床面BLからの各高さHC,HMは等しいとして説明するが、異なっても良い。
(2)カメラ装置11から対象音源位置Aまでの水平角θCAh及び垂直角θCAvは、図35(B)に示すステップST12においてカメラ装置11の公知技術の機能によって取得される。
(4)対象音源位置Aの床面BLからの高さHAは、指向性制御装置3の信号処理部33の音源高さ判定部34aにより判定された固定値、即ち、図35(A)に示すステップST4において設定ファイルCF1に書き込まれた固定値(所定値)である。
第8の実施形態では、指向性制御装置3Aは、対象音源位置Aの床面BLからの高さの選択肢をディスプレイ装置36に表示させ、例えばユーザにいずれかの選択肢を選択させるように促す。指向性制御装置3Aは、いずれかの選択肢の選択に応じて、対象音源位置Aの床面BLからの高さを判定する。
第9の実施形態では、指向性制御装置3Bは、対象音源位置Aの床面BLからの高さを直接に入力させるための入力フォーム画面WD3をディスプレイ装置36に表示させ、例えばユーザに入力を促す。指向性制御装置3Bは、入力フォーム画面WD3に入力されたデータを、対象音源位置Aの床面BLからの高さHAとして判定する。
第10の実施形態では、指向性制御装置3Cは、ディスプレイ装置36に表示された映像データの表示画面WD4において、例えばユーザの指FGにより、対象物(例えば人物)の周囲の第1指定位置A1’と第2指定位置A1’の鉛直下方向(真下)の床面BL上の第2指定位置A2’とが指定されると、第1指定位置A1’に対応する対象音源位置A1の床面BLからの高さを算出する。指向性制御装置3Cは、算出された高さを、対象音源位置A1の床面BLからの高さHA1として判定する。
ここで、指向性制御装置3Cの信号処理部33Cの音源高さ判定部34aCが対象音源位置A1の床面BLからの高さHA1を算出する方法(以下、「本高さ算出方法」という)について、図44及び図45を参照して詳細に説明する。
(1)カメラ装置11の床面BLからの高さHCと、
(2)カメラ装置11から対象音源位置A1までの距離(水平成分)LCA1h及び角度(垂直角)θCA1vと、
(3)カメラ装置11から床面BL上の位置A2までの距離(水平成分)LCA2h及び角度(垂直角)θCA2vと、を基に、
指定された第1指定位置A1’に対応する対象音源位置A1の床面BLからの高さHA1を算出する。
以下、本発明に係る指向性制御システム及び指向性制御方法の第11の実施形態として、例えば収音システム及び収音制御方法の実施形態(以下、「本実施形態」という)について、図面を参照して説明する。本実施形態の収音システムは、例えば工場、公共施設(例えば図書館、イベント会場)、又は店舗(例えば小売店、銀行)に設置される監視システム(有人監視システム及び無人監視システムを含む)として用いられる。
図47は、本実施形態の収音システム10のシステム構成を示すブロック図である。図47に示す収音システム10は、PTZ(Pan Tilt Zoom)カメラ装置1と、全方位マイクアレイ装置2と、指向性制御装置3と、レコーダ装置4とを含む構成である。PTZカメラ装置1と、全方位マイクアレイ装置2と、指向性制御装置3と、レコーダ装置4とは、ネットワークNWを介して相互に接続されている。ネットワークNWは、有線ネットワーク(例えばイントラネット、インターネット)でも良いし、無線ネットワーク(例えば無線LAN(Local Area Network))でも良い。図47に示す収音システム10では、説明を簡単にするために、PTZカメラ装置1は1台だけ図示されているが、複数のPTZカメラ装置が含まれる構成でも良い。
次に、指向性制御装置3の収音方向算出部34がキャリブレーション方法毎に異なるキャリブレーションパラメータと収音方向座標(θMAh,θMAv)とを算出する方法について、図50~図79を参照して詳細に説明する。ここでは、合計10通りのキャリブレーション方法、即ち10通りのキャリブレーションパラメータの算出方法を説明し、合計4通りの収音方向座標(θMAh,θMAv)の算出方法を説明する。なお、合計10通りのキャリブレーションパラメータの算出方法のうちいずれかと、合計4通りの収音方向座標(θMAh,θMAv)の算出方法のうちいずれかとが対応している。
先ず、第1のキャリブレーション方法と、第1のキャリブレーション方法によりキャリブレーションパラメータを算出した後における第1の収音方向座標の算出方法について、図50、図51(A)~(C)、図52(A)~(C)及び図53(A)~(C)を参照して説明する。
(1)PTZカメラ装置1から全方位マイクアレイ装置2に向かう水平角θCMhと、
(2)PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhと、
(3)全方位マイクアレイ装置2からPTZカメラ装置1に向かう水平角θMChと、
(4)PTZカメラ装置1から目的音源位置Aに向かう水平角θCAh,垂直角θCAvと、を用いて、全方位マイクアレイ装置2から目的音源位置Aに向かう収音方向の収音方向座標(θMAh,θMAv)を算出する。図52(B),(C)及び図53(B),(C)を参照して、収音方向算出部34における第1の収音方向座標(θMAh,θMAv)の算出方法を具体的に説明する。
(2)PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhは、第1のキャリブレーション方法におけるキャリブレーションパラメータである(数式(84)参照)。
(3)全方位マイクアレイ装置2からPTZカメラ装置1に向かう水平角θMChは、第1のキャリブレーション方法においてPTZカメラ装置1及び全方位マイクアレイ装置2の位置関係から与えられる既定値(ゼロ)である。
(4)PTZカメラ装置1から目的音源位置Aに向かう水平角θCAh,垂直角θCAvは、ディスプレイ装置36に表示された撮像画像データにおいて指定位置A’の指定に応じてPTZカメラ装置1により算出された値が用いられる。
次に、第2のキャリブレーション方法と、第2のキャリブレーション方法によりキャリブレーションパラメータを算出した後における第2の収音方向座標の算出方法について、図54、図55(A)~(C)、図56(A)~(C)及び図57(A)~(C)を参照して説明する。
(1)PTZカメラ装置1から全方位マイクアレイ装置2に向かう水平角θCMhと、
(2)PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhと、
(3)全方位マイクアレイ装置2からPTZカメラ装置1に向かう水平角θMChと、
(4)PTZカメラ装置1,全方位マイクアレイ装置2の床からの各高さHC,HMと、
(5)PTZカメラ装置1から目的音源位置Aに向かう水平角θCAh,垂直角θCAvと、
(6)ユーザにより入力された、目的音源位置Aの床からの高さHAと、
を用いて、全方位マイクアレイ装置2から目的音源位置Aに向かう収音方向の収音方向座標(θMAh,θMAv)を算出する。図56(B),(C)及び図57(B),(C)を参照して、収音方向算出部34における第2の収音方向座標(θMAh,θMAv)の算出方法を具体的に説明する。
(2)PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhは、第2のキャリブレーション方法におけるキャリブレーションパラメータである(数式(85)参照)。
(3)全方位マイクアレイ装置2からPTZカメラ装置1に向かう水平角θMChは、第2のキャリブレーション方法においてPTZカメラ装置1及び全方位マイクアレイ装置2の位置関係から与えられる既定値(ゼロ)である。
(4)PTZカメラ装置1,全方位マイクアレイ装置2の床からの各高さHC,HMは、第2の収音方向座標の算出方法では既定値である(HC=HM)。
(5)PTZカメラ装置1から目的音源位置Aに向かう水平角θCAh,垂直角θCAvは、ディスプレイ装置36に表示された撮像画像データにおいて指定位置A’の指定に応じてPTZカメラ装置1により算出された値が用いられる。
(6)目的音源位置Aの床からの高さHAは、ユーザの入力操作によって入力された値である。
次に、第3のキャリブレーション方法と、第3のキャリブレーション方法によりキャリブレーションパラメータを算出した後における第2の収音方向座標の算出方法について、図58及び図59(A)~(C)を参照して説明する。
次に、第4のキャリブレーション方法と、第4のキャリブレーション方法によりキャリブレーションパラメータを算出した後における第2の収音方向座標の算出方法について、図60及び図61(A)~(C)を参照して説明する。
次に、第5のキャリブレーション方法と、第5のキャリブレーション方法によりキャリブレーションパラメータを算出した後における第1の収音方向座標の算出方法について、図62及び図63(A)~(C)を参照して説明する。
次に、第6のキャリブレーション方法と、第6のキャリブレーション方法によりキャリブレーションパラメータを算出した後における第3の収音方向座標の算出方法について、図64、図65(A)~(C)、図66(A)~(C)及び図67(A)~(C)を参照して説明する。
(1)PTZカメラ装置1から全方位マイクアレイ装置2に向かう水平角θCMh,垂直角θCMvと、
(2)PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhと、
(3)全方位マイクアレイ装置2からPTZカメラ装置1に向かう水平角θMChと、
(4)PTZカメラ装置1の目的音源位置Aからの高さLMOv’と、全方位マイクアレイ装置2の目的音源位置Aからの高さLMOと、
(5)PTZカメラ装置1から目的音源位置Aに向かう水平角θCAh,垂直角θCAvと、を用いて、全方位マイクアレイ装置2から目的音源位置Aに向かう収音方向の収音方向座標(θMAh,θMAv)を算出する。図66(B),(C)及び図67(B),(C)を参照して、収音方向算出部34における第3の収音方向座標(θMAh,θMAv)の算出方法を具体的に説明する。
(2)PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhは、第6のキャリブレーション方法におけるキャリブレーションパラメータである(数式(109)参照)。
(3)全方位マイクアレイ装置2からPTZカメラ装置1に向かう水平角θMChは、第6のキャリブレーション方法においてPTZカメラ装置1及び全方位マイクアレイ装置2の位置関係から与えられる既定値(ゼロ)である。
(4)PTZカメラ装置1の目的音源位置Aからの高さLMOv’は、第6のキャリブレーション方法におけるキャリブレーションパラメータであり(数式(110)参照)、全方位マイクアレイ装置2の目的音源位置Aからの高さLMOは既定値である。
(5)PTZカメラ装置1から目的音源位置Aに向かう水平角θCAh,垂直角θCAvは、ディスプレイ装置36に表示された撮像画像データにおいて指定位置A’の指定に応じてPTZカメラ装置1により算出された値が用いられる。
次に、第7のキャリブレーション方法と、第7のキャリブレーション方法によりキャリブレーションパラメータを算出した後における第4の収音方向座標の算出方法について、図68、図69(A)~(C)、図70(A)~(C)及び図71(A)~(C)を参照して説明する。
(1)PTZカメラ装置1から全方位マイクアレイ装置2に向かう水平角θCMhと、
(2)PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhと、
(3)全方位マイクアレイ装置2からPTZカメラ装置1に向かう水平角θMChと、
(4)PTZカメラ装置1,全方位マイクアレイ装置2の床からの各高さHC,HMと、
(5)PTZカメラ装置1から目的音源位置Aに向かう水平角θCAh,垂直角θCAvと、
(6)ユーザにより入力された、目的音源位置Aの床からの高さHAと、
を用いて、全方位マイクアレイ装置2から目的音源位置Aに向かう収音方向の収音方向座標(θMAh,θMAv)を算出する。図70(B),(C)及び図71(B),(C)を参照して、収音方向算出部34における第4の収音方向座標(θMAh,θMAv)の算出方法を具体的に説明する。
(2)PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhは、第7のキャリブレーション方法におけるキャリブレーションパラメータである(数式(112)参照)。
(3)全方位マイクアレイ装置2からPTZカメラ装置1に向かう水平角θMChは、第7のキャリブレーション方法においてPTZカメラ装置1及び全方位マイクアレイ装置2の位置関係から与えられる既定値(ゼロ)である。
(4)PTZカメラ装置1,全方位マイクアレイ装置2の床からの各高さHC,HMは、第7のキャリブレーション方法におけるキャリブレーションパラメータである(数式(119),(121)参照)。
(5)PTZカメラ装置1から目的音源位置Aに向かう水平角θCAh,垂直角θCAvは、ディスプレイ装置36に表示された撮像画像データにおいて指定位置A’の指定に応じてPTZカメラ装置1により算出された値が用いられる。
(6)目的音源位置Aの水平面(例えば床)からの高さHAは、ユーザの入力操作によって入力された値である。
次に、第8のキャリブレーション方法と、第8のキャリブレーション方法によりキャリブレーションパラメータを算出した後における第4の収音方向座標の算出方法について、図72及び図73(A)~(C)を参照して説明する。
次に、第9のキャリブレーション方法と、第9のキャリブレーション方法によりキャリブレーションパラメータを算出した後における第4の収音方向座標の算出方法について、図74及び図75(A)~(C)を参照して説明する。
最後に、第10のキャリブレーション方法と、第10のキャリブレーション方法によりキャリブレーションパラメータを算出した後における第4の収音方向座標の算出方法について、図76、図77(A)~(C)、図78(A)~(C)及び図79(A)~(C)を参照して説明する。
(1)PTZカメラ装置1から全方位マイクアレイ装置2に向かう水平角θCMhと、
(2)PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhと、
(3)全方位マイクアレイ装置2からPTZカメラ装置1に向かう水平角θMChと、
(4)PTZカメラ装置1,全方位マイクアレイ装置2の床からの各高さHC,HMと、
(5)PTZカメラ装置1から目的音源位置Aに向かう水平角θCAh,垂直角θCAvと、
(6)ユーザにより入力された、目的音源位置Aの床からの高さHAと、
を用いて、全方位マイクアレイ装置2から目的音源位置Aに向かう収音方向の収音方向座標(θMAh,θMAv)を算出する。図78(B),(C)及び図79(B),(C)を参照して、収音方向算出部34における収音方向座標(θMAh,θMAv)の算出方法を具体的に説明する。
(2)PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhは、第10のキャリブレーション方法におけるキャリブレーションパラメータである(数式(134)参照)。
(3)全方位マイクアレイ装置2からPTZカメラ装置1に向かう水平角θMChは、第10のキャリブレーション方法においてPTZカメラ装置1及び全方位マイクアレイ装置2の位置関係から与えられる既定値(ゼロ)である。
(4)PTZカメラ装置1,全方位マイクアレイ装置2の床からの各高さHC,HMは、第10のキャリブレーション方法におけるキャリブレーションパラメータである(数式(135),(137)参照)。
(5)PTZカメラ装置1から目的音源位置Aに向かう水平角θCAh,垂直角θCAvは、ディスプレイ装置36に表示された撮像画像データにおいて指定位置A’の指定に応じてPTZカメラ装置1により算出された値が用いられる。
(6)目的音源位置Aの床からの高さHAは、ユーザの入力操作によって入力された値である。
次に、第11の実施形態の収音システム10におけるキャリブレーション方法の変形例(以下、「本変形例」という)について、図82~図91を参照して説明する。本変形例では、上述した収音方向座標の算出方法とは異なり、PTZカメラ装置1及び全方位マイクアレイ装置2における各水平角の正面方向(0度方向)が対向せず、更に、異なる方向を指す。
第11のキャリブレーション方法では、PTZカメラ装置1及び全方位マイクアレイ装置2の水平面(床面)からの高さHC,HMは等しいが、HC及びHMは既知でないとする。第11のキャリブレーション方法におけるキャリブレーションパラメータは、PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhと、PTZカメラ装置1の床からの高さHCと、全方位マイクアレイ装置2の床からの高さHMと、全方位マイクアレイ装置2のX軸方向のずれ量を示す水平角θMChである。
第12のキャリブレーション方法では、PTZカメラ装置1及び全方位マイクアレイ装置2の水平面(床面)からの高さHC,HMが等しく、HC及びHMは既知とする。第12のキャリブレーション方法におけるキャリブレーションパラメータは、PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhと、全方位マイクアレイ装置2のX軸方向のずれ量を示す水平角θMChである。なお、PTZカメラ装置1の床からの高さHCは既知であるが、第12のキャリブレーション方法により具体的に算出されても良い。なお、第12のキャリブレーション方法の説明では、第11のキャリブレーション方法の説明と同一の内容の説明は省略又は簡略化し、異なる内容について説明する。
第13のキャリブレーション方法では、PTZカメラ装置1及び全方位マイクアレイ装置2の水平面(床面)からの高さHC,HMは等しいが、HC及びHMは既知でないとする。第13のキャリブレーション方法におけるキャリブレーションパラメータは、PTZカメラ装置1と全方位マイクアレイ装置2との間の水平方向の距離LCMhと、PTZカメラ装置1の床からの高さHCと、全方位マイクアレイ装置2の床からの高さHMと、全方位マイクアレイ装置2のX軸方向のずれ量を示す水平角θMChである。なお、第13のキャリブレーション方法の説明では、第11のキャリブレーション方法の説明と同一の内容の説明は省略又は簡略化し、異なる内容について説明する。
1Z、1A、1B、1C 収音システム
2、2A、2B、2C、2D マイクアレイ装置(全方位マイクアレイ装置)
3、3A、3B、3C 指向性制御装置
3z、3A、3C、3D 全方位カメラ装置
4 レコーダ(レコーダ装置)
5、5A、5B、5C 全方位マイクアレイ装置
7 取付部材
7a、7b 係止片
7c、7d 孔部
7e ビス孔
8 天井面
10、10A 指向性制御システム(収音システム)
11、11n カメラ装置
11y キー
11z 全方位カメラ装置
13 開口部
15 キー溝
17 筐体
18 マイクロホンユニット
21z、23z マーカ
26 加算部
31、31A 通信部
32 操作部
33、33A、33B、33C 信号処理部
34 収音方向算出部
34a 音源高さ判定部
34b 収音指向方向算出部
34c 出力制御部
34w 水平偏差角算出部
34x 座標算出部
34z 座標変換処理部
35 出力制御部
36 ディスプレイ装置
37 スピーカ装置
38、38A、38B メモリ
61 冶具
63 目印
61A 冶具画像
63A 目印画像
71 係合孔
221、222、223、22(n-1)、22n マイクロホン
241、242、243、24(n-1)、24n A/D変換器
251、252、253、25(n-1)、25n 遅延器
C1 キャリブレーション用全方位カメラ装置
CF1、CF2、CF3 設定ファイル
Claims (43)
- 映像を撮像する少なくとも1つの撮像部と、
音声を収音する少なくとも1つの収音部と、
前記少なくとも1つの撮像部により撮像された映像データを表示する表示部と、
表示された前記映像データの任意の位置の指定に応じて、前記収音部から、指定された前記映像データの位置に対応する音声位置に向かう収音方向を算出する収音方向算出部と、を備え、
前記収音方向算出部は、
所定の収音方向算出パラメータと、
前記少なくとも1つの撮像部から、前記映像データの指定位置に対応する音声位置に向かう方向と、を用いて、
前記収音部から、前記映像データの指定位置に対応する音声位置に向かう収音方向を算出する、
指向性制御システム。 - 請求項1に記載の指向性制御システムであって、
前記収音方向算出部により算出された前記収音方向に、前記収音部により収音された音声の収音指向性を形成する出力制御部、を更に備える、
指向性制御システム。 - 請求項1に記載の指向性制御システムであって、
前記収音方向算出部は、
前記少なくとも1つの撮像部から、前記映像データの指定位置に対応する音声位置に向かう方向として、
前記少なくとも1つの撮像部から前記音声位置までの水平角及び垂直角と、
前記少なくとも1つの撮像部から前記音声位置までの距離若しくは前記音声位置の所定の基準面からの高さと、を用い、
前記収音部から、前記映像データの指定位置に対応する音声位置に向かう収音方向として、
前記収音部から前記音声位置までの水平角及び垂直角を算出する、
指向性制御システム。 - 請求項3に記載の指向性制御システムであって、
前記少なくとも1つの撮像部から前記音声位置までの水平角及び垂直角は、
前記表示部に表示された前記映像データの指定位置を基にして算出される、
指向性制御システム。 - 請求項1に記載の指向性制御システムであって、
前記収音方向算出部は、
前記少なくとも1つの撮像部と前記収音部との間の水平方向の距離と、
前記少なくとも1つの撮像部から光軸方向に存在する所定の基準位置までの距離及び俯角と、
前記収音部から前記所定の基準位置までの距離及び俯角と、
前記少なくとも1つの撮像部、前記収音部及び前記所定の基準位置の水平面からの各高さと、
前記少なくとも1つの撮像部から前記音声位置までの水平角及び垂直角と、
前記音声位置の水平面からの高さと、を基に、
前記収音部から前記音声位置に向かう方向の水平角及び垂直角を前記収音方向として算出する、
指向性制御システム。 - 請求項1に記載の指向性制御システムであって、
前記収音方向算出部は、
前記少なくとも1つの撮像部から前記収音部の鉛直方向につり下げられたマーカまでの距離と、
前記収音部から前記マーカまでの距離と、
前記少なくとも1つの撮像部の光軸方向と前記少なくとも1つの撮像部から前記マーカに向かう方向との間の水平角及び垂直角と、
前記少なくとも1つの撮像部から前記音声位置までの距離、水平角及び垂直角と、を基に、
前記収音部から前記音声位置に向かう水平角及び垂直角を前記収音方向として算出する、
指向性制御システム。 - 請求項1に記載の指向性制御システムであって、
前記収音方向算出部は、
前記少なくとも1つの撮像部と前記収音部との間の距離と、
前記少なくとも1つの撮像部から光軸方向に存在する所定の音源位置までの距離と、
前記収音部から前記所定の音源位置までの水平角及び垂直角と、
前記少なくとも1つの撮像部、前記収音部及び前記所定の音源位置の水平面からの各高さと、
前記少なくとも1つの撮像部から前記音声位置までの距離、水平角及び垂直角と、を基に、
前記収音部から前記音声位置に向かう水平角及び垂直角を前記収音方向として算出する、
指向性制御システム。 - 請求項1に記載の指向性制御システムであって、
前記少なくとも1つの撮像部と前記収音部とは、所定の長さを有する冶具を用いて所定の水平角及び垂直角にて連結して設置され、
前記収音方向算出部は、
前記冶具の長さ、前記所定の水平角及び垂直角と、
前記少なくとも1つの撮像部から前記音声位置までの距離、水平角及び垂直角と、を基に、
前記収音部から前記音声位置に向かう水平角及び垂直角を前記収音方向として算出する、
指向性制御システム。 - 請求項1~8のうちいずれか一項に記載の指向性制御システムであって、
前記少なくとも1つの撮像部と前記収音部とが、室内の天井に配置され、水平面からの高さが同一である、
指向性制御システム。 - 映像を撮像する少なくとも1つの撮像部と音声を収音する少なくとも1つの収音部とを含む指向性制御システムにおける指向性制御方法であって、
前記少なくとも1つの撮像部により撮像された映像データを画面に表示するステップと、
前記画面に表示された前記映像データの任意の位置の指定を受け付けるステップと、
前記収音部から、指定された前記映像データの位置に対応する音声位置に向かう収音方向を算出するステップと、を有し、
前記収音部から、前記映像データの指定位置に対応する音声位置に向かう収音方向は、
所定の収音方向算出パラメータと、前記少なくとも1つの撮像部から、前記映像データの指定位置に対応する音声位置に向かう方向とを用いて算出される、
指向性制御方法。 - 請求項10に記載の指向性制御方法であって、
算出された前記収音方向に、前記収音部により収音された音声の収音指向性を形成するステップ、を更に有する、
指向性制御方法。 - 所定の撮像範囲の映像を撮像するカメラ装置と、前記カメラ装置の撮像範囲の音声を収音するマイクアレイ装置とを同軸上に位置合わせする工程と、
前記マイクアレイ装置の筐体中心に形成された開口部の周縁部に、前記カメラ装置を取り付ける工程と、
前記開口部の内側への前記カメラ装置の取り付けにより、前記同軸に対して直交する平面における、前記カメラ装置及び前記マイクアレイ装置の各水平角の基準方向を一致させる工程と、を有する、
キャリブレーション方法。 - 請求項12に記載のキャリブレーション方法であって、
前記一致させる工程では、
前記開口部の周縁部に形成された係止溝に前記カメラ装置の筐体外周に形成された係止部材を嵌合させることで、前記マイクアレイ装置の水平角の基準方向と前記カメラ装置の水平角の基準方向とを一致させる、
キャリブレーション方法。 - 請求項12に記載のキャリブレーション方法であって、
前記一致させる工程では、
前記開口部の周縁部に付加された第1マーカ部と前記カメラ装置の筐体外周に付加された第2マーカ部とを対向させることで、前記マイクアレイ装置の水平角の基準方向と前記カメラ装置の水平角の基準方向とを一致させる、
キャリブレーション方法。 - 請求項12に記載のキャリブレーション方法であって、
前記取り付ける工程では、
前記カメラ装置の筐体が、所定の取付具に取り付けられ、
前記マイクアレイ装置の筐体が、前記所定の取付具に取り付けられた前記カメラ装置の筐体を前記開口部の内側に嵌め入れるように挿通して前記所定の取付具に取り付けられる、
キャリブレーション方法。 - 所定の撮像範囲の映像を撮像するカメラ装置と、前記カメラ装置の撮像範囲の音声を収音するマイクアレイ装置とを同軸上に位置合わせする工程と、
前記マイクアレイ装置の筐体中心に形成された開口部の周縁部に、前記カメラ装置を取り付ける工程と、
前記マイクアレイ装置の水平角の基準方向を表す冶具を、前記マイクアレイ装置の筐体の対向する両端部に取り付ける工程と、
前記カメラ装置により、前記冶具を撮像する工程と、
前記冶具の撮影画像を基に、前記カメラ装置の水平角の基準方向と前記マイクアレイ装置の水平角の基準方向とのズレ量を算出する工程と、
算出された前記ズレ量を用いて、前記マイクアレイ装置の収音方向の水平角を調整することで、前記同軸に対して直交する平面における、前記カメラ装置及び前記マイクアレイ装置の各水平角の基準方向を一致させる工程と、を有する、
キャリブレーション方法。 - 被写体の画像を撮像する第1撮像部と、
前記被写体の画像を撮像する第2撮像部と、
前記被写体の音声を収音する収音部と、
前記第1撮像部により撮像された画像データを表示する表示部と、
表示された前記画像データの任意の位置の指定に応じて、前記第1撮像部から前記画像データの指定位置に対応する音声位置に向かう第1撮像方向の水平角の第1基準方向からの第1水平偏差角と、前記第2撮像部から前記音声位置に向かう第2撮像方向の水平角の第2基準方向からの第2水平偏差角とを算出する偏差量算出部と、を備える、
指向性制御システム。 - 請求項17に記載の指向性制御システムであって、
前記偏差量算出部は、
前記第1撮像部から前記第2撮像部までの距離と、
前記第1撮像部から前記音声位置に向かう水平角及び垂直角と、
前記第2撮像部から前記音声位置に向かう水平角及び垂直角と、
前記第1撮像部の水平面からの高さと、
前記第2撮像部の前記水平面からの高さと、
前記音声位置の前記水平面からの高さと、を基に、
前記第1水平偏差角及び前記第2水平偏差角を算出する、
指向性制御システム。 - 請求項17に記載の指向性制御システムであって、
前記第1水平偏差角及び前記第2水平偏差角を基に、前記収音部から前記音声位置に向かう方向の水平角及び垂直角を、前記被写体の音声を前記収音部に収音させる収音方向を示す座標として算出する座標算出部、を更に備える、
指向性制御システム。 - 請求項19に記載の指向性制御システムであって、
算出された前記収音方向を示す座標に対応する前記収音方向に、音声データの収音指向性を形成する出力制御部、を更に備える、
指向性制御システム。 - 請求項17に記載の指向性制御システムであって、
前記収音部の筐体中心に円柱状の開口部が形成され、
前記収音部と前記第2撮像部とは、前記第2撮像部が前記開口部の内周空間に嵌め入れられることで一体として形成される、
指向性制御システム。 - 第1撮像部と、第2撮像部と、収音部とを含む指向性制御システムにおける水平偏差角算出方法であって、
前記第1撮像部において、被写体の画像を撮像するステップと、
前記第2撮像部において、前記被写体の画像を撮像するステップと、
前記収音部において、前記被写体の音声を収音するステップと、
前記第1撮像部により撮像された画像データを表示部に表示するステップと、
前記表示部に表示された前記画像データの任意の位置の指定に応じて、前記第1撮像部から前記画像データの指定位置に対応する音声位置に向かう第1撮像方向の水平角の第1基準方向からの第1水平偏差角と、前記第2撮像部から前記音声位置に向かう第2撮像方向の水平角の第2基準方向からの第2水平偏差角とを算出するステップと、を有する、
水平偏差角算出方法。 - 映像を撮像する少なくとも1つの撮像部と、
音声を収音する収音部と、
前記少なくとも1つの撮像部により撮像された映像データを表示する表示部と、
前記映像データの位置指定に応じて、前記映像データの指定位置に対応する音声位置の基準面からの高さを判定する高さ判定部と、
前記音声位置の基準面からの高さを基に、前記収音部から前記音声位置に向かう収音方向を算出する収音方向算出部と、
算出された前記収音方向に、前記音声の収音指向性を形成する制御部と、を備える、
指向性制御システム。 - 請求項23に記載の指向性制御システムであって、
前記映像データの指定位置に対応する音声位置と、前記音声位置の基準面からの高さとが予め対応付けられた第1設定データを記憶する記憶部を更に備え、
前記高さ判定部は、前記第1設定データを基に、指定された前記指定位置に対応する音声位置の基準面からの高さを判定する、
指向性制御システム。 - 請求項23に記載の指向性制御システムであって、
前記音声位置の基準面からの高さとして、複数種類の高さデータを含む第2設定データを記憶する記憶部を更に備え、
前記制御部は、前記音声位置の基準面からの高さとして、前記第2設定データの前記複数種類の高さデータの選択肢を前記表示部に表示させ、
前記高さ判定部は、表示された前記複数種類の高さデータの選択肢のうちいずれかの選択に応じて、指定された前記指定位置に対応する音声位置の基準面からの高さを判定する、
指向性制御システム。 - 請求項23に記載の指向性制御システムであって、
前記制御部は、前記音声位置の基準面からの高さの入力フォームを前記表示部に表示させ、
前記高さ判定部は、表示された前記入力フォームへの前記音声位置の基準面からの高さの入力に応じて、指定された前記指定位置に対応する音声位置の基準面からの高さを判定する、
指向性制御システム。 - 請求項23に記載の指向性制御システムであって、
前記高さ判定部は、表示された前記映像データの第1指定位置と前記第1指定位置に対応する前記基準面上の第2指定位置との指定に応じて、指定された前記第1指定位置に対応する前記音声位置の基準面からの高さを算出する、
指向性制御システム。 - 請求項27に記載の指向性制御システムであって、
前記高さ判定部は、
前記少なくとも1つの撮像部の前記基準面からの高さと、
前記少なくとも1つの撮像部から前記第1指定位置に対応する前記音声位置までの距離及び角度と、
前記少なくとも1つの撮像部から前記第2指定位置に対応する前記基準面の位置までの距離及び角度と、を基に、
指定された前記第1指定位置に対応する前記音声位置の基準面からの高さを算出する、
指向性制御システム。 - 映像を撮像する少なくとも1つの撮像部と、音声を収音する収音部とを含む指向性制御システムにおける指向性制御方法であって、
前記少なくとも1つの撮像部により撮像された映像データを表示するステップと、
前記映像データの位置指定に応じて、指定された前記映像データの指定位置に対応する対象音源位置の基準面からの高さを判定するステップと、
前記対象音源位置の基準面からの高さを基に、前記収音部から前記対象音源位置に向かう収音指向方向を算出するステップと、
算出された前記収音指向方向に、前記音声の収音指向性を前記収音部に形成させるステップと、を有する、
指向性制御方法。 - 請求項1に記載の指向性制御システムであって、
前記撮像部及び前記収音部の水平面からの各高さは同一であり、
前記収音方向算出部は、
前記撮像部から前記収音部に向かう水平角と、
前記撮像部と前記収音部との間の水平方向の距離と、
前記収音部から前記撮像部に向かう水平角と、
前記撮像部から前記音声位置に向かう水平角及び垂直角と、を用いて、
前記収音部から前記音声位置に向かう収音方向の水平角及び垂直角を算出する、
指向性制御システム。 - 請求項30に記載の指向性制御システムであって、
前記収音部の真下に、前記収音部からの距離が一定の第1マーカが吊り下げられ、
前記第1マーカの前記水平面からの高さと、前記音声位置の前記水平面からの高さとは同一であり、
前記収音方向算出部は、
前記表示部に表示された前記第1マーカの指定に応じて、前記収音部と前記第1マーカとの間の距離を用いて、
前記撮像部と前記収音部との間の水平方向の距離を算出する、
指向性制御システム。 - 請求項1に記載の指向性制御システムであって、
前記撮像部及び前記収音部の水平面からの各高さは同一であり、
前記収音方向算出部は、
前記撮像部から前記収音部に向かう水平角と、
前記撮像部と前記収音部との間の水平方向の距離と、
前記収音部から前記撮像部に向かう水平角と、
前記撮像部及び前記収音部の水平面からの各高さと、
前記撮像部から前記音声位置に向かう水平角及び垂直角と、
前記音声位置の水平面からの高さの入力値と、を用いて、
前記収音部から前記音声位置に向かう収音方向の水平角及び垂直角を算出する、
指向性制御システム。 - 請求項32に記載の指向性制御システムであって、
前記収音部の真下に、前記収音部からの距離が一定の第1マーカが吊り下げられ、
前記収音部の前記水平面からの高さは所定値であり、
前記収音方向算出部は、
前記表示部に表示された前記第1マーカの指定に応じて、前記収音部と前記第1マーカとの間の距離を用いて、
前記撮像部と前記収音部との間の水平方向の距離を算出する、
指向性制御システム。 - 請求項32に記載の指向性制御システムであって、
前記収音部の真下の前記水平面上の位置に、円形状の第2マーカが設置され、
前記収音方向算出部は、
前記表示部に表示された前記第2マーカの中心点の指定に応じて、前記撮像部から前記第2マーカまでの距離と、前記撮像部から前記第2マーカに向かう垂直角と、を用いて、
前記撮像部と前記収音部との間の水平方向の距離と、前記収音部の前記水平面からの高さとを算出する、
指向性制御システム。 - 請求項32に記載の指向性制御システムであって、
前記収音部の真下の前記水平面上の位置に、所定の半径を有する円形状の第2マーカが設置され、
前記収音方向算出部は、
前記表示部に表示された前記第2マーカの中心点及び円周上の端点の指定に応じて、前記撮像部から前記第2マーカの中心点に向かう垂直角と、前記撮像部から前記第2マーカの円周上の端点に向かう垂直角と、前記第2マーカの半径と、を用いて、
前記撮像部と前記収音部との間の水平方向の距離と、前記収音部の前記水平面からの高さとを算出する、
指向性制御システム。 - 請求項30に記載の指向性制御システムであって、
前記収音部の真下に、前記水平面からの高さが一定の第3マーカが設置され、
前記第3マーカの前記水平面からの高さと、前記音声位置の前記水平面からの高さとは同一であり、
前記収音方向算出部は、
前記表示部に表示された前記第3マーカの指定に応じて、前記撮像部から前記第3マーカまでの距離と、前記撮像部から前記第3マーカに向かう垂直角と、を用いて、
前記撮像部と前記収音部との間の水平方向の距離と、前記収音部の前記第3マーカからの高さとを算出する、
指向性制御システム。 - 請求項1に記載の指向性制御システムであって、
前記撮像部及び前記収音部の水平面からの各高さは異なり、
前記収音部は、前記撮像部の撮像視野角内に存在するように設置され、
前記収音方向算出部は、
前記撮像部から前記収音部に向かう水平角及び垂直角と、
前記撮像部と前記収音部との間の水平方向の距離と、
前記収音部から前記撮像部に向かう水平角と、
前記撮像部の前記音声位置からの高さと、前記収音部の前記音声位置からの高さと、
前記撮像部から前記音声位置に向かう水平角及び垂直角と、を用いて、
前記収音部から前記音声位置に向かう収音方向の水平角及び垂直角を算出する、
指向性制御システム。 - 請求項37に記載の指向性制御システムであって、
前記収音部の真下に、前記収音部からの距離が一定の第1マーカが吊り下げられ、
前記第1マーカの前記水平面からの高さと、前記音声位置の前記水平面からの高さとは同一であり、
前記収音方向算出部は、
前記表示部に表示された前記収音部及び前記第1マーカの指定に応じて、前記撮像部から前記収音部に向かう垂直角と、前記撮像部から前記第1マーカに向かう垂直角と、を用いて、
前記撮像部と前記収音部との間の水平方向の距離と、前記撮像部の前記音声位置からの高さとを算出する、
指向性制御システム。 - 請求項1に記載の指向性制御システムであって、
前記撮像部及び前記収音部の水平面からの各高さは異なり、
前記収音部は、前記撮像部の撮像視野角内に存在するように設置され、
前記収音方向算出部は、
前記撮像部から前記収音部に向かう水平角と、
前記撮像部と前記収音部との間の水平方向の距離と、
前記収音部から前記撮像部に向かう水平角と、
前記撮像部の前記水平面からの高さと、前記収音部の前記水平面からの高さと、
前記撮像部から前記音声位置に向かう水平角及び垂直角と、
前記音声位置の水平面からの高さの入力値と、を用いて、
前記収音部から前記音声位置に向かう収音方向の水平角及び垂直角を算出する、
指向性制御システム。 - 請求項39に記載の指向性制御システムであって、
前記収音部の真下の前記水平面上の位置に、円形状の第2マーカが設置され、
前記収音方向算出部は、
前記表示部に表示された前記収音部及び前記第2マーカの中心点の指定に応じて、前記撮像部から前記収音部に向かう垂直角と、前記撮像部から前記第2マーカに向かう垂直角と、前記撮像部から前記第2マーカまでの距離と、を用いて、
前記撮像部と前記収音部との間の水平方向の距離と、前記撮像部の前記水平面からの高さと、前記収音部の前記水平面からの高さとを算出する、
指向性制御システム。 - 請求項1に記載の指向性制御システムであって、
前記撮像部及び前記収音部の水平面からの各高さは異なり、
前記収音部は、前記撮像部の撮像視野角外に存在するように設置され、
前記収音方向算出部は、
前記撮像部から前記収音部に向かう水平角と、
前記撮像部と前記収音部との間の水平方向の距離と、
前記収音部から前記撮像部に向かう水平角と、
前記撮像部の前記水平面からの高さと、前記収音部の前記水平面からの高さと、
前記撮像部から前記音声位置に向かう水平角及び垂直角と、
前記音声位置の水平面からの高さの入力値と、を用いて、
前記収音部から前記音声位置に向かう収音方向の水平角及び垂直角を算出する、
指向性制御システム。 - 請求項40に記載の指向性制御システムであって、
前記収音部の真下に、前記収音部からの距離が一定の第1マーカが吊り下げられ、
前記収音部の真下の前記水平面上の位置に、円形状の第2マーカが設置され、
前記収音方向算出部は、
前記表示部に表示された前記第1マーカ及び前記第2マーカの中心点の指定に応じて、前記撮像部から前記第1マーカに向かう垂直角と、前記撮像部から前記第2マーカに向かう垂直角と、前記撮像部から前記第2マーカまでの距離と、を用いて、
前記撮像部と前記収音部との間の水平方向の距離と、前記撮像部の前記水平面からの高さと、前記収音部の前記水平面からの高さとを算出する、
指向性制御システム。 - 請求項41に記載の指向性制御システムであって、
前記収音部の真下の前記水平面上の位置に、円形状の第2マーカが設置され、
前記収音方向算出部は、
前記表示部に表示された前記第2マーカの中心点の指定に応じて、前記撮像部から前記第2マーカに向かう垂直角と、前記撮像部から前記第2マーカまでの距離と、を用いて、
前記撮像部と前記収音部との間の水平方向の距離と、前記撮像部の前記水平面からの高さとを算出する、
指向性制御システム。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2015500154A JP6253031B2 (ja) | 2013-02-15 | 2014-02-14 | キャリブレーション方法 |
EP14751319.6A EP2958339B1 (en) | 2013-02-15 | 2014-02-14 | Directionality control system and directionality control method |
CN201480017437.4A CN105075288B (zh) | 2013-02-15 | 2014-02-14 | 指向性控制系统、校准方法、水平偏差角计算方法及指向性控制方法 |
US14/767,411 US9860439B2 (en) | 2013-02-15 | 2014-02-14 | Directionality control system, calibration method, horizontal deviation angle computation method, and directionality control method |
US15/830,776 US10244162B2 (en) | 2013-02-15 | 2017-12-04 | Directionality control system, calibration method, horizontal deviation angle computation method, and directionality control method |
Applications Claiming Priority (10)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2013028402 | 2013-02-15 | ||
JP2013-028402 | 2013-02-15 | ||
JP2013-119850 | 2013-06-06 | ||
JP2013119850 | 2013-06-06 | ||
JP2013-129964 | 2013-06-20 | ||
JP2013129964 | 2013-06-20 | ||
JP2013-154867 | 2013-07-25 | ||
JP2013154867 | 2013-07-25 | ||
JP2013-217707 | 2013-10-18 | ||
JP2013217707 | 2013-10-18 |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/767,411 A-371-Of-International US9860439B2 (en) | 2013-02-15 | 2014-02-14 | Directionality control system, calibration method, horizontal deviation angle computation method, and directionality control method |
US15/830,776 Continuation US10244162B2 (en) | 2013-02-15 | 2017-12-04 | Directionality control system, calibration method, horizontal deviation angle computation method, and directionality control method |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014125835A1 true WO2014125835A1 (ja) | 2014-08-21 |
Family
ID=51353846
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2014/000775 WO2014125835A1 (ja) | 2013-02-15 | 2014-02-14 | 指向性制御システム、キャリブレーション方法、水平偏差角算出方法及び指向性制御方法 |
Country Status (5)
Country | Link |
---|---|
US (2) | US9860439B2 (ja) |
EP (1) | EP2958339B1 (ja) |
JP (2) | JP6253031B2 (ja) |
CN (1) | CN105075288B (ja) |
WO (1) | WO2014125835A1 (ja) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016140055A (ja) * | 2015-01-23 | 2016-08-04 | 株式会社リコー | 動画像音収録システム、動画像音収録装置、動画像音収録プログラム、および動画像音収録方法 |
JP2016191633A (ja) * | 2015-03-31 | 2016-11-10 | 株式会社熊谷組 | 診断装置 |
WO2017170007A1 (ja) * | 2016-03-30 | 2017-10-05 | 日本電気株式会社 | 監視装置、監視方法、及び記録媒体 |
JP2018506243A (ja) * | 2014-12-15 | 2018-03-01 | 華為技術有限公司Huawei Technologies Co.,Ltd. | ビデオチャットにおける録音方法、および端末 |
JP2018101987A (ja) * | 2018-01-31 | 2018-06-28 | パナソニックIpマネジメント株式会社 | 監視エリアの音源表示システム及び音源表示方法 |
US10063967B2 (en) | 2016-03-22 | 2018-08-28 | Panasonic Intellectual Property Management Co., Ltd. | Sound collecting device and sound collecting method |
JP2019103011A (ja) * | 2017-12-05 | 2019-06-24 | 日本電信電話株式会社 | 変換装置、変換方法、およびプログラム |
US10824876B2 (en) | 2015-11-06 | 2020-11-03 | Panasonic Intellectual Property Management Co., Ltd. | Pilotless flying object detection system and pilotless flying object detection method |
US10909384B2 (en) | 2015-07-14 | 2021-02-02 | Panasonic Intellectual Property Management Co., Ltd. | Monitoring system and monitoring method |
CN112466323A (zh) * | 2020-11-24 | 2021-03-09 | 中核检修有限公司 | 一种光学图像与声学图像融合方法及系统 |
US11234074B2 (en) | 2017-09-08 | 2022-01-25 | Panasonic Intellectual Property Management Co., Ltd. | Sound pickup device, sound pickup system, sound pickup method, program, and calibration method |
Families Citing this family (36)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9294839B2 (en) | 2013-03-01 | 2016-03-22 | Clearone, Inc. | Augmentation of a beamforming microphone array with non-beamforming microphones |
US9565493B2 (en) | 2015-04-30 | 2017-02-07 | Shure Acquisition Holdings, Inc. | Array microphone system and method of assembling the same |
US9554207B2 (en) | 2015-04-30 | 2017-01-24 | Shure Acquisition Holdings, Inc. | Offset cartridge microphones |
KR20170017401A (ko) * | 2015-08-06 | 2017-02-15 | 엘지이노텍 주식회사 | 이미지 처리 장치 |
EP3188504B1 (en) * | 2016-01-04 | 2020-07-29 | Harman Becker Automotive Systems GmbH | Multi-media reproduction for a multiplicity of recipients |
KR102465227B1 (ko) | 2016-05-30 | 2022-11-10 | 소니그룹주식회사 | 영상 음향 처리 장치 및 방법, 및 프로그램이 저장된 컴퓨터 판독 가능한 기록 매체 |
US20170366897A1 (en) * | 2016-06-15 | 2017-12-21 | Robert Azarewicz | Microphone board for far field automatic speech recognition |
CN108089152B (zh) * | 2016-11-23 | 2020-07-03 | 杭州海康威视数字技术股份有限公司 | 一种设备控制方法、装置及系统 |
US10299060B2 (en) * | 2016-12-30 | 2019-05-21 | Caavo Inc | Determining distances and angles between speakers and other home theater components |
US10367948B2 (en) | 2017-01-13 | 2019-07-30 | Shure Acquisition Holdings, Inc. | Post-mixing acoustic echo cancellation systems and methods |
CN109696658B (zh) * | 2017-10-23 | 2021-08-24 | 京东方科技集团股份有限公司 | 采集设备、声音采集方法、声源跟踪系统及其方法 |
US10567746B2 (en) * | 2017-11-14 | 2020-02-18 | Caterpillar Inc. | Calibration jig |
WO2019149337A1 (en) | 2018-01-30 | 2019-08-08 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatuses for converting an object position of an audio object, audio stream provider, audio content production system, audio playback apparatus, methods and computer programs |
EP3525482B1 (en) * | 2018-02-09 | 2023-07-12 | Dolby Laboratories Licensing Corporation | Microphone array for capturing audio sound field |
US10873727B2 (en) * | 2018-05-14 | 2020-12-22 | COMSATS University Islamabad | Surveillance system |
US11523212B2 (en) | 2018-06-01 | 2022-12-06 | Shure Acquisition Holdings, Inc. | Pattern-forming microphone array |
TWI655451B (zh) * | 2018-06-11 | 2019-04-01 | 臺中榮民總醫院嘉義分院 | 空間定位及方向指引動態回饋系統 |
US11297423B2 (en) | 2018-06-15 | 2022-04-05 | Shure Acquisition Holdings, Inc. | Endfire linear array microphone |
CN112889296A (zh) | 2018-09-20 | 2021-06-01 | 舒尔获得控股公司 | 用于阵列麦克风的可调整的波瓣形状 |
EP3942842A1 (en) | 2019-03-21 | 2022-01-26 | Shure Acquisition Holdings, Inc. | Housings and associated design features for ceiling array microphones |
US11558693B2 (en) | 2019-03-21 | 2023-01-17 | Shure Acquisition Holdings, Inc. | Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition and voice activity detection functionality |
CN113841421A (zh) | 2019-03-21 | 2021-12-24 | 舒尔获得控股公司 | 具有抑制功能的波束形成麦克风瓣的自动对焦、区域内自动对焦、及自动配置 |
CN110009916A (zh) * | 2019-05-05 | 2019-07-12 | 江苏聆世科技有限公司 | 汽车鸣笛检测设备及方法 |
TW202101422A (zh) | 2019-05-23 | 2021-01-01 | 美商舒爾獲得控股公司 | 可操縱揚聲器陣列、系統及其方法 |
TW202105369A (zh) | 2019-05-31 | 2021-02-01 | 美商舒爾獲得控股公司 | 整合語音及雜訊活動偵測之低延時自動混波器 |
US11297426B2 (en) | 2019-08-23 | 2022-04-05 | Shure Acquisition Holdings, Inc. | One-dimensional array microphone with improved directivity |
CN110493690B (zh) * | 2019-08-29 | 2021-08-13 | 北京搜狗科技发展有限公司 | 一种声音采集方法及装置 |
JP6739064B1 (ja) * | 2020-01-20 | 2020-08-12 | パナソニックIpマネジメント株式会社 | 撮像装置 |
US11552611B2 (en) | 2020-02-07 | 2023-01-10 | Shure Acquisition Holdings, Inc. | System and method for automatic adjustment of reference gain |
CN111625025B (zh) * | 2020-04-24 | 2023-04-28 | Oppo(重庆)智能科技有限公司 | 参数校准方法、装置、存储介质及电子设备 |
US11706562B2 (en) | 2020-05-29 | 2023-07-18 | Shure Acquisition Holdings, Inc. | Transducer steering and configuration systems and methods using a local positioning system |
CN113923562A (zh) * | 2020-07-10 | 2022-01-11 | 北京搜狗智能科技有限公司 | 拾音方法和装置 |
US11714157B2 (en) * | 2020-11-10 | 2023-08-01 | Amazon Technologies, Inc. | System to determine direction toward user |
CN112379330B (zh) * | 2020-11-27 | 2023-03-10 | 浙江同善人工智能技术有限公司 | 一种多机器人协同的3d声源识别定位方法 |
WO2022165007A1 (en) | 2021-01-28 | 2022-08-04 | Shure Acquisition Holdings, Inc. | Hybrid audio beamforming system |
CN112967488B (zh) * | 2021-04-09 | 2022-05-06 | 宁波奥克斯电气股份有限公司 | 一种空调器显示控制方法及装置、空调器 |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH08286680A (ja) * | 1995-02-17 | 1996-11-01 | Takenaka Komuten Co Ltd | 音抽出装置 |
JPH1051889A (ja) * | 1996-08-05 | 1998-02-20 | Toshiba Corp | 音声収集装置及び音声収集方法 |
JP2006222618A (ja) * | 2005-02-09 | 2006-08-24 | Casio Comput Co Ltd | カメラ装置、カメラ制御プログラム及び記録音声制御方法 |
JP2007214753A (ja) * | 2006-02-08 | 2007-08-23 | Matsushita Electric Ind Co Ltd | 制御方法及び制御装置 |
JP4252377B2 (ja) | 2002-06-27 | 2009-04-08 | マイクロソフト コーポレーション | 全方位カメラ及びマイクロフォンアレイのためのシステム |
JP2010213091A (ja) * | 2009-03-11 | 2010-09-24 | Ikegami Tsushinki Co Ltd | 音源位置推定装置 |
JP2012186551A (ja) | 2011-03-03 | 2012-09-27 | Hitachi Ltd | 制御装置、制御システムと制御方法 |
Family Cites Families (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS5957596A (ja) * | 1982-09-27 | 1984-04-03 | Sony Corp | マイクロホン装置 |
US5508734A (en) * | 1994-07-27 | 1996-04-16 | International Business Machines Corporation | Method and apparatus for hemispheric imaging which emphasizes peripheral content |
US6157403A (en) | 1996-08-05 | 2000-12-05 | Kabushiki Kaisha Toshiba | Apparatus for detecting position of object capable of simultaneously detecting plural objects and detection method therefor |
US20030160862A1 (en) * | 2002-02-27 | 2003-08-28 | Charlier Michael L. | Apparatus having cooperating wide-angle digital camera system and microphone array |
JP4269883B2 (ja) | 2003-10-20 | 2009-05-27 | ソニー株式会社 | マイクロホン装置、再生装置及び撮像装置 |
JP2008271157A (ja) * | 2007-04-19 | 2008-11-06 | Fuji Xerox Co Ltd | 音声強調装置及び制御プログラム |
CN101137008A (zh) * | 2007-07-11 | 2008-03-05 | 裘炅 | 一种将位置信息隐藏于视频、音频或图的摄像装置及方法 |
US20100123785A1 (en) | 2008-11-17 | 2010-05-20 | Apple Inc. | Graphic Control for Directional Audio Input |
US20100254543A1 (en) * | 2009-02-03 | 2010-10-07 | Squarehead Technology As | Conference microphone system |
US8363512B2 (en) * | 2009-02-27 | 2013-01-29 | Honda Motors | Method and apparatus for estimating sound source |
JP5198530B2 (ja) * | 2010-09-28 | 2013-05-15 | 株式会社東芝 | 音声付き動画像呈示装置、方法およびプログラム |
US10048933B2 (en) * | 2011-11-30 | 2018-08-14 | Nokia Technologies Oy | Apparatus and method for audio reactive UI information and display |
EP2680616A1 (en) * | 2012-06-25 | 2014-01-01 | LG Electronics Inc. | Mobile terminal and audio zooming method thereof |
US20150186109A1 (en) * | 2012-08-10 | 2015-07-02 | Nokia Corporation | Spatial audio user interface apparatus |
US9007524B2 (en) * | 2012-09-25 | 2015-04-14 | Intel Corporation | Techniques and apparatus for audio isolation in video processing |
JP6216169B2 (ja) * | 2012-09-26 | 2017-10-18 | キヤノン株式会社 | 情報処理装置、情報処理方法 |
WO2014155153A1 (en) * | 2013-03-27 | 2014-10-02 | Nokia Corporation | Image point of interest analyser with animation generator |
US9516412B2 (en) | 2014-03-28 | 2016-12-06 | Panasonic Intellectual Property Management Co., Ltd. | Directivity control apparatus, directivity control method, storage medium and directivity control system |
US9414153B2 (en) * | 2014-05-08 | 2016-08-09 | Panasonic Intellectual Property Management Co., Ltd. | Directivity control apparatus, directivity control method, storage medium and directivity control system |
JP6217930B2 (ja) * | 2014-07-15 | 2017-10-25 | パナソニックIpマネジメント株式会社 | 音速補正システム |
-
2014
- 2014-02-14 EP EP14751319.6A patent/EP2958339B1/en active Active
- 2014-02-14 JP JP2015500154A patent/JP6253031B2/ja active Active
- 2014-02-14 CN CN201480017437.4A patent/CN105075288B/zh active Active
- 2014-02-14 US US14/767,411 patent/US9860439B2/en active Active
- 2014-02-14 WO PCT/JP2014/000775 patent/WO2014125835A1/ja active Application Filing
-
2017
- 2017-11-16 JP JP2017221003A patent/JP6485715B2/ja active Active
- 2017-12-04 US US15/830,776 patent/US10244162B2/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH08286680A (ja) * | 1995-02-17 | 1996-11-01 | Takenaka Komuten Co Ltd | 音抽出装置 |
JPH1051889A (ja) * | 1996-08-05 | 1998-02-20 | Toshiba Corp | 音声収集装置及び音声収集方法 |
JP4252377B2 (ja) | 2002-06-27 | 2009-04-08 | マイクロソフト コーポレーション | 全方位カメラ及びマイクロフォンアレイのためのシステム |
JP2006222618A (ja) * | 2005-02-09 | 2006-08-24 | Casio Comput Co Ltd | カメラ装置、カメラ制御プログラム及び記録音声制御方法 |
JP2007214753A (ja) * | 2006-02-08 | 2007-08-23 | Matsushita Electric Ind Co Ltd | 制御方法及び制御装置 |
JP2010213091A (ja) * | 2009-03-11 | 2010-09-24 | Ikegami Tsushinki Co Ltd | 音源位置推定装置 |
JP2012186551A (ja) | 2011-03-03 | 2012-09-27 | Hitachi Ltd | 制御装置、制御システムと制御方法 |
Non-Patent Citations (1)
Title |
---|
See also references of EP2958339A4 |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10152985B2 (en) | 2014-12-15 | 2018-12-11 | Huawei Technologies Co., Ltd. | Method for recording in video chat, and terminal |
JP2018506243A (ja) * | 2014-12-15 | 2018-03-01 | 華為技術有限公司Huawei Technologies Co.,Ltd. | ビデオチャットにおける録音方法、および端末 |
JP2016140055A (ja) * | 2015-01-23 | 2016-08-04 | 株式会社リコー | 動画像音収録システム、動画像音収録装置、動画像音収録プログラム、および動画像音収録方法 |
JP2016191633A (ja) * | 2015-03-31 | 2016-11-10 | 株式会社熊谷組 | 診断装置 |
US10909384B2 (en) | 2015-07-14 | 2021-02-02 | Panasonic Intellectual Property Management Co., Ltd. | Monitoring system and monitoring method |
US10824876B2 (en) | 2015-11-06 | 2020-11-03 | Panasonic Intellectual Property Management Co., Ltd. | Pilotless flying object detection system and pilotless flying object detection method |
US10063967B2 (en) | 2016-03-22 | 2018-08-28 | Panasonic Intellectual Property Management Co., Ltd. | Sound collecting device and sound collecting method |
WO2017170007A1 (ja) * | 2016-03-30 | 2017-10-05 | 日本電気株式会社 | 監視装置、監視方法、及び記録媒体 |
JPWO2017170007A1 (ja) * | 2016-03-30 | 2019-02-28 | 日本電気株式会社 | 監視装置、監視方法、及びプログラム |
US11234074B2 (en) | 2017-09-08 | 2022-01-25 | Panasonic Intellectual Property Management Co., Ltd. | Sound pickup device, sound pickup system, sound pickup method, program, and calibration method |
JP2019103011A (ja) * | 2017-12-05 | 2019-06-24 | 日本電信電話株式会社 | 変換装置、変換方法、およびプログラム |
JP2018101987A (ja) * | 2018-01-31 | 2018-06-28 | パナソニックIpマネジメント株式会社 | 監視エリアの音源表示システム及び音源表示方法 |
CN112466323A (zh) * | 2020-11-24 | 2021-03-09 | 中核检修有限公司 | 一种光学图像与声学图像融合方法及系统 |
Also Published As
Publication number | Publication date |
---|---|
JPWO2014125835A1 (ja) | 2017-02-02 |
CN105075288A (zh) | 2015-11-18 |
JP6253031B2 (ja) | 2017-12-27 |
EP2958339A4 (en) | 2017-01-18 |
EP2958339B1 (en) | 2019-09-18 |
EP2958339A1 (en) | 2015-12-23 |
US9860439B2 (en) | 2018-01-02 |
US20180160032A1 (en) | 2018-06-07 |
JP2018057023A (ja) | 2018-04-05 |
JP6485715B2 (ja) | 2019-03-20 |
US20160142620A1 (en) | 2016-05-19 |
CN105075288B (zh) | 2018-10-19 |
US10244162B2 (en) | 2019-03-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6485715B2 (ja) | 指向性制御システム及び指向性制御方法 | |
JP4607095B2 (ja) | 測量機器において画像処理を行う方法と装置 | |
US9377303B2 (en) | Surveying appliance and method having a targeting functionality which is based on the orientation of a remote control unit and is scalable | |
JP5958717B2 (ja) | 指向性制御システム、指向性制御方法、収音システム及び収音制御方法 | |
KR101520894B1 (ko) | 로봇식 측량 기계 및 오토콜리메이션 타겟을 포함하는 측량 기계의 망원경의 자동화된 오토콜리메이션을 위한 방법 | |
US4724480A (en) | Method for optical alignment of one object with respect to another | |
EP1737265A1 (en) | Determination of the position of sound sources | |
WO2004068326A1 (ja) | 運動検出装置及び通信装置 | |
US9503628B1 (en) | Camera mounting and control device | |
WO2009100773A1 (en) | Localization of a surveying instrument in relation to a ground mark | |
CN103389486B (zh) | 控制方法和电子设备 | |
US20130193855A1 (en) | System and method for controlling media projectors | |
JP2003279351A (ja) | 測量装置及び測量装置を用いて画像データを取得する方法 | |
JP2001285692A (ja) | 全方向ステレオ画像撮影装置及びステレオ画像撮影装置 | |
JP2021189179A (ja) | ポイントフロムフォーカスタイプの測定動作を実行するためのビジョンプローブを備えた座標測定機 | |
JP6425860B1 (ja) | 機器操作装置、機器操作システムおよび機器操作方法 | |
EP2607951B1 (en) | Method for setting up a monitoring camera | |
US11832016B2 (en) | 3D tour photographing apparatus and method | |
JP2016119621A (ja) | キャリブレーション方法 | |
Bellotto | A multimodal smartphone interface for active perception by visually impaired | |
JP2003023562A (ja) | 画像撮影システムおよびカメラシステム | |
JP3359241B2 (ja) | 撮像方法および装置 | |
KR101962755B1 (ko) | 3차원 지도 작성을 위한 데이터 수집 장치 | |
JP2012169811A (ja) | 撮像装置、マイクロホン装置、撮像システム及びプログラム | |
JP2012112896A (ja) | 測量システム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201480017437.4 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14751319 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2015500154 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2014751319 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14767411 Country of ref document: US |