JP2009239346A - Photographing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide technique for following up and photographing a specific subject by a photographing device having simpler constitution than the conventional ones. <P>SOLUTION: The photographing device 1 has a plurality of microphones 15 arranged in a column. A control unit 11 of the photographing device 1 analyzes sound data representing sounds picked up by the respective microphones 15 and estimates a plurality of directions of sound sources according to analysis results. Further, the control unit 11 collates the sound data representing the sounds from the estimated sound source directions with collation data stored in a collation data storage area 121 to specify the direction of the specific subject according to matching degrees thereof. Further, the control unit 11 analyzes the sound data by the microphones 15 to detect transition of the direction of the subject according to analysis results, and controls a rotating mechanism 70 to change the direction of the photographing device 1 such that the detected direction is included in a photographing range of the photographing device 1. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for performing photographing.

Techniques have been proposed for tracking and shooting a specific subject in a shooting device such as a digital camera for shooting still images and moving images. For example, Patent Document 1 proposes an apparatus for automatically tracking and photographing a subject to which a transmitter is attached. Further, in Patent Document 2, a tag is attached to each of a plurality of subjects, a tuning signal is transmitted with a spread angle substantially equal to the angle of view, and a response signal transmitted from each tag is received in response to the tuning signal. There has been proposed an apparatus that detects a direction and performs imaging by making the imaging direction substantially coincide with the detected tag direction. According to this apparatus, even when there are a plurality of subjects, only the subject within the angle of view can be reliably captured and photographed.
JP 2003-69884 A JP 2006-261999 A

However, in the techniques described in Patent Documents 1 and 2, it is necessary to separately provide a receiving unit for receiving a signal from a transmitter (tag) in the photographing device, and the device configuration of the photographing device becomes complicated. There was a problem.
The present invention has been made under the above-described background, and an object of the present invention is to provide a technique capable of tracking and shooting a specific subject with a simpler configuration than in the past.

  In order to solve the above problems, a photographing apparatus according to a preferred aspect of the present invention includes a photographing unit that sets a photographing range and outputs video data representing a video in the photographing range, and a microphone around the photographing unit. A plurality of microphones that collect sound for each time and output as sound data, and an estimation unit that analyzes sound data output from each of the plurality of microphones and estimates one or a plurality of sound source directions according to the analysis result And specifying means for specifying at least one of the directions of the sound source estimated by the estimating means; and analyzing the sound data for each microphone and identifying the sound source specified by the specifying means according to the analysis result Detecting means for detecting a transition in the direction of the image capturing area, and the imaging range for changing the imaging range of the imaging means to a range including the direction of the sound source detected by the detecting means Characterized by comprising a further means.

  In the above-described aspect, there may be provided video data storage control means for storing video data output from the photographing means in a predetermined storage means when the photographing range is changed by the photographing range changing means.

  In the above aspect, the estimating means calculates a sound pressure distribution around the photographing means based on the correlation of the sound data for each microphone, and determines the direction in which the sound pressure peak appears in the calculated distribution. The direction of the sound source may be estimated.

In the above-described aspect, the photographing range changing unit may change the direction of the photographing unit so that the direction of the sound source estimated by the direction estimating unit is included in the photographing range.
In the above-described aspect, the detection unit may detect a transition of a sound pressure peak in the sound pressure distribution calculated by the estimation unit.

  Further, in the above-mentioned aspect, the collation data storage means for storing the collation data representing the characteristics of the voice, and the direction-specific voice data corresponding to each of the directions estimated by the estimation means from the voice data for each microphone. Direction-specific audio data generation means for generating the direction-specific audio data generated by the direction-specific audio data generation means and the verification data stored in the verification data storage means The direction of the sound source may be specified based on the degree of matching between the two.

In the above-described aspect, collation data generating means for generating voice data corresponding to the direction specified by the specifying means from the voice data for each of the plurality of microphones as verification data; and the voice data for each microphone Directional audio data generation means for generating direction-specific audio data corresponding to each of the directions estimated by the estimation means, and the detection means is a direction generated by the direction-specific audio data generation means. Each of the different audio data may be collated with the collation data generated by the collation data generating means, and the direction change of the sound source may be detected based on the degree of coincidence.
In the above-described aspect, the specifying unit may specify the direction of the sound source according to a signal output from the operation unit.

In the above-described aspect, the verification data may include specific personal voice feature information.
In the above-described aspect, the direction-specific sound data generation unit generates direction-specific sound data by mixing each direction from the direction estimated by the estimation unit so that the sound pressure from the direction becomes high. May be.

In the above-described aspect, the direction-specific sound data generation means may generate direction-specific sound data by estimating sound data corresponding to a sound source from the sound data using independent component analysis.
In the above-described aspect, the estimation means may estimate the direction of the sound source using independent component analysis.

  According to the present invention, it is possible to track and photograph a specific subject with a simpler configuration than in the past.

Embodiments of the present invention will be described below with reference to the drawings.
<A: Configuration>
FIG. 1 is a block diagram illustrating a hardware configuration of a photographing apparatus 1 according to an embodiment of the present invention, and FIG. 2 is a perspective view illustrating an appearance of the photographing apparatus 1. The photographing device 1 is a device having a function of photographing a still image or a moving image, for example, a digital camera. In FIG. 1, the control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and reads and executes a computer program stored in the ROM or the storage unit 12. Thus, each unit of the photographing apparatus 1 is controlled via the bus BUS. The storage unit 12 is a storage unit for storing a computer program executed by the control unit 11 and data used at the time of execution, and is, for example, a hard disk device. The display unit 13 includes a liquid crystal panel and the like, and displays various images under the control of the control unit 11. The operation unit 14 outputs a signal corresponding to an operation by the user of the photographing apparatus 1 to the control unit 11. The operation unit 14 has various buttons such as a cross key (not shown), a recording button B1 for starting / ending recording, and a shooting button B2 for starting / ending still image shooting and moving image shooting. The user of the photographing apparatus 1 can perform various operations such as still image photographing and moving image photographing by pressing these buttons. Note that switching between still image shooting and moving image shooting can be switched by a change-over switch (not shown) provided in the shooting apparatus 1.

  The photographing unit 18 includes a photographing lens 18a and the like, photographs and outputs video data representing the photographed video. The photographing unit 18 can change the photographing range by moving the photographing lens 18a back and forth. The user of the photographing apparatus 1 can set the photographing range of the photographing unit 18 using the cross key or the like of the operation unit 14, and the photographing unit 18 performs photographing according to a signal from the operation unit 14. The imaging range is set by moving the lens 18a. Note that the video data according to the present embodiment includes data representing still images and data representing moving images.

  The microphone array MA is configured by arranging a plurality of microphones 151, 152,..., 15n (n is a natural number of 2 or more) in a row. As shown in FIG. 2, a plurality of microphones 151, 152,..., 15 n are arranged in a row on the front surface of the photographing apparatus 1 (the same surface as the surface on which the photographing lens 18 a is provided). The microphones 151, 152,... 15n collect sound around the photographing apparatus 1. The plurality of microphones 151, 152,..., 15n are preferably directional microphones. In the following description, when it is not necessary to distinguish the microphones 151, 152,..., 15n, these will be referred to as “microphones 15”. The microphone 15 is a sound collection unit that collects sound and outputs an analog signal representing the collected sound. The sound processing unit 16 A / D converts the analog signal output from the microphone 15 to generate digital data. In addition, under the control of the control unit 11, the audio processing unit 16 D / A converts digital audio data to generate an analog signal, and outputs the generated analog signal to the speaker 17. The speaker 17 is a sound emitting unit that emits sound with an intensity corresponding to the analog signal supplied from the sound processing unit 16.

  In this embodiment, the case where the microphone 15 and the speaker 17 are included in the photographing apparatus 1 will be described. However, the audio processing unit 16 is provided with an input terminal and an output terminal, and the input terminal is connected to the input terminal via an audio cable. An external microphone may be connected, and similarly, an external speaker may be connected to the output terminal via an audio cable. In this embodiment, the case where the audio signal input from the microphone 15 to the audio processing unit 16 and the audio signal output from the audio processing unit 16 to the speaker 17 are analog audio signals will be described. You may make it input / output. In such a case, the audio processing unit 16 does not need to perform A / D conversion or D / A conversion. The same applies to the display unit 13, the operation unit 14, and the imaging unit 18, and may be a format built in the imaging device 1 or an externally attached format.

  The storage unit 12 includes a verification data storage area 121 and a moving image data storage area 122 as shown in the figure. The collation data storage area 121 stores collation data representing the characteristics (frequency characteristics, etc.) of a specific person's voice recorded in advance. This data for collation is data used when the control part 11 performs the collation process mentioned later. In the following explanation, for convenience of explanation, the voice represented by the collation data stored in the collation data storage area 121 is referred to as “specific voice”. The moving image data storage area 122 stores moving image data including video data output from the photographing unit 18 and sound data representing sound collected by the microphone array MA. When shooting is performed by operating the shooting button B2 of the operation unit 14 by the photographer, the control unit 11 includes video data output from the shooting unit 18 and audio data representing the sound collected by the microphone array MA. Is stored in the moving image data storage area 122.

  The imaging device 1 is attached to a rotation mechanism 70 as shown in FIG. The rotation mechanism 70 is installed on a table 61 such as a desk, and the photographing apparatus 1 can be rotated in the direction of arrow P by the rotation device 70. FIG. 3 is a diagram illustrating an example of the configuration of the rotation mechanism 70. The rotation mechanism 70 has a rotation part 71 and a fixing part 72 as shown in the figure. The rotating unit 71 is fixed to the photographing apparatus 1. The fixed portion 72 is provided with a shaft 721, and the shaft 721 is inserted into the bearing 711 of the rotating portion 71, and the rotating portion 71 is rotatably supported by the shaft 721. The fixed portion 72 is provided with a drive gear 722 and a motor 723 that rotates the drive gear 722. The motor 723 rotates the drive gear 722 under the control of the control unit 11. By rotating the drive gear 722, the passive gear 712 meshed with the drive gear 722 is rotated, whereby the rotating portion 71 is rotated about the shaft 721 by driving the motor 723.

<B: Operation>
<B-1: Collation data registration operation>
Next, the operation of this embodiment will be described. First, the user of the photographing apparatus 1 operates the operation unit 14 to perform an operation for registering verification data. When the photographer presses the recording button B1 to start recording, the operation unit 14 outputs an operation signal corresponding to the operated content, and the control unit 11 responds to the signal supplied from the operation unit 14. The audio processing unit 16 is controlled to start recording. The user's voice is picked up by the microphone 15, converted into a voice signal, and output to the voice processing unit 16. The audio processing unit 16 converts the audio signal output from the microphone 15 into digital data (hereinafter referred to as “audio data”). The control unit 11 performs predetermined filtering processing or the like on the audio data output from the audio processing unit 16 to generate feature data representing the features of the audio from the audio data, and uses the generated feature data as verification data. Store in the data storage area 121. When the user performs an operation of ending recording by pressing the recording button B1, the control unit 11 ends the recording in response to a signal supplied from the operation unit 14.

<B-2: Shooting operation>
Next, a photographing operation performed by the photographing apparatus 1 will be described. The photographing apparatus 1 can switch between three modes: a moving image photographing mode, a still image photographing mode, and an automatic photographing mode. The user can switch the photographing mode by operating the operation unit 14 of the photographing apparatus 1. The control unit 11 performs shooting processing in the selected mode in accordance with a signal output from the operation unit 14. Hereinafter, an operation when the automatic shooting mode is selected will be described. Note that the operation of moving image shooting and still image shooting is the same as the operation of a conventional shooting apparatus, and detailed description thereof is omitted here.

  FIG. 4 is a flowchart showing the flow of the photographing process performed by the photographing apparatus 1. The process shown in FIG. 4 is disclosed when the photographer turns on the power of the photographing apparatus 1 and selects the automatic photographing mode. When the automatic shooting mode is selected, the control unit 11 starts shooting a moving image. The microphone 15 converts the collected sound into a sound signal and outputs the sound signal to the sound processing unit 16. The audio processing unit 16 converts the audio signal output from the microphone 15 into audio data. The control unit 11 mixes audio data corresponding to each of the plurality of microphones 15 to generate overall audio data representing the entire audio, the generated overall audio data, and video data output from the imaging unit 18. Are stored in the moving image data storage area 122 as moving image data.

  Further, the control unit 11 analyzes the sound data for each microphone 15 and estimates the direction of the sound source (hereinafter referred to as “sound source direction”) according to the analysis result (step S1). Here, the control unit 11 detects the sound pressure of the audio signal output from each of the plurality of microphones 15, calculates the sound pressure distribution based on the detected sound pressure correlation for each microphone 15, and calculates The direction in which the sound pressure peak appears in the distribution is estimated as the direction of the sound source. An example of specific contents of this estimation process will be described with reference to FIG.

  FIG. 5 is a diagram illustrating an example of a sound pressure distribution calculated by the control unit 11. In the figure, the horizontal axis indicates the angle with respect to the center direction of the microphone array MA, and the vertical axis indicates the sound pressure. The time required for the sound wave generated by a certain sound source to reach each of the plurality of microphones 15 varies depending on the direction (angle) of the sound source as viewed from the photographing apparatus 1. Using this principle, in this operation example, for each predetermined unit amount of angle, a delay time corresponding to the angle is set in advance for each microphone 15, and the control unit 11 stores the audio data for each microphone 15. Each delay time corresponding to the microphone 15 is delayed, the sound data for each delayed microphone 15 is mixed, and the sound pressure corresponding to each angle is calculated. Next, the control unit 11 detects one or a plurality of angles at which a peak appears in the calculated sound pressure for each angle (that is, sound pressure distribution), and sets the detected angle as the direction of the sound source. In the example shown in FIG. 5, the control unit 11 estimates the angles θ1, θ2, and θ3 at which the sound pressure peaks appear as the sound source directions.

  Next, the control unit 11 specifies at least one of the estimated sound source directions as a direction in which the specific subject is present (hereinafter referred to as “specific direction”). In this operation example, the control unit 11 first mixes the audio data for each of the plurality of microphones 15 for each predetermined unit amount of angle so that the sound pressure of the sound from each angle becomes high. Audio data (hereinafter referred to as “directional audio data”) is generated (step S2). Next, the control unit 11 performs a predetermined filtering process or the like on the generated direction-specific sound data to generate feature data representing the features of the sound, and the generated feature data is stored in the matching data storage area 121. The direction with the highest degree of coincidence is identified as a specific direction (step S3).

  When the specific direction is specified, the control unit 11 controls the motor 723 of the rotation mechanism 70 to change the orientation of the photographing apparatus 1 so that the specific direction is included in the photographing range of the photographing apparatus 1 (step S4). . At this time, the control unit 11 may change the orientation of the photographing apparatus 1 so that the central direction of the photographing range of the photographing apparatus 1 matches the specific direction.

  The control unit 11 outputs the moving image data including the video data output from the photographing unit 18 and the audio data representing the sound collected by the microphone 15 to the moving image data storage area 122 (step S5). Next, the control unit 11 determines whether or not to end shooting (step S6). If the determination result is affirmative (step S6; YES), the shooting ends (step S7). On the other hand, when the determination result is negative (step S6; NO), the control unit 11 continuously performs photographing.

  During the shooting, the control unit 11 analyzes the sound data for each microphone 15 and detects a transition in a specific direction according to the analysis result (step S8). In this operation example, the control unit 11 calculates the sound pressure distribution with respect to the direction based on the correlation of the sound pressure of the sound data for each microphone 15, and sets the direction in which the sound pressure peak appears in the calculated distribution to a predetermined unit. It is detected every time (for example, 10 ms). And the control part 11 detects the transition of the peak of a sound pressure, and detects the transition of a specific direction according to a detection result. For example, the control unit 11 detects a sound pressure peak, and the difference between the detected peak angle and the previously detected peak angle is equal to or less than a predetermined threshold. In some cases, it is determined that the sound source at the previous peak position has moved. Specifically, for example, in the example shown in FIG. 5, when the angle θ2 is specified as the specific direction, the sound pressure distribution is changed to that shown in FIG. 6 after a predetermined unit time in the state shown in FIG. Suppose that a transition occurs. At this time, when the difference between the angle θ21 shown in FIG. 6 and the angle θ2 shown in FIG. 5 is equal to or smaller than a predetermined threshold, the control unit 11 is in the direction of the angle θ2 at the time shown in FIG. It is determined that the sound source has moved in the direction of the angle θ21, and the angle θ21 is detected as a specific direction.

  Thus, the control unit 11 detects the peak of the sound pressure every predetermined unit time, and detects the movement in the specific direction according to the difference between the detected peak direction and the specific direction. The control unit 11 sequentially detects this movement over the period of shooting, controls the rotation mechanism 70 according to the detection result, and tracks and changes the direction of the imaging device 1 in a specific direction. Thereby, even when the specific subject moves, it is possible to perform shooting while tracking the direction of the specific subject.

  As described above, the imaging apparatus 1 tracks the transition in the specific direction, changes the imaging range of the imaging unit 18, generates video data of the video within the imaging range, and reproduces the entire audio within the imaging range. Audio data to be represented is generated, and moving image data including these data is stored in the moving image data storage area 122.

<C: Effect of the embodiment>
As described above, according to the present embodiment, the control unit 11 estimates a plurality of subject (sound source) directions, specifies a sound source desired by the user from the estimated plurality of sound source directions, and captures the specified direction. The orientation of the photographing apparatus 1 is changed so as to be included in the range. Thereby, the photographer can track and photograph a specific subject (for example, his / her family, his / her favorite bird, etc.).

  In the present embodiment, since the control unit 11 detects a change in the direction of the sound source, the photographer desires while tracking the movement even when the subject to be photographed moves. It is possible to shoot by including the subject in the shooting range.

  Further, in this embodiment, since the sound source direction is specified using the microphone array MA, it is not necessary to attach a transmitter to the subject or to separately provide a receiver for receiving a signal from the transmitter in the photographing apparatus 1. Thus, a specific subject can be tracked and photographed with a simpler configuration than in the past.

  Further, according to the present embodiment, the sound pressure distribution with respect to the direction is calculated based on the sound pressure correlation for each microphone 15, and the direction in which the sound pressure peak appears in the calculated distribution is estimated as the direction of the sound source. . Since the direction of the sound source is estimated from the sound pressure distribution in this way, the direction of the sound source can be estimated without performing complicated processing. In addition, the processing time required for the sound source direction estimation process can be shortened.

  In addition, according to the present embodiment, direction-specific audio data is generated for each angle of a predetermined unit amount, the generated direction-specific audio data is compared with the verification data stored in the verification data storage area 121, and The direction is specified based on the degree of coincidence. That is, simply by registering the sound of the desired subject in the photographing device 1, the photographing device 1 tracks and records the registered subject, so that the photographer moves the desired subject. There is no need to perform complicated operations such as moving the photographing device 1 or changing the orientation of the photographing device 1 each time.

Incidentally, there are cases where the photographer cannot easily identify the position of the subject to be photographed. For example, if you want to shoot a cicada that is ringing in a tree or if you want to shoot a specific wild bird in the forest, it is difficult to specify where the subject (such as a cicada or a bird) is. In many cases, even if a subject is found, the subject moves quickly and often loses sight of the subject again. In addition, for example, in a children's athletic meet or a school performance, it is often difficult for a photographer to find his or her child to be photographed from a large number of children. There was a case of missing. In the conventional technology, there is a method for recognizing a subject by image analysis at the start of shooting, but for subjects that are difficult to recognize by image analysis (semi-, birds, children among the children wearing the same gym clothes, etc.) Could not be applied.
On the other hand, in this embodiment, since the direction of the subject is specified by collating the voice of the subject, the subject can be tracked and photographed even when the subject is difficult to visually recognize. Note that the frequency characteristics of the microphone 15 may be in a range that exceeds the human audible range, and can be applied to, for example, ultrasonic waves.

<D: Modification>
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. An example is shown below. In addition, you may combine each following aspect suitably.
(1) The sound source direction may be estimated by using independent component analysis. Independent component analysis is a method in which each signal from multiple signal sources is mixed in space and arrives at multiple sensors. The separation is performed without knowing the information of the mixed system of the source signal, and is described in the background art of, for example, Japanese Patent No. 3881367 (Patent Document 3). Further, a technique for obtaining the arrival direction of the signal source described in Patent Document 3 may be used.

(2) The method of generating the direction-specific audio data or the specific direction audio data is not limited to the method described in the above embodiment, and the audio data corresponding to the sound source from the audio data of the microphone 15 using the above-described independent component analysis. You may obtain | require by estimating. Further, the technique described in Patent Document 3 may be used.

(3) The voice feature stored in the verification data may be a personal voice feature (voice print or the like). The control unit 11 may determine the direction of the sound source by analyzing the direction-specific audio data and determining whether or not a specific individual voice feature is detected.

(4) In the above-described embodiment, the control unit 11 changes the orientation of the photographing apparatus 1 so that the specific direction is included in the photographing range, but instead, this notifies the photographer of the specific direction. You may do it. Specifically, for example, the photographing apparatus 1 may output a voice message for guiding a specific direction to the photographing company. For example, a message for notifying the specific direction may be displayed on the display unit 13. Also good. Also, for example, a vibrator that rotates in the horizontal direction is provided inside the photographing apparatus 1, and the specific direction is vibrated by rotating the vibrator in a direction that rotates in the specific direction from the center direction of the photographing range of the photographing apparatus 1. You may make it alert | report to a photographer.

(5) In the above-described embodiment, the control unit 11 drives the motor 723 to change the orientation of the photographing apparatus 1, but the manner of changing the photographing range is not limited to this, and for example, as shown in FIG. As described above, the photographing apparatus 1A is provided with a rotation mechanism for rotating the photographing unit 18A including the photographing lens 18b, the CCD 18c, and the like in the P direction in the drawing, and the photographing range is changed by rotating the photographing unit 18A. You may do it. In the example shown in FIG. 7, the rotating unit 75 is rotatably supported by the photographing apparatus 1 </ b> A by rollers 76 a, 76 b, and 76 c, and the photographing unit 18 </ b> A is fixed to the rotating unit 75. The imaging unit 18 </ b> A rotates with the rotation of the rotation unit 75. The motor 77 rotates the roller 78 under the control of the control unit 11, and the rotating unit 75 rotates with the rotation of the roller 78. Note that the configuration for rotating the imaging unit 18A is not limited to this, and the imaging unit may be rotated using another rotation mechanism.
As described above, as a mode of changing the photographing range of the photographing apparatus, the photographing apparatus main body may be rotated, or the photographing mechanism may be rotated. The photographing apparatus 1 may be controlled so as to change the photographing range so that the direction determined to have sound generation is included in the photographing range.

(6) In the above-described embodiment, the control unit 11 detects the movement of the subject by detecting the transition in the direction in which the peak of the sound pressure appears in the sound pressure distribution. Instead, the control unit 11 However, for each direction of a predetermined unit amount, mixing is performed so that the sound pressure of the sound from each direction becomes high, and direction-specific sound data is generated, and each of the generated direction-specific sound data is stored in the matching data It is possible to collate and detect a change in the direction of the subject (sound source) based on the degree of coincidence. At this time, as the verification data, the same data as the verification data stored in the verification data storage area 121 of the above-described embodiment may be used. That is, the feature data obtained by filtering the voice data to extract the voice features is used as the matching data, and the control unit 11 performs the filtering process on the voice data for each direction to extract and extract the voice features. The feature data representing the feature and the matching data may be collated, and the transition of the direction of a specific subject may be detected based on the degree of coincidence.

(7) In the above-described embodiment, the control unit 11 collates the sound data for each sound source direction with the collation data stored in the collation data storage area 121 and identifies the specific direction based on the degree of coincidence. I did it. Alternatively, the photographer may input the direction of the subject to be photographed by viewing the position of the subject displayed on the display unit 13 by operating the operation unit 14. Specifically, for example, the control unit 11 displays the sound source direction estimated by the sound source direction estimation process on the display unit 13 after the sound source direction estimation process shown in Step S1 of FIG. The photographer may select one of the sound source directions displayed on the display unit 13 by using the operation unit 14. In this case, the control unit 11 specifies the sound source direction selected by the photographer as the specific direction among the estimated sound source directions in accordance with the operation signal from the operation unit 14.

As described above, the control unit 11 may specify the specific direction from the plurality of sound source directions by analyzing the audio data for each sound source direction, and may specify the specific direction according to the operation signal from the operation unit 14. In short, it is only necessary that the control unit 11 specifies at least one of the estimated sound source directions.
Moreover, in the above-mentioned embodiment, although the control part 11 specified one specific direction, you may make it specify two or more specific directions.

  Further, as a specific direction specifying method, for example, the photographer may be able to select an arbitrary direction. In this case, the photographer performs an operation of designating a direction in which sound is desired to be collected using the operation unit 14, and the control unit 11 sets the designated direction as a specific direction according to a signal from the operation unit 14. You may make it produce | generate the specific audio | voice data showing the audio | voice from a direction. Specifically, for example, in a case where a peak cannot be detected in the direction of the subject due to a high ambient noise, the photographing apparatus 1 is more suitably specified by specifying the direction in which the photographer wants to collect sound. Specific direction voice data can be generated.

  In addition, the photographer uses the operation unit 14 to select a mode in which the photographer wants to collect sound and a mode in which the photographing apparatus 1 automatically detects a specific direction as described in the above-described embodiment. You may be able to do it. In this case, the control unit 11 performs specific direction specific processing, specific direction audio data generation processing, and the like according to the selected mode in response to an operation signal from the operation unit 14.

(8) In the above-described embodiment, the imaging apparatus 1 including the microphone array configured by arranging a plurality of microphones 15 as shown in FIG. 2 has been described. However, the arrangement of the microphones 15 is not limited thereto. For example, the microphone 15 may be configured to include a microphone array in which the microphone 15 is arranged in a planar shape (two-dimensional shape). For example, as shown in FIG. 8, it is good also as a structure provided with the microphone array comprised by the three-dimensional arrangement | positioning on the front surface and side surface of the imaging device 1. FIG. In this case, the imaging apparatus 1A can estimate not only the angle in the x-axis direction (see FIG. 8) but also the angle in the y-axis direction and the z-axis direction (see FIG. 8) as the angle of the sound source. The direction of the sound source can be estimated in three dimensions, and a more detailed direction can be estimated. In this case, the sound source can be detected in a wider range.

  In addition, in the case of using an imaging device including a microphone array in which microphones are arranged two-dimensionally or three-dimensionally, in addition to rotating the imaging device in the horizontal direction (arrow P direction in FIG. 8), the vertical direction It is good also as a structure provided with 70 A of rotation mechanisms rotated in the (arrow Q direction in FIG. 8).

  The microphone 15 in the above embodiment may be a small silicon microphone. In addition, a small silicon microphone that can pick up ultrasonic waves in addition to normal sound waves is used, an ultrasonic tag that emits ultrasonic waves is attached to the subject, and the ultrasonic waves that are emitted from the ultrasonic tags are siliconized. The microphone may receive the control unit 11 may determine the direction of the subject based on the ultrasonic waves received by the silicon microphone. In this case, although it is necessary to attach an ultrasonic tag to the subject, a microphone array for recording can be used together as an ultrasonic receiver, so the direction of the subject can be specified with a simple device configuration. it can.

(9) In the above-described embodiment, the storage unit 12 such as a hard disk device is used as the storage unit for storing the moving image data. However, the storage unit for storing the moving image data is not limited to the hard disk device, for example, an SD card, a CD- The recording medium may be a recording medium such as R or CD-R / W. In short, the control unit 11 may record the moving image data on a computer-readable recording medium. Moreover, you may make it the control part 11 output a moving image data to a predetermined | prescribed server apparatus via a communication network.

(10) In the above-described embodiment, the photographing apparatus 1 executes all the processes according to the above-described embodiment. On the other hand, the processing according to the above embodiment is divided and executed by two or more devices connected by a communication network, a communication I / F, and the like, and a system including the plurality of devices is an imaging device according to the embodiment. 1 may be realized. Specifically, for example, a system in which a digital camera and a computer apparatus are connected via a communication I / F such as a USB may be configured.

(11) In the above-described embodiment, the control unit 11 of the photographing apparatus 1 calculates the sound pressure distribution and estimates the angle at which the peak value appears as the sound source direction. The method of estimating the sound source direction is not limited to this. For example, the sound pressure is detected for each predetermined unit amount of angle, and the angle at which the detected sound pressure is equal to or greater than a predetermined threshold may be detected as the sound source direction. In short, the control unit 11 may detect the sound pressure of the audio data output from the microphone 15 for each angle of a predetermined unit amount, and estimate the sound source direction from the sound pressure for each detected angle. .

In the above-described embodiment, the sound source direction is estimated based on the sound pressure of the sound data. However, the present invention is not limited to this, and the frequency characteristic of the sound data for each direction is detected, and the sound source direction is determined based on the detected frequency characteristic. It may be estimated.
Thus, the sound source direction may be detected based on the sound pressure of the audio data, or may be detected based on the frequency. In short, the control unit 11 is output from the microphone 15. Any sound analysis may be used as long as sound data is analyzed and the sound source direction is estimated according to the analysis result.

  Further, the control unit 11 analyzes the video data output from the photographing unit 18 and performs person extraction (or face extraction) processing, and estimates the direction corresponding to the position of the extracted person (or face) as the sound source direction. You may do it. Further, the sound source direction may be estimated by using both the above-described audio analysis and the image analysis. Thus, the accuracy of the sound source estimation process can be increased by using the image analysis result in addition to the voice analysis result.

(12) In the above-described embodiment, the control unit 11 collates the feature data representing the characteristics of the voice data representing the sound from the sound source direction with the collation data stored in the collation data storage area 121, Although the specific direction is specified based on the degree of coincidence, the specific direction specifying method is not limited to this. For example, data representing an image of a specific subject is stored in the verification data storage area 121 in advance as verification data. In addition, the control unit 11 performs image analysis on the video data output from the photographing unit 18, performs person extraction (or face extraction) processing according to the analysis result, and extracts image data of the person (or face). The specific direction may be specified based on the degree of coincidence by collating with the collation data stored in the collation data storage area 121.

  In the above-described embodiment, the control unit 11 performs filtering processing or the like on the sound data representing the sound collected by the microphone 15 to generate feature data representing the sound characteristics, and the generated feature data is used as the matching data. However, the present invention is not limited to this, and sound data representing the sound picked up by the microphone 15 may be used as it is as collation data.

(13) In the above-described embodiment, the control unit 11 may control the photographing unit 18 to focus on the sound source (subject). In this case, for example, the control unit 11 analyzes audio data representing the sound collected by a plurality of different microphones 15 (for example, the microphone 151 and the microphone 15n illustrated in FIG. 2), and the sound is transmitted to each of the plurality of microphones 15. It is also possible to calculate the time difference to reach, calculate the distance between the imaging device 1 and the sound source using the calculated time difference, and perform focus control according to the calculation result.

  In addition, the photographing apparatus 1 may change the direction of the camera to the direction of the sound source, and may automatically zoom in or out according to the approach or distance of the subject. In this case, the control unit 11 calculates the distance between the photographing apparatus 1 and the sound source by the above-described processing, and automatically zooms up (for example, so that the human face is increased) according to the calculated distance. You may control to.

(14) In the above-described embodiment, the example in which the photographing apparatus according to the present invention is applied to a digital camera has been described. However, the apparatus to which the photographing apparatus according to the present invention is applied is not limited to a digital camera, for example, a personal computer, A mobile communication terminal, a computer game machine, etc. may be sufficient and the imaging device concerning this invention is applicable to various apparatuses.

(15) A program executed by the control unit 11 of the photographing apparatus 1 in the above-described embodiment is readable by a computer such as a magnetic tape, a magnetic disk, a flexible disk, an optical recording medium, a magneto-optical recording medium, a RAM, and a ROM. It can be provided in a state of being recorded on a recording medium. It is also possible to download the image capturing apparatus 1 via a network such as the Internet.

It is a block diagram which shows an example of the hardware constitutions of an imaging device. It is a perspective view which shows an example of the external appearance of an imaging device. It is a figure which shows an example of a structure of a rotation mechanism. It is a flowchart which shows the flow of the imaging | photography process which an imaging device performs. It is a figure which shows an example of the sound pressure distribution which a control part calculates. It is a figure which shows an example of the sound pressure distribution which a control part calculates. It is a figure which shows an example of a structure of a rotation mechanism. It is a perspective view which shows an example of the external appearance of an imaging device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Imaging | photography apparatus, 11 ... Control part, 12 ... Memory | storage part, 13 ... Display part, 14 ... Operation part, 15 ... Microphone, 16 ... Audio | voice processing part, 17 ... Speaker, 18 ... Shooting part, 121 ... Data storage for collation Area 122: A moving image data storage area.

Claims (11)

A shooting means for setting a shooting range and outputting video data representing a video in the shooting range;
A plurality of microphones that collect sound for each microphone around the photographing means and output as sound data;
Analyzing audio data output from each of the plurality of microphones, and estimating means for estimating one or a plurality of sound source directions according to the analysis result;
Specifying means for specifying at least one of the directions of the sound sources estimated by the estimating means;
Detecting means for analyzing voice data for each microphone, and detecting a change in direction of a sound source specified by the specifying means according to an analysis result;
An imaging apparatus comprising: an imaging range changing unit that changes the imaging range of the imaging unit to a range including a direction of a sound source detected by the detection unit. The video data storage control means for storing video data output from the photographing means in a predetermined storage means when the photographing range is changed by the photographing range changing means. The imaging device described. The estimation means calculates a sound pressure distribution around the photographing means based on the correlation of the sound data for each microphone, and estimates the direction in which the sound pressure peak appears in the calculated distribution as the direction of the sound source. The photographing apparatus according to claim 1 or 2, characterized in that The photographing range changing means changes the direction of the photographing means so that the direction of the sound source estimated by the direction estimating means is included in the photographing range. The imaging device according to item. The imaging device according to claim 3, wherein the detection means detects a transition of a peak of sound pressure in the distribution of sound pressure calculated by the estimation means. Collation data storage means for storing collation data representing the characteristics of speech;
Direction-specific sound data generating means for generating direction-specific sound data corresponding to each of the directions estimated by the estimation means from the sound data for each microphone; and
The specifying unit collates the direction-specific audio data generated by the direction-specific audio data generation unit with the verification data stored in the verification data storage unit, and determines the direction of the sound source based on the degree of coincidence of the two The imaging device according to claim 1, wherein the imaging device is specified. Collation data generating means for generating, as collation data, voice data corresponding to the direction specified by the specifying means from the voice data for each of the plurality of microphones;
Direction-specific sound data generating means for generating direction-specific sound data corresponding to each of the directions estimated by the estimation means from the sound data for each microphone; and
The detection means collates each direction-specific sound data generated by the direction-specific sound data generation means with matching data generated by the matching data generation means, and based on the degree of coincidence, the direction of the sound source The imaging device according to claim 1, wherein a transition of the image is detected.   8. The photographing apparatus according to claim 6, wherein the collation data includes characteristic information of a specific individual voice. The direction-specific sound data generation means generates direction-specific sound data by mixing each direction from the direction estimated by the estimation means so that the sound pressure from the direction becomes high. The imaging device according to any one of 6 to 8. 9. The direction-specific sound data generating means generates direction-specific sound data by estimating sound data corresponding to a sound source from the sound data using independent component analysis. The imaging device according to item 1.   The imaging apparatus according to claim 1, wherein the estimation unit estimates a direction of a sound source using independent component analysis.

Images