JP2010183252A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus which can obtain an image and a voice. <P>SOLUTION: The imaging apparatus includes an imaging portion taking an image, a voice obtaining portion which can obtain a voice corresponding to an image by changing a directivity, a determining portion determining an object included in the image, and a control portion which controls so that the directivity of the voice obtaining portion may change according to a determination result of the determining portion. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus capable of acquiring an image and sound.

  2. Description of the Related Art Conventionally, in a digital camera, a video camera, and the like, various techniques for controlling the directivity of a microphone according to a shooting situation have been developed, in addition to being able to record an image and sound simultaneously when capturing a moving image.

  For example, Patent Document 1 discloses a technique for recording a sound from a selected desired subject with emphasis or suppression without being affected by the movement of a digital camera or a video camera in a situation where there is ambient sound. Yes.

  However, according to the conventional technology, the directivity of the microphone is directed to the subject based on the focus information such as the shooting distance and the focal length to the subject to which the digital camera or the like is directed. However, the directivity of the microphone cannot be changed in accordance with the type and scene of the subject (hereinafter referred to as “scene”).

  SUMMARY OF THE INVENTION In view of the above-described problems of the related art, an object of the present invention is to provide an imaging apparatus that can acquire images and sounds.

  In order to solve the above-described problems, an imaging apparatus according to the present invention determines an imaging unit that captures an image, a sound acquisition unit that can acquire sound corresponding to the image by changing directivity, and a subject included in the image. And a control unit that controls the directivity of the voice acquisition unit to change according to the determination result of the determination unit.

  In addition, an operation unit that can be operated by the photographer and a shooting condition changing unit that changes the shooting condition according to the state of the operation unit, the control unit is based on the determination result of the determination unit and the state of the operation unit May be controlled.

  Further, the control unit may perform control by giving priority to the determination result of the determination unit over the state of the operation unit.

The determination unit may determine a subject included in the image by performing pattern recognition based on a feature amount obtained from the image. The determination unit uses a plurality of images for a plurality of subjects in advance. And having a plurality of statistical parameters that relate each feature quantity of each of the plurality of images and each target of the plurality of subjects based on a pattern characterizing each target, and imaging by statistical processing using the plurality of statistical parameters The pattern characterizing the subject of the subject in the image captured by the unit may be recognized.

  The statistical processing may be a neural network.

  The voice acquisition unit may include a directivity changing circuit for changing the directivity of the voice.

  The voice acquisition unit may include a plurality of microphones having different directivities, and the control unit may control the gains of the plurality of microphones according to the determination result of the determination unit.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can acquire an image and an audio | voice can be provided.

1 is a schematic diagram showing a configuration example of a digital camera 1 according to a first embodiment of the present invention. The figure and functional block diagram which were seen from the to-be-photographed object side of the digital camera 1 which concerns on the 1st Embodiment of this invention 6 is a flowchart showing the procedure of imaging processing in the digital camera 1 according to the first embodiment of the present invention. 1 is a conceptual diagram of a hierarchical neural network in a digital camera 1 according to a first embodiment of the present invention. FIG. 3 is a conceptual diagram of a hierarchical neural network in a digital camera 1 according to a second embodiment of the present invention. Schematic diagram showing a configuration example of a digital camera 50 according to the third embodiment of the present invention. The figure seen from the to-be-photographed object side and the functional block diagram of the digital camera 50 concerning the 3rd Embodiment of this invention The figure and functional block diagram which were seen from the to-be-photographed object side of the digital camera 1 which concerns on the 4th Embodiment of this invention The flowchart which shows the procedure of the imaging process in the digital camera 1 which concerns on the 4th Embodiment of this invention. The figure and functional block diagram which were seen from the to-be-photographed object side of the digital camera 1 which concerns on the 5th Embodiment of this invention The flowchart which shows the procedure of the imaging process in the digital camera 1 which concerns on the 5th Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, a single-lens reflex digital camera will be described as an example of the camera of the present invention.
<< First Embodiment >>
FIG. 1 is a diagram showing a configuration of a digital camera 1 according to the first embodiment of the present invention. As shown in FIG. 1, the digital camera 1 includes a camera body 1a and an interchangeable lens unit 1b.

  The interchangeable lens unit 1b includes an imaging lens 2, an aperture 3, a focal length detection unit 4, a distance detection unit 5, an aperture control unit 6, and a lens side CPU 7. On the other hand, the camera body 1a includes a pentaprism 8, a re-imaging lens 9, a photometric sensor 10, a quick return mirror 11, a sub mirror 12, a focusing screen 13, a shutter 14, an eyepiece lens 15, an image sensor 16, a display unit 17, and a body side. Each unit includes a CPU 18, a focus detection unit 19, an image processing unit 20, a shutter control unit 21, a storage unit 22, and microphones 23a to 23b. Each component of the digital camera 1 is connected to the lens side CPU 7 of the interchangeable lens unit 1b or the body side CPU 18 of the camera body 1a so as to be able to transmit information. The lens side CPU 7 and the main body side CPU 18 are connected via a connection point. FIG. 1 shows only the main part of the digital camera 1. For example, in FIG. 1, a timing generator that emits timing pulses for shooting instructions to the image sensor 16 in accordance with commands from the main body CPU 18 is omitted.

  The imaging lens 2 is composed of a plurality of optical lenses, and forms an image of a light beam from a subject on the light receiving surface of the imaging element 16.

  The photometric sensor 10 performs photometry by finely dividing the screen of the finder image formed on the focusing screen 13 (for example, dividing into five, etc.) with the quick return mirror 11 lowered as shown in FIG. And photometric information such as brightness and color in the screen is detected. The photometric sensor 10 detects contrast AF information that is a focus state of the imaging lens 2 by a contrast method based on the luminance measured based on the light flux from the imaging lens 2 and is imaged by the re-imaging lens 9. A low-resolution image (through image) displayed on the display unit 17 is generated for composition confirmation before imaging of the subject by the imaging element 16.

  For example, the focus detection unit 19 performs focus detection by a phase difference method, and detects a defocus amount that is a focus state of the imaging lens 2 at a plurality of measurement points of the subject image to be formed. The main body side CPU 18 causes the imaging lens driving unit (not shown) and the diaphragm control unit 6 to drive the imaging lens 2 and the diaphragm 3 via the lens side CPU 7 based on the obtained focus state, respectively. The subject is imaged on the light receiving surface of the image sensor 16. The main body side CPU 18 acquires the focal length, the imaging distance to the subject, and the aperture value at that time from the focal length detection unit 4, the distance detection unit 5, and the aperture control unit 6 via the lens side CPU 7, respectively. The lens side CPU 7 and the main body side CPU 18 can use CPUs of general computers. Each of the lens side CPU 7 and the main body side CPU 18 is provided with a memory (not shown), and a program for controlling each component is recorded in advance.

  Note that whether to perform focus detection of phase difference type focus detection or contrast type focus detection may be determined according to an operation setting by the user. Alternatively, the digital camera 1 may be configured to detect the focus state of the photographing lens 2 by combining phase detection focus detection and contrast focus detection.

  When the photographing is not performed, the quick return mirror 11 is disposed at an angle of 45 ° with respect to the direction of the light beam that has passed through the photographing lens 2 and the diaphragm 3 as shown in FIG. The light beam that has passed through the photographic lens 2 and the diaphragm 3 is reflected by the quick return mirror 11 and forms an image on the focusing screen 13 to project a finder image. The projected finder image is guided to the photometric sensor 10 and the eyepiece 15 via the pentaprism 8. The user confirms the composition by viewing the subject image through the eyepiece 15. At the same time, as described above, the image is re-imaged on the imaging surface of the photometric sensor 10 via the re-imaging lens 9. Further, a part of the light beam that has passed through the quick return mirror 11 is guided to the focus detection unit 19 via the sub mirror 12.

  On the other hand, at the time of shooting, the quick return mirror 11 is retracted from the optical path of the light beam from the imaging lens 2 based on an instruction from the main body side CPU 18, and the shutter 14 is opened by the shutter control unit 21. Are guided to the image sensor 16.

  The image sensor 16 operates based on a timing pulse generated by a timing generator (not shown) in response to a command from the main body side CPU 18 and images a subject imaged by the imaging lens 2 provided in front. As the image sensor 16, a CCD or CMOS semiconductor image sensor or the like can be appropriately selected and used. In the present embodiment, the imaging element 16 performs imaging with all pixels in the case of a still image, and performs imaging by thinning readout in the case of a moving image.

  On the other hand, the camera body 1a of the digital camera 1 is provided with microphones 23a to 23b for acquiring sound when capturing a still image or a moving image. FIG. 2A is a diagram of the digital camera 1 as viewed from the subject side. In addition, the same code | symbol is attached | subjected to the same thing as the component in FIG. As shown in FIG. 2A, the microphones 23a to 23b are provided horizontally in the upper right of the entire surface of the camera body 1a. In this embodiment, as shown in FIG. 2B, the microphone 23a has a narrow directivity characteristic, and the microphone 23b has a wide directivity. FIG. 2B is a functional block diagram of the digital camera 1. The microphones 23a to 23b can be used to pick up ambient sound or a predetermined subject using either one or both of the microphones 23a to 23b depending on the scene of the subject to be captured when the digital camera 1 captures a still image or a moving image. Capture audio effectively.

  The operation member 30 outputs an operation signal corresponding to the content of the member operation by the user to the main body side CPU 18. The operation member 30 includes, for example, operation members such as a power button, a mode setting button such as a shooting mode, and a release button. The operation member 30 may be a touch panel type button provided in front of the screen of the display unit 17. In addition, a liquid crystal monitor or the like can be appropriately selected and used for the display unit 17.

  Image data picked up by the image pickup device 16 is converted into a digital signal by an A / D converter (not shown) and taken into the image processor 20. As shown in FIG. 2B, the image processing unit 20 is a digital front-end circuit composed of an image data processing unit 28 and an image feature amount calculation unit 29. The transferred image data is image data. The data is input to the processing unit 28 and the image feature amount calculation unit 29. The image data processing unit 28 performs image processing such as interpolation processing and white balance processing on the image data, records the image data in the storage unit 22, and displays the image data on the display unit 17 as a still image, a moving image, or the like.

  On the other hand, the image feature amount calculation unit 29 divides the captured image into a plurality of regions such as 50 × 50 or 100 × 100, and image feature amounts such as R / G, B / G, and luminance for each region. And an average value, a variance value, and the like are obtained from the calculated image feature amounts in the respective regions. The image feature quantities calculated by the image feature quantity computing unit 29 such as the average value and the variance value are transferred to the main body CPU 18.

  The main body side CPU 18 determines a scene of the captured subject based on the image feature amount obtained by the image feature amount calculation unit 29, and the scene determined by the scene determination unit 26. The directivity control unit 27 controls the directivity of the microphones 23a to 23b. In the present embodiment, as described above, a scene determination and directivity control unit 27 by the scene determination unit 26 is stored in a memory (not shown) provided in the main body CPU 18 together with a program for controlling each component. It is assumed that a program for performing directivity control by using is recorded in advance.

  In the present embodiment, the A / D converters 24a to 24b are provided with an amplifier circuit in the analog front end, and amplify the analog audio signal transferred from the microphones 23a to 23b and then convert it into a digital signal. Convert. The amplification factors of the amplification circuits of the A / D conversion units 24a to 24b are controlled by the directivity control unit 27 of the main body side CPU 18. The audio signals from the microphones 23 a to 23 b converted into digital signals are combined by the mixer 25 and recorded in the storage unit 22 in association with the image processed by the image data processing unit 28.

  Next, the imaging procedure of the digital camera 1 according to the present embodiment will be described with reference to the flowchart of FIG. In the following description, it is assumed that the user uses the digital camera 1 to capture a moving image. That is, it is assumed that the moving image mode is selected and set in advance by the user using the mode setting button of the operation member 30.

  When the power button of the operation member 30 is pressed by the user, the lens side CPU 7 and the main body side CPU 18 read the control programs stored in the respective memory (not shown), and the camera main body 1a and the interchangeable lens portion 1b of the digital camera 1 are read. Is initialized respectively. The main body side CPU 18 acquires the focal length, the imaging distance, and the aperture value from the focus detection unit 4, the distance detection unit 5, and the aperture control unit 6 via the lens side CPU 7, and from the photometric sensor 10 and the focus detection unit 19. The focus state of the imaging lens 2 is acquired. At the same time, the main body side CPU 18 is used to determine a scene based on a program that controls the directivity of voice recording by the microphones 23a to 23b and an image captured by the scene determination unit 26 according to a scene in capturing a moving image. Data of statistical parameters necessary for statistical processing by the neural network and directivity data which are amplification factors of the A / D conversion units 24a to 24b corresponding to each scene are read from the storage unit 22, respectively. In the present embodiment, it is assumed that statistical parameter data and directivity data are recorded in the storage unit 22 in advance. And the process from step S10 is performed.

  Step S10: When the user presses the release button of the operation member 30 and receives an imaging start instruction signal, the main body side CPU 18 passes the quick return mirror 11 through the imaging lens 2 via a timing generator (not shown). The shutter control unit 21 opens the shutter 14 by retracting from the optical path of the luminous flux from the incoming subject, and at the predetermined frame rate (for example, 30 fps), the imaging device 16 performs moving image imaging by thinning and reading out the subject. Let it begin.

  Step S11: The main body side CPU 18 sequentially outputs the image of each frame acquired at the above frame rate to the image sensor 16 and causes the A / D converter (not shown) to convert it from an analog signal to a digital signal. Transfer to the processing unit 20. The image processing unit 20 inputs the transferred image to the image data processing unit 28 and the image feature amount calculation unit 29. The image data processing unit 28 performs image processing such as interpolation processing and white balance processing on the image. On the other hand, the image feature quantity computing unit 29 divides the image into, for example, 50 × 50 areas, calculates a plurality of image feature quantities in each area, and uses the plurality of image feature quantities obtained in each area. Then, an average value and a variance value of each image feature amount in the image are calculated. The image feature amount calculation unit 29 transfers the calculated average value and variance value of the image feature amounts to the scene determination unit 26 of the main body side CPU 18.

  The plurality of image feature amounts in the present embodiment are R / G, B / G, luminance, motion vector X in the horizontal scanning line (X axis) direction of the image, and vertical scanning line (Y axis) direction of the image. This is a motion vector Y. In the present embodiment, the motion vector X and the motion vector Y are calculated by calculating the correlation for each divided area using the image to be calculated and the image one frame before. However, since the motion vector X and the motion vector Y cannot be obtained for the image of the first frame, their average value and variance value are set to 0, for example.

  Step S12: The scene determination unit 26 of the main body side CPU 18 uses the average and variance values of the plurality of image feature amounts calculated by the image feature amount calculation unit 29 in step S11, and the statistical parameter data read in advance, Statistical processing using a neural network as shown in FIG. 4 is performed, and it is determined whether the scene captured in the image is portrait, landscape, snap, sports, stage shooting, or the like.

  Here, FIG. 4 shows a conceptual diagram of statistical processing by a neural network, particularly a hierarchical neural network. In this hierarchical neural network, an average value and a variance value of each image feature amount calculated in step S11 are input, and a value (for example, a port) indicating the adaptability of each scene is obtained by weighting processing using statistical parameters described later. Rate fitness = 0.8, landscape fitness = 0.2, and other fitness values are all 0). Then, the scene determination unit 26 determines what the scene of the image is according to the value of the fitness value. In this embodiment, the value indicating the fitness is a value normalized so that the sum is 1.

  Next, statistical parameter data will be briefly described. As described above, the statistical parameter associates the average value and variance value of each input image feature amount with each scene based on the fitness obtained by the superimposition process through a statistical process called a neural network. This is a necessary parameter. In order to obtain the statistical parameters, first, an image obtained by imaging a plurality of arbitrary subjects is prepared in advance, and an image feature amount such as R / G in each image is based on the same processing as the image feature amount calculation unit 29. The average value and the variance value are obtained. At the same time, the scene of the subject captured in each image is determined in advance (for example, if the scene of a certain image is a portrait, the portrait adaptability is set to 1, and the adaptation of other scenes is determined. Degree is 0. This is called teacher value). Then, the average value and variance value of each image feature amount of each of the plurality of images and the initial value of the statistical parameter determined in advance are substituted into the hierarchical neural network shown in FIG. Find the fitness of the scene. By comparing each scene fitness obtained in each image with the above teacher value, the statistical parameter is adjusted so that the scene fitness value in each image approaches the teacher value (this is called the learning process) The data of various statistical parameters. The obtained statistical parameter data is recorded in the storage unit 22.

  Step S13: The directivity control unit 27 of the main body side CPU 18 determines whether or not the scene determination by the scene determination unit 26 in step S12 is appropriate. In this embodiment, the directivity control unit 27 has a threshold value of 0.5 or more as the largest value among the fitness values shown in FIG. .2 or more, it is determined that the scene determination by the scene determination unit 26 is appropriate, and the process proceeds to step S14 (YES side). On the other hand, if the largest fitness value is smaller than the threshold value 0.5 or the difference from the second largest fitness value is smaller than the threshold value 0.2, the scene determination by the scene determination unit 26 is not appropriate. It determines with "indefinite", and transfers to step S15 (NO side).

  For example, in step S12, the scene determination unit 26 determines the portrait fitness = 0.8, the landscape fitness = 0.1, the snap fitness = 0.1, and the other fitness is 0 in step S12. In this case, the scene determination unit 26 determines that the scene of the subject is a portrait, and the directivity control unit 27 determines that the determination is appropriate. On the other hand, when the portrait adaptability = 0.5, the landscape adaptability = 0.4, the snap adaptability = 0.1, and the other adaptability is 0, the scene determination unit 26 determines that the subject scene is a port. Although the rate is determined to be a rate, the directivity control unit 27 determines that the determination is not appropriate and determines “undefined”.

  Step S14: The directivity control unit 27 controls each directivity in each of the microphones 23a to 23b based on the result of the scene determination by the scene determination unit 26 and the directivity data. The amplification factor of 24b is determined.

  Here, the directivity data is a list of amplification factors for each of the A / D conversion units 24a to 24b for controlling the microphones 23a to 23b corresponding to each scene. For example, in the case of a portrait scene, since the subject is a person, the amplification factor of the A / D conversion unit 24a and the A / D conversion unit 24b is set to 100 to 0, and only the microphone 23a having a narrow directivity is used. Use to record the voice of that person only. In the case of a landscape scene, the A / D conversion unit 24a and the A / D conversion unit 24b have an amplification factor of 0 to 100, respectively, and ambient sounds are recorded using the microphone 23b having a wide directivity. Further, in the case of a scene of sports or theater shooting, for example, the amplification factor is 70 to 30 respectively, and the voice from the player or the performer is mainly recorded by the microphone 23a having a narrow directivity, and the directivity is wide. The voice of the audience is also recorded with the microphone 23b. Finally, in other cases, for example, the amplification factor is 50 to 50, respectively, and ambient sounds are recorded along with specific sounds.

  Step S15: If the directivity control unit 27 determines that the result of the scene determination by the scene determination unit 26 is “indefinite” in step S13, the amplification factor of each of the A / D conversion units 24a to 24b is set to zero. As 100, sound is recorded only by the microphone 23b having a wide directivity.

  Step S16: The directivity control unit 27 sets the amplification factor determined in step S14 or step S15 in each of the A / D conversion units 24a to 24b, and controls the directivity of the microphones 23a to 23b to record the sound. .

  Step S17: The main body side CPU 18 associates the image processed by the image data processing unit 28 in Step S11 with the audio data recorded by the microphones 23a to 23b in Step S16, and sequentially records them in the storage unit 22.

  Step S18: The main body CPU 18 determines whether or not the release button that the user has pressed is returned to the original position, or the user has pressed the release button again to receive a moving image capturing end command signal. . If it is determined that an image capture end command signal has been received (YES side), the main body side CPU 18 closes the shutter 14 to the shutter control unit 21 via the timing generator (not shown), and opens the quick return mirror 11. Then, the image is returned to the original position on the optical path of the light beam from the imaging lens 2, and a series of moving image imaging is completed. On the other hand, if the main body CPU 18 determines that the moving image imaging end command has not been issued, the process proceeds to step S11 to acquire an image of the next frame, and the processing of steps S11 to S17 is performed by the user. Until a termination command is issued, and a series of moving image capturing is terminated.

As described above, in this embodiment, for each frame image of a moving image captured at a predetermined frame rate, the scene type and scene of the subject to be captured are determined, and the microphone directivity is determined according to the determination result. By controlling automatically, even when the direction of the voice to be recorded changes according to the scene, the voice can be recorded in the best state.
<< Second Embodiment >>
The digital camera according to the second embodiment of the present invention is the same as the digital camera 1 according to the first embodiment. Therefore, the same digital camera 1 as in the first embodiment shown in FIGS. 1 and 2 is used as the digital camera in this embodiment, and detailed description of each component is omitted. Also, the imaging procedure of the digital camera 1 according to the present embodiment is the same as the flowchart of steps S10 to S18 shown in FIG.

  However, the digital camera 1 in the present embodiment is different from the digital camera in the first embodiment in that the number of image feature amount parameters input in the statistical processing by the hierarchical neural network in step S12 increases. It is in. That is, as shown in FIG. 5, each of the image feature values of R / G, B / G, luminance, motion vector X, and motion vector Y obtained from the image by the image feature value calculation unit 29 is input as the image feature value. In addition to the statistical feature values of the average value and the variance value, the imaging lens detected by the phase difference method at a plurality of measurement points by the focal length by the focus detector 4, the shooting distance by the distance detector 5, and the focus detector 19 This is performed by newly adding a dispersion value of the defocus amount (focus detection value) that is the focus state of 2 and a face size parameter in which the face is detected in the captured image.

  With the addition of these image feature amounts, the statistical parameter data stored in advance in the storage unit 22 includes the image feature amount in the first embodiment, the focal length, the shooting distance, What is calculated based on the learning process, which is performed using the variance of the focus detection values and the face size, is used. Further, although the value detected by the phase difference method is used as the focus detection value, a value detected by the hill-climbing method may be used. Furthermore, the method for detecting a face in an image can be performed using any known method.

  As described above, in this embodiment, for each frame image of a moving image captured at a predetermined frame rate, the scene type and scene of the subject to be captured are determined, and the microphone directivity is determined according to the determination result. By controlling automatically, even when the direction of the voice to be recorded changes according to the scene, the voice can be recorded in the best state.

Moreover, the scene determination by the scene determination unit 26 can be performed more accurately by increasing the feature amount input in the hierarchical neural network.
<< Third Embodiment >>
A digital camera 50 according to the third embodiment of the present invention is basically the same as the configuration of the digital camera 1 according to the first or second embodiment, as shown in FIGS. Therefore, in FIGS. 6 and 7 showing the configuration of the digital camera 50 in this embodiment, detailed description of the same components as those of the digital camera 1 according to the first or second embodiment is omitted. The imaging procedure of the digital camera 50 according to the present embodiment is also the same as the flowchart of steps S10 to S18 shown in FIG.

  However, the difference between the digital camera 50 according to the present embodiment and the digital camera 1 according to the first embodiment is that there is only one microphone 23c, as shown in FIGS. Although not shown, the microphone 23c divides the audio band received by the microphone 23c into a plurality of bands by a plurality of band pass filters, and the gain of each divided band is obtained from the directivity control unit 27. A directivity changing circuit that changes the directivity of the microphone 23c by adjusting based on the instruction is provided. Therefore, in this embodiment, together with the statistical parameter data, the amplification factor of the A / D converter 24c corresponding to each scene and the bandpass filter corresponding to each band of the microphone 23c by the directivity changing circuit (not shown). It is assumed that directivity data listing gains is recorded in the storage unit 22 in advance.

  That is, when it is determined in step S13 that the scene determination by the scene determination unit 26 is not “indefinite”, in step S14 (NO side), the directivity control unit 27 stores the scene determination result by the scene determination unit 26 and the storage unit. Based on the directivity data stored in 22, the gain of each bandpass filter corresponding to each band of the microphone 23 c by the amplification factor of the A / D converter 24 c and the directivity changing circuit (not shown) is set. Then, the directivity of the microphone 23c is controlled. On the other hand, if it is determined as “indefinite”, in step S15 (YES side), the directivity control unit 27 sends all the microphones 23c to the directivity changing circuit (not shown) together with the amplification factor of the A / D conversion unit 24c. The gain of each band-pass filter is set so as to use this band, and control is performed so as to obtain the widest directivity.

  As described above, in this embodiment, for each frame image of a moving image captured at a predetermined frame rate, the type and scene of the subject to be photographed are determined, and the directivity of the microphone is automatically set according to the determination result. By controlling automatically, even when the direction of the voice to be recorded changes according to the scene, the voice can be recorded in the best state.

In addition, since only one microphone 23c is required, the circuit scale of the digital camera 50 can be reduced.
<< Fourth Embodiment >>
A digital camera according to the fourth embodiment of the present invention is basically the same as the digital camera 1 according to the first embodiment. However, as shown in FIG. 8B, the configuration of functional blocks in the digital camera 1 in the present embodiment is different from the configuration of functional blocks in the digital camera 1 according to the first embodiment shown in FIG. That is, in the digital camera 1 according to the present embodiment, the directivity control unit 27 is set by the user selecting and setting an imaging mode such as portrait or landscape using the mode setting button of the operation member 30. The amplification factors of the A / D converters 24a to 24b corresponding to the scene corresponding to the imaging mode are determined based on the directivity, and the directivity of the microphones 23a to 23b is controlled. This is a difference from the digital camera of the first embodiment. In the digital camera 1 of the present embodiment and the first embodiment, the same constituent elements are denoted by the same reference numerals and detailed description thereof is omitted.

  Next, the imaging procedure of the digital camera 1 according to the present embodiment will be described with reference to the flowchart of FIG. In the following description, it is assumed that the user uses the digital camera 1 to capture a moving image. That is, it is assumed that the moving image mode is selected and set in advance by the user using the mode setting button of the operation member 30.

  When the power button of the operation member 30 is pressed by the user, the lens side CPU 7 and the main body side CPU 18 read the control programs stored in the respective memory (not shown), and the camera main body 1a and the interchangeable lens portion 1b of the digital camera 1 are read. Is initialized respectively. The main body side CPU 18 acquires the focal length, the imaging distance, and the aperture value from the focus detection unit 4, the distance detection unit 5, and the aperture control unit 6 via the lens side CPU 7, and from the photometric sensor 10 and the focus detection unit 19. The focus state of the imaging lens 2 is acquired. At the same time, the main body side CPU 18 controls each A / A corresponding to each scene together with a program for controlling the directivity of voice recording by the microphones 23a to 23b according to the imaging mode set by the mode selection button of the operation member 30 by the user. Directivity data that is the amplification factors of the D conversion units 24 a to 24 b is read from the storage unit 22. In the present embodiment, it is assumed that the directivity data is recorded in advance in the storage unit 22. In addition, the directivity data in the present embodiment is the same as the directivity data in the first embodiment and the second embodiment. And the process from step S20 is performed.

  Step S20: The user uses the mode selection button of the operation member 30 to select and set the imaging mode.

  Step S21: The main body side CPU 18 identifies the imaging mode set by the user, and the directivity control unit 27 of the main body side CPU 18 determines the A / D conversion units 24a to 24a according to the scene corresponding to the identified imaging mode. The amplification factor of 24b is determined based on the directivity data.

  Step S22: The directivity control unit 27 sets the amplification factor determined in step S21 in the amplification circuits of the A / D conversion units 24a to 24b, and controls the directivity of the microphones 23a to 23b.

  Step S23: When the user presses the release button of the operation member 30 and receives an imaging start instruction signal, the main body side CPU 18 passes the quick return mirror 11 through the imaging lens 2 via a timing generator (not shown). The shutter control unit 21 opens the shutter 14 by retracting from the optical path of the luminous flux from the incoming subject, and at the predetermined frame rate (for example, 30 fps), the imaging device 16 performs moving image imaging by thinning and reading out the subject. Let it begin.

  Step S24: The main body side CPU 18 sequentially outputs the image of each frame acquired at the above frame rate to the image sensor 16 and causes the A / D conversion unit (not shown) to convert the signal from analog to digital. Transfer to the processing unit 20. The image processing unit 20 inputs the transferred image to the image data processing unit 28, and performs image processing such as interpolation processing and white balance processing on the image.

  Step S25: The main body side CPU 18 associates the image processed by the image data processing unit 28 in Step S24 with the audio data recorded by the microphones 23a to 23b whose directivities are controlled in Step S22, and stores the storage unit 22. Record sequentially.

  Step S26: The main body CPU 18 determines whether or not the release button that the user has pressed is returned to the original position, or the user has pressed the release button again to receive a moving image capturing end command signal. . If it is determined that an image capture end command signal has been received (YES side), the main body side CPU 18 closes the shutter 14 to the shutter control unit 21 via the timing generator (not shown), and opens the quick return mirror 11. Then, the image is returned to the original position on the optical path of the light beam from the imaging lens 2, and a series of moving image imaging is completed. On the other hand, if the main body CPU 18 determines that the moving image imaging end command has not been issued, the process proceeds to step S24, an image of the next frame is acquired, and the processes in steps S24 to S25 are performed by the user. Until a termination command is issued, and a series of moving image capturing is terminated.

  As described above, in the present embodiment, by controlling the directivity of the microphone in accordance with the scene of the subject corresponding to the imaging mode set by the user in advance with the mode selection button of the operation member 30, the sound can be heard in the best state. You can record.

In addition, since the directivity of the microphones 23a to 23b is controlled by the imaging mode using the mode selection button of the operation member 30, the circuit scale of the digital camera 1 can be reduced.
<< Fifth Embodiment >>
A digital camera according to the fifth embodiment of the present invention is basically a combination of the digital camera according to the first embodiment shown in FIGS. 1 and 2 and the digital camera according to the fourth embodiment shown in FIG. This is a functional block as shown in FIG.

  Therefore, in the digital camera 1 according to the present embodiment, first, the user selects and sets an imaging mode such as portrait or landscape using the mode setting button of the operation member 30, so that the set imaging mode is obtained. The amplification factors of the A / D converters 24a to 24b corresponding to the corresponding scenes are set, and the directivity of the microphones 23a to 23b is controlled. Thereafter, the digital camera 1 performs scene determination for each image of each frame of the moving image to be captured, appropriately changes the amplification factors of the A / D conversion units 24a to 24b according to the determination result, and The directivity of 23a-23b is controlled. This point is different from the digital camera 1 of the first embodiment and the fourth embodiment. Note that in the digital camera 1 of the present embodiment and the first embodiment, the same components are denoted by the same reference numerals and detailed description thereof is omitted.

  Next, the imaging procedure of the digital camera 1 according to the present embodiment will be described with reference to the flowchart of FIG. In the following description, it is assumed that the user uses the digital camera 1 to capture a moving image. That is, it is assumed that the moving image mode is selected and set in advance by the user using the mode setting button of the operation member 30.

  When the power button of the operation member 30 is pressed by the user, the lens side CPU 7 and the main body side CPU 18 read the control programs stored in the respective memory (not shown), and the camera main body 1a and the interchangeable lens portion 1b of the digital camera 1 are read. Is initialized respectively. The main body side CPU 18 acquires the focal length, the imaging distance, and the aperture value from the focus detection unit 4, the distance detection unit 5, and the aperture control unit 6 via the lens side CPU 7, and from the photometric sensor 10 and the focus detection unit 19. The focus state of the imaging lens 2 is acquired. At the same time, the main body side CPU 18 controls a sound recording directivity by the microphones 23a to 23b according to an imaging mode set by a mode selection button of the operation member 30 by the user or a scene determination by the scene determination unit 26, The statistical parameter data necessary for the statistical processing by the neural network and the amplification factor of each A / D conversion unit 24a to 24b used to determine the scene based on the image captured by the scene determination unit 26 and each scene. Certain directivity data is read from the storage unit 22. In the present embodiment, it is assumed that statistical parameter data and directivity data are recorded in the storage unit 22 in advance. The directivity data in the present embodiment is the same as the directivity data in the first embodiment, the second embodiment, and the fourth embodiment. And the process from step S30 is performed.

  Step S30: The user selects and sets the imaging mode using the mode selection button of the operation member 30.

  Step S31: The main body side CPU 18 identifies the imaging mode set by the user, and the directivity control unit 27 of the main body side CPU 18 determines each A / D conversion unit 24a to 24 according to the scene corresponding to the identified imaging mode. The amplification factor of 24b is determined based on the directivity data.

  Step S32: The directivity control unit 27 sets the amplification factor determined in step S31 in the amplification circuits of the A / D conversion units 24a to 24b, and controls the directivity of the microphones 23a to 23b.

  Step S33: When the user presses the release button of the operation member 30 and receives an imaging start instruction signal, the main body side CPU 18 passes the imaging lens 2 through the quick return mirror 11 via a timing generator (not shown). The shutter control unit 21 opens the shutter 14 by retracting from the optical path of the luminous flux from the incoming subject, and at the predetermined frame rate (for example, 30 fps), the imaging device 16 performs moving image imaging by thinning and reading out the subject. Let it begin.

  Step S34: The main body side CPU 18 sequentially outputs the image of each frame acquired at the above frame rate to the image sensor 16 and causes the A / D conversion unit (not shown) to convert the signal from analog to digital. Transfer to the processing unit 20. The image processing unit 20 inputs the transferred image to the image data processing unit 28 and the image feature amount calculation unit 29. The image data processing unit 28 performs image processing such as interpolation processing and white balance processing on the image. On the other hand, the image feature amount calculation unit 29 divides the image into, for example, 50 × 50 regions, and calculates image feature amounts of R / G, B / G, luminance, motion vector X, and motion vector Y in each region. The average value and the variance value of each image feature amount in the image are calculated from those image feature amounts calculated and obtained in each region. The image feature amount calculation unit 29 transfers the average value and the variance value of the image feature amounts to the scene determination unit 26 of the main body side CPU 18.

  Step S35: The scene determination unit 26 of the main body CPU 18 uses the average value and variance value of the plurality of image feature amounts calculated by the image feature amount calculation unit 29 in step S34, and the statistical parameter data read in advance, Statistical processing using a neural network as shown in FIG. 4 is performed, and it is determined whether the scene captured in the image is portrait, landscape, snap, sports, stage shooting, or the like.

  Step S36: The directivity control unit 27 of the main body side CPU 18 determines whether or not the scene determination by the scene determination unit 26 in step S35 is appropriate. In the present embodiment, the directivity control unit 27 has the highest fitness value of the threshold value 0.5 or more among the fitness values shown in FIG. 4 and is three times the second largest fitness value. If it is the above value, it is determined that the scene estimation by the scene determination unit 26 is appropriate, and the process proceeds to step S37 (YES side). On the other hand, when the largest fitness value is smaller than the threshold value 0.5 or smaller than three times the second largest fitness value, the directivity control unit 27 performs scene determination by the scene determination unit 26. Since it is not appropriate, it is determined as “undefined”, and the process proceeds to step S38 (NO side).

  Step S37: The directivity control unit 27 controls the directivity of each of the microphones 23a to 23b based on the result of the scene determination by the scene determination unit 26 and the directivity data. Determine the amplification factor.

  Step S38: When the directivity control unit 27 determines that the result of the scene determination by the scene determination unit 26 is “indefinite” in step S35, the directivity control unit 27 determines to use the amplification factor according to the imaging mode set in step S32. I do.

  Step S39: The directivity control unit 27 sets the amplification factor determined in step S37 or step S38 in each of the A / D conversion units 24a to 24b, and controls the directivity of the microphones 23a to 23b to record the sound. To do.

  Step S40: The main body side CPU 18 associates the image processed by the image data processing unit 28 in step S34 with the audio data recorded by the microphones 23a to 23b in step S39, and sequentially records them in the storage unit 22.

  Step S41: The main body CPU 18 determines whether or not the release button that the user has pressed is returned to the original position, or the user has pressed the release button again to receive a moving image imaging end command signal. . If it is determined that an image capture end command signal has been received (YES side), the main body side CPU 18 closes the shutter 14 to the shutter control unit 21 via the timing generator (not shown), and opens the quick return mirror 11. Then, the image is returned to the original position on the optical path of the light beam from the imaging lens 2, and a series of moving image imaging is completed. On the other hand, if the main body CPU 18 determines that the moving image capturing end command has not been issued, the process proceeds to step S11 to acquire an image of the next frame, and the processing of steps S34 to S40 is performed by the user. Until a termination command is issued, and a series of moving image capturing is terminated.

As described above, in the present embodiment, the directivity of the microphone determined by the imaging mode set in advance by the mode selection button of the operation member 30 by the user, or the subject to be imaged for each frame image of the captured video. By selecting and controlling the directivity of the microphone that is determined according to the type of scene and the scene judgment result, even if the direction of the sound to be recorded changes according to the scene, the sound is recorded in the best condition. be able to.
≪Supplementary items for the embodiment≫
In the first embodiment, the second embodiment, the fourth embodiment, and the fifth embodiment, the positions of the microphones 23a to 23b are as shown in FIGS. 2 (a), 8 (a), and 10 (a). As shown in FIG. 2, when viewed from the subject side, two are arranged horizontally in the upper right of the camera body 1a, but the present invention is not limited to this. A microphone may be arranged at an arbitrary position.

  In the first embodiment, the second embodiment, the fourth embodiment, and the fifth embodiment, sound recording was performed with two microphones 23a to 23b having different directivities. However, the present invention may be performed using three or more microphones having different directivities.

  In the first embodiment, the second embodiment, the fourth embodiment, and the fifth embodiment, the directivity data is an amplification factor for each of the A / D conversion units 24a to 24b according to each scene. For example, in the case of a portrait scene, since the subject is a person, the amplification factors of the A / D conversion unit 24a and the A / D conversion unit 24b are set to 100 to 0, respectively. The present invention is not limited to this, and a combination of amplification factors of the A / D converters 24a to 24b for controlling the directivity of the microphones 23a to 23b is appropriately determined according to the scene. Is preferred.

  In the first embodiment, the second embodiment, and the fifth embodiment, the case of capturing a moving image has been described. However, the present invention is not limited to this, and can also be applied to the case of capturing a still image. It is. However, in that case, in order to control the directivity of the microphones 23a to 23b before capturing a still image, the amplification factors of the A / D conversion units 24a to 24b corresponding to the scene determination by the scene determination unit 26 are set. It is preferable to decide in advance. That is, for example, the scene determination unit 26 performs scene determination based on a through image captured by the photometric sensor 10, and directivity control is performed based on the determination result and directivity data stored in the storage unit 22. The unit 27 determines the amplification factors of the A / D conversion units 24a to 24b before capturing a still image.

  In the first to third embodiments and the fifth embodiment, the image feature amount calculation unit 29 divides the frame image into 50 × 50 regions and performs a calculation for obtaining the image feature amount. However, the present invention is not limited to this. For example, the calculation may be performed by dividing the frame image into an arbitrary number of regions such as 100 × 100.

  In the first to third embodiments and the fifth embodiment, the values of the fitness values of the respective scenes, which are the output results of the hierarchical neural network of FIG. 4 or FIG. Although the value normalized to 1 is used, the present invention is not limited to this. For example, each fitness value may be expressed as a percentage or may be expressed as a non-standardized value.

  In the first to third embodiments and the fifth embodiment, 0.5 is used as a reference threshold value for determining whether or not the scene determination unit 26 can determine the scene by the directivity control unit 27. However, the present invention is not limited to this, and any value is preferably selected and used.

  In the first to third embodiments and the fifth embodiment, scene determination is performed based on a hierarchical neural network having 10 or 14 input parameters as shown in FIG. 4 or FIG. Although the unit 26 determines the scene of the image, the present invention is not limited to this, and another image feature amount such as an edge amount may be used as an input parameter.

  In the first to third embodiments and the fifth embodiment, as the fitness of each scene, portrait fitness, landscape fitness, snap fitness, sports fitness, stage shooting fitness, Although six other fitness values are used, the present invention is not limited to this, and other arbitrary scene fitness values may be added.

  In the third embodiment and the fifth embodiment, the scene determination unit 26 calculates the average of R / G, B / G, luminance, motion vector X, and motion vector Y, which are 10 image feature amounts. The scene determination of the image is performed based on the statistical processing by the hierarchical neural network shown in FIG. 4 using the value and the variance as input parameters, but the present invention is not limited to this. For example, the scene determination unit 26 is a focal point that is the focal state of the imaging lens 2 detected by the phase difference method at a plurality of measurement points by the focal point detection unit 4, the focal length by the focal point detection unit 4, the photographing distance by the distance detection unit 5. The scene determination may be performed based on statistical processing by a hierarchical neural network using the 14 input parameters shown in FIG. 5 to which the variance value of the detection values and the face size detected in the frame image are added.

  In the third embodiment, as shown in FIG. 7A, the position of the microphone 23c is arranged at the upper right of the camera body 1a when viewed from the subject side. However, the present invention is not limited to this. . A microphone may be arranged at an arbitrary position.

  In the third embodiment, the combination of the gain value of the band pass filter corresponding to each band of the microphone 23c by the amplification factor and directivity changing circuit (not shown) of the A / D conversion unit 24c corresponding to each scene. Is preferably determined according to the scene.

  In the third embodiment, the case of capturing a moving image has been described. However, the present invention is not limited to this, and can also be applied to the case of capturing a still image. However, in that case, in order to control the directivity of the microphone 23c before capturing a still image, the amplification factor and directivity changing circuit of the A / D conversion unit 24c according to the scene determination by the scene determination unit 26 It is preferable to determine in advance the gain of the bandpass filter corresponding to each band of the microphone 23c (not shown). That is, for example, the scene determination unit 26 performs scene determination based on a through image captured by the photometric sensor 10, and directivity control is performed based on the determination result and directivity data stored in the storage unit 22. The unit 27 determines the gain of the band pass filter corresponding to each band of the microphone 23c by the amplification factor and directivity changing circuit (not shown) of the A / D conversion unit 24c before capturing a still image.

  In the fourth embodiment, the case of capturing a moving image has been described. However, the present invention is not limited to this, and can also be applied to the case of capturing a still image.

The present invention can also be applied to a still camera, a video camera, a mobile phone with a built-in digital camera, etc. The present invention can be applied in various other forms without departing from the spirit or main features thereof. Can be implemented. For this reason, the above-described embodiment is merely an example in all respects and should not be construed in a limited manner. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Digital camera, 1a Camera body, 1b Interchangeable lens part 1b, 2 Imaging lens, 3 Aperture, 4 Focal length detection part, 5 Distance detection part, 6 Aperture control part, 7 Lens side CPU, 8 Penta prism, 9 Re-imaging Lens, 10 Photometric sensor, 11 Quick return mirror, 12 Sub mirror, 13 Focus plate, 14 Shutter, 15 Eyepiece, 16 Image sensor, 17 Display unit, 18 Main body side CPU, 19 Focus detection unit, 20 Image processing unit, 21 Shutter Control unit, 22 storage unit, 23a-23b microphone

JP 2006-222618 A

Claims (8)

An imaging unit that captures an image;
A sound acquisition unit capable of acquiring sound corresponding to the image by changing directivity;
A determination unit for determining a subject included in the image;
And a control unit that controls the directivity of the voice acquisition unit to change according to the determination result of the determination unit. The imaging apparatus according to claim 1, wherein
An operation unit that can be operated by the photographer;
A shooting condition changing unit that changes shooting conditions according to the state of the operation unit,
The said control part performs control based on the determination result of the said determination part, and the state of the said operation part, The imaging device characterized by the above-mentioned. An imaging apparatus according to claim 2,
The image pickup apparatus, wherein the control unit performs control with priority given to a determination result of the determination unit over a state of the operation unit. An imaging apparatus according to any one of claims 1 to 3, wherein
The said determination part determines the said to-be-photographed object contained in the said image by performing pattern recognition based on the feature-value calculated | required from the said image. The imaging device characterized by the above-mentioned. An imaging apparatus according to claim 4, wherein
The determination unit uses a plurality of images of a plurality of subjects in advance, and associates the feature amount of each of the plurality of images with each target of the plurality of subjects based on a pattern characterizing each target. An imaging apparatus having a plurality of statistical parameters and recognizing a pattern characterizing the subject of the subject in the image captured by the imaging unit by statistical processing using the plurality of statistical parameters.
An imaging apparatus characterized by that. An imaging apparatus according to claim 5, wherein
The image processing apparatus, wherein the statistical processing is a neural network. An imaging apparatus according to any one of claims 1 to 6, comprising:
The imaging apparatus, wherein the voice acquisition unit includes a directivity changing circuit for changing the directivity of the voice. The imaging apparatus according to any one of claims 1 to 6,
The voice acquisition unit includes a plurality of microphones having different directivities,
The image pickup apparatus, wherein the control unit controls gains of the plurality of microphones according to a determination result of the determination unit.

Images