JP7245060B2 - IMAGING DEVICE AND CONTROL METHOD THEREOF, PROGRAM, STORAGE MEDIUM - Google Patents

Description

The present invention relates to an imaging apparatus that determines an imaging environment and performs imaging.

2. Description of the Related Art Conventionally, there have been proposed imaging apparatuses that perform imaging by determining an appropriate imaging mode or appropriate imaging timing for various imaging environments.

For example, in a shooting environment such as a golf course or a concert hall, the sound of the mechanical shutter due to the shooting operation is prohibited when the surroundings are quiet. Therefore, the user needs to shoot in the silent shooting mode. In the silent shooting mode, shooting is performed not by the mechanical shutter but by the electronic shutter, so even in the above situation, shooting can be performed at the timing intended by the user.

In addition, in a shooting environment where the subject is irregularly illuminated by spotlights, such as a wedding ceremony or a light-up event at night, even if the user instructs shooting at the intended timing, the shooting timing based on the instruction will be taken. The spotlight does not necessarily hit the subject. In that case, desired brightness cannot be obtained. Therefore, in such a case, the automatic shooting mode of the camera is used. In the automatic photographing mode, for example, the camera can determine the brightness of the subject and automatically photograph the subject when predetermined brightness conditions are met.

In addition to this, there are the following examples in which the camera judges the shooting timing without being limited to the luminance judgment. Japanese Patent Application Laid-Open No. 2002-200000 discloses analyzing sounds collected by a microphone and automatically capturing a picture when a certain emotion is recognized. Japanese Patent Application Laid-Open No. 2002-200000 discloses that when recognizing a face and performing automatic photographing, the photographing is performed only when a specific condition is met.

JP 2011-119964 A JP 2008-78951 A

However, in the silent shooting mode and the automatic shooting mode described above, only one of the shooting timing intended by the user and the image quality intended by the user can be satisfied. That is, in the silent shooting mode at a golf course or concert venue, shooting can be performed at the timing intended by the user, but in the electronic shutter shooting, image distortion occurs due to the rolling shutter. In addition, in the automatic shooting mode at a wedding ceremony or a light-up event at night, a photographed image can be obtained with the luminance expected by the user, but the photographing is not performed at the timing intended by the user.

SUMMARY OF THE INVENTION The present invention has been made in view of the problems described above, and an object thereof is to provide an imaging apparatus that enables both the shooting timing intended by the user and the image quality intended by the user in various shooting environments. be.

An imaging apparatus according to the present invention comprises imaging means for imaging a subject, instruction means for instructing an imaging operation by a user, detection means for detecting environmental sounds , and when an imaging operation is instructed by the instruction means, when the volume of the environmental sound detected by the detection means is less than a predetermined level , the imaging operation is performed by the first shutter method in accordance with the timing when the imaging operation is instructed, and then the detected environmental sound and a control means for driving the imaging means so as to perform an imaging operation by a second shutter method with a louder drive sound than the first shutter method in accordance with the timing when the sound volume reaches or exceeds the predetermined value. It is characterized by

According to the present invention, it is possible to provide an imaging apparatus that enables both the timing of imaging intended by the user and the image quality intended by the user in various imaging environments.

1 is a block diagram showing the configuration of an imaging device according to one embodiment of the present invention; FIG. 4 is a flowchart showing the first operation of the imaging device of one embodiment; 4 is a flowchart showing a second operation of the imaging device of one embodiment; 9 is a flowchart showing the third operation of the imaging device of one embodiment;

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In addition, the following embodiments do not limit the invention according to the scope of claims. Although multiple features are described in the embodiments, not all of these multiple features are essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant description is omitted.

FIG. 1 is a block diagram showing the configuration of an imaging device 100 according to one embodiment of the present invention. In FIG. 1, the imaging device 100 has the following configuration.

First, the system control unit 101 is composed of, for example, a CPU (MPU), memory (DRAM, SRAM), etc., and executes various processes (programs) to control each block of the imaging apparatus 100, and to control each block. control data transfer. Further, the system control unit 101 controls each block of the imaging device 100 according to operation signals from the operation unit 102 that receives operations from the user. Note that the system control unit 100 controls the entire imaging apparatus 100 by expanding the control program stored in the ROM 117 into the RAM 119 and executing it. The RAM 119 is also used as a work area for the system controller 101 .

The operation unit 102 includes switches for inputting various operations related to shooting, such as a power button and a shutter button. It also has a mode selection dial, a menu display button, an enter button, cursor keys, a pointing device, a touch panel, and the like. With the mode selection dial, it is possible to set various shooting modes to perform optimal shooting operations according to various environments. For example, golf course shooting mode, concert venue shooting mode, wedding ceremony / night light up event Equipped with a spotlight shooting mode for

A lens 103 is a lens for forming an optical image on the imaging element 106 . The lens 103 includes a focus lens for adjusting the focus of an optical image formed on the image pickup device 106, an aperture for adjusting the amount of light for exposing the image pickup device 106, and a focus of the optical image formed on the image pickup device 106. Equipped with a zoom lens or the like for adjusting the distance.

A mechanical shutter (hereinafter referred to as a shutter) 104 operates to mechanically block light that exposes the image sensor 106 . A shutter control unit 105 receives an instruction from the system control unit 101 and controls the shutter 104 . In the case of still image shooting, the imaging device 106 is exposed and the shutter 104 is controlled to be opened during the period of charge accumulation, and the shutter 104 is controlled to be closed during the period of readout of the accumulated charges. It should be noted that the readout of the accumulated charges may be performed with the shutter 104 closed, or may be performed by electronic shutter control with the shutter 104 open. When the shutter 104 is operated, a driving sound is generated, so it may not be suitable for shooting in a quiet environment such as a golf course or a concert hall. Therefore, in such a case, by performing still image shooting by electronic shutter control, shooting is performed more quietly than when the shutter operation is performed only by the shutter 104 .

The imaging device 106 includes a CMOS sensor and its peripheral circuits, converts an optical image of a subject formed by the lens 103 into an image signal, performs analog/digital conversion, and transmits the signal to the image processing unit 108 . The imaging device 106 has a plurality of driving methods. One is a through image sensor drive that continuously generates images at a predetermined frame rate for framing before photographing. Furthermore, there is a still image sensor drive for capturing a still image for recording in accordance with a shooting instruction from a user. Still image sensor driving includes a mechanical shutter readout mode in which the shutter 104 is closed to read out accumulated charges, and an electronic shutter readout mode in which the accumulated charges are read out by electronic shutter control while the shutter 104 is open. In the electronic shutter readout mode, an image is distorted due to a difference in readout timing between the upper portion and the lower portion of the image sensor 106 . Therefore, when photographing a moving object, photographing in the mechanical shutter readout mode is more advantageous in terms of image quality. The imaging control unit 107 controls the switching of the driving method of the imaging element 106 and the timing of charge accumulation and readout.

The image processing unit 108 performs white balance processing and image quality adjustment processing for adjusting colors, brightness, and the like on the input digital signal based on set values. The image signal processed by the image processing unit 108 is transmitted to the display unit 109 as an image for display, or compressed as an image for recording and transmitted to the image memory 110 . Furthermore, the image processing unit 108 can extract subject information in the image by performing predetermined image analysis processing on the input digital signal. For example, it is possible to detect whether or not there is a person (face) or object in the image, and determine where the subject is in the image. Furthermore, it is possible to detect the brightness of the subject and the specific motion of the subject. The detected subject information is transmitted to the system control unit 101 .

A display unit 109 displays an image signal transmitted from the image processing unit 108, an operation screen (menu screen) for operating the imaging apparatus 100, and the like. The display unit 109 may be any display device such as a liquid crystal display, an organic EL display, electronic paper, or the like. The image memory 110 temporarily stores the compressed image signal encoded for recording and sequentially writes it to the recording unit 111 .

A recording unit 111 records the compressed image signal stored in the image memory 110 on a recording medium. The recording medium may be a recording medium built into the imaging apparatus 100 or a removable recording medium, as long as the compressed image signal generated by the imaging apparatus 100 and various data can be recorded. For example, all types of recording media such as hard disks, optical disks, magneto-optical disks, CD-Rs, DVD-Rs, magnetic tapes, non-volatile semiconductor memories, and flash memories can be used.

The audio input unit 112 collects sounds around the imaging device 100 using, for example, a built-in omnidirectional microphone or an external microphone connected via an audio input terminal, and converts analog/digital It converts and transmits to the audio processing unit 113 . The audio processing unit 113 performs audio-related processing such as identification of the level of the input digital audio signal, optimization processing, and pattern recognition, and transmits the identification result to the system control unit 101 .

Next, the operation of the imaging device 100 of this embodiment will be described. The imaging apparatus 100 of the present embodiment is capable of executing a characteristic operation of realizing both the shooting timing intended by the user and the image quality intended by the user in various shooting environments.

The first operation of the imaging device 100 of this embodiment will be described below. FIG. 2 is a flow chart showing the first operation of the imaging device 100. As shown in FIG. Note that the flow shown in FIG. 2 exemplifies the processing when the golf course photographing mode intended for photographing at the golf course is set.

When the user changes the mode selection dial included in the operation unit 102 to the golf course photographing mode, the system control unit 101 issues an instruction to each block of the imaging device 100 to start operation in the golf course photographing mode.

In S<b>200 , the system control unit 101 starts collecting environmental sounds using the audio input unit 112 and identifying audio signal levels using the audio processing unit 113 .

In S201, the system control unit 101 starts the process of determining from the through image whether or not the subject has swung the golf club. Here, first, the imaging element 106 is driven by the through image sensor driving, images are continuously generated at a predetermined frame rate, and the images are transmitted to the image processing unit 108 . The image processing unit 108 determines whether or not the subject has swung a golf club by performing specific image analysis processing on the input through image. For example, the swing motion pattern of the subject may be stored in advance in the ROM, and the swing determination may be performed by pattern matching processing in which the pattern is compared with the input through-the-lens image. The image processing unit 108 transmits the object swing determination result to the system control unit 101 .

In S202, it is determined whether or not the shutter button included in the operation unit 102 has been pressed by the user (whether or not an imaging operation has been instructed). If so, just wait.

In S203, the system control unit 101 determines whether or not the level of the audio signal identified by the audio processing unit 113 is equal to or higher than a predetermined level. If not, proceed to S205. Here, the predetermined level is a level higher than the sound level of the driving sound generated when the imaging apparatus 100 operates the shutter 104 to capture a still image.

In S204, the system control unit 101 operates the shutter 104 and reads the accumulated charges from the image sensor 106 in the mechanical shutter readout mode. That is, the entire imaging apparatus 100 is operated in the mechanical shutter imaging mode to perform still image imaging processing.

In S205, the image processing unit 108 determines whether or not the swing of the subject has ended. If the swing has ended, the process proceeds to S204, and if the swing has not ended, the process proceeds to S206. As the processing of S205, it may be determined whether or not the timing at which the SW2 is instructed by the user is during the period during which the subject is swinging (referred to as the swing period). In this case, if the instruction timing of SW2 is not included in the swing period, the process proceeds to S204, and if it is included in the swing period, the process proceeds to S206.

In S<b>206 , the system control unit 101 reads the accumulated charges from the image sensor 106 in the electronic shutter readout mode without operating the shutter 104 . That is, the entire imaging apparatus 100 is operated in the electronic shutter imaging mode, and still image imaging processing is performed.

In S207, the system control unit 101 determines whether or not the shutter button included in the operation unit 102 is continuously pressed by the user. If not, the process returns to S202.

In S208, the system control unit 101 determines whether or not the level of the audio signal identified by the audio processing unit 113 is equal to or higher than a predetermined level. If not, proceed to S209.

In S209, the image processing unit 108 determines whether or not the swing of the subject has ended. If the swing has ended, the process proceeds to S204, and if the swing has not ended, the process returns to S207. As in the processing of S205 described above, it may be determined whether or not the swing period has ended as the processing of S209 based on the swing period.

In S210, the system control unit 101 determines whether the mode selection dial included in the operation unit 102 has been changed by the user to a mode other than the golf course shooting mode. If the golf course photographing mode is to be terminated, the process proceeds to S211, and if the golf course photographing mode is not to be terminated, the process returns to S202.

In S211, the system control unit 101 stops the swing determination process started in S201. In S212, the system control unit 101 stops the environmental sound measurement process started in S200, and ends the golf course shooting mode.

Note that if the user continues to press the shutter button after performing still image shooting processing in the mechanical shutter shooting mode in S204, the period during which the environmental sound or the subject swing determination condition is satisfied , continuous shooting may be performed in a mechanical shutter shooting mode.

According to the first operation described above, if the environmental sound is sufficiently louder than the shutter sound of the imaging device 100 at the timing when the user presses the shutter button (when the imaging operation is instructed), If the volume is equal to or higher than a predetermined value), it can be determined that the shutter sound has no effect on the surroundings, and therefore photography is performed in the mechanical shutter photography mode (imaging mode). Thereby, an image without distortion can be obtained. Also, if the user can confirm that the subject has finished swinging the golf club when the user presses the shutter button, it can be determined that the swing is not affected by the shutter sound. You can get an image without

Also, if the ambient sound is low and the subject has not finished swinging at the timing when the user presses the shutter button, it is possible to obtain an image shot quietly in the electronic shutter shooting mode. Then, if the user continues to press the shutter button after that, an image without distortion in the mechanical shutter shooting mode can be obtained at the timing when the environmental sound or the subject swing determination condition is satisfied.

Therefore, it is possible to obtain both an image at the shooting timing intended by the user and an image with the image quality intended by the user in the shooting environment of the golf course. In the example shown in FIG. 2, whether or not to perform electronic shutter photography is determined based on the results of both environmental sound determination and swing determination. It may be configured as follows.

Next, a second operation of the imaging device 100 will be described. FIG. 3 is a flow chart showing the second operation of the imaging device 100. As shown in FIG. Note that the flow shown in FIG. 3 exemplifies processing when a concert venue shooting mode intended for shooting at a concert venue or the like is set.

When the mode selection dial included in the operation unit 102 is changed to the concert shooting mode by the user, the system control unit 101 issues an instruction to each block of the imaging device 100 to start operation in the concert shooting mode.

S200 to S212 are the same as S200 to S212 in FIG. 1 except that the steps S201, S205, S209 and S211 are omitted and S210 is replaced with S213.

In S213, it is determined whether or not the mode selection dial included in the operation unit 102 has been changed to a mode other than the concert shooting mode by the user. return.

This embodiment is the same as the operation without subject swing determination processing in the golf course photographing mode described above. Therefore, according to the present embodiment, if the environmental sound is sufficiently louder than the shutter sound of the imaging device 100 when the user presses the shutter button, it can be determined that the shutter sound does not affect the surroundings. Shoot in the mechanical shutter shooting mode. Thereby, an image without distortion can be obtained.

Also, if the ambient sound is low at the timing when the user presses the shutter button, an image can be obtained quietly shot in the electronic shutter shooting mode. Then, if the user continues to press the shutter button after that, an image without distortion in the mechanical shutter shooting mode can be obtained at the timing when the environmental sound determination condition is satisfied.

Therefore, according to the above-described second operation, it is possible to obtain both an image at the shooting timing intended by the user and an image with the image quality intended by the user in the shooting environment of the concert venue.

Next, a third operation of the imaging device 100 will be described. FIG. 4 is a flow chart showing the third operation of the imaging device 100. As shown in FIG. Note that the flow shown in FIG. 4 exemplifies processing when a spotlight shooting mode intended for shooting at a light-up event or the like is set.

When the mode selection dial included in the operation unit 102 is changed to the spotlight shooting mode by the user, the system control unit 101 issues an instruction to each block of the imaging apparatus 100 to start operation in the spotlight shooting mode.

In S220, the system control unit 101 starts the process of determining which part in the imaging plane is the subject from the through image. Here, first, the imaging element 106 is driven by the through image sensor driving, images are continuously generated at a predetermined frame rate, and the images are transmitted to the image processing unit 108 . The image processing unit 108 determines which part in the imaging plane is the subject by performing specific image analysis processing on the input through image.

In S221, the system control unit 101 starts the process of determining from the through image whether or not the brightness of the subject has reached or exceeded a predetermined value. Here, the image processing unit 108 determines whether the brightness of the subject portion determined in S210 is equal to or higher than a predetermined value.

In S222, the system control unit 101 determines whether or not the shutter button included in the operation unit 102 has been pressed by the user. If the shutter button has been pressed, the process proceeds to S223. stand by.

In S223, the system control unit 101 determines whether or not the determination result of the subject brightness by the image processing unit 108 is equal to or higher than a predetermined level. goes to S225. Here, the predetermined level is a level at which sufficient exposure can be obtained when the imaging apparatus 100 performs still image shooting in basic operation.

In S<b>224 , the system control unit 101 performs proper exposure control according to the determination result of the subject luminance by the image processing unit 108 . For example, the shutter speed, aperture value, ISO sensitivity, etc. are adjusted to predetermined values.

In S225, the system control unit 101 operates the entire imaging apparatus 100 in the low-luminance shooting mode to shoot a still image. The low-luminance shooting mode is a mode in which still image shooting is performed by, for example, ultra-high-sensitivity shooting, image combining shooting, or the like. In the case of ultra-high-sensitivity photography, the image sensor 106 is driven with an ISO sensitivity setting higher than that in the basic operation so that the desired exposure can be obtained at the cost of deteriorating the noise level. In the case of image synthesis shooting, shooting is performed a plurality of times with low-luminance exposure, and the image processing unit 108 aligns and synthesizes a plurality of images so that a desired exposure can be obtained. Image compositing photography can keep the noise level lower than the above-described ultra-high-sensitivity photography.

In S226, the system control unit 101 determines whether or not the shutter button included in the operation unit 102 is continuously pressed by the user. If not, the process returns to S222.

In S227, the system control unit 101 determines whether or not the determination result of the subject luminance by the image processing unit 108 is equal to or higher than a predetermined level. returns to S226.

In S228, the system control unit 101 operates the entire imaging apparatus 100 in the normal-luminance shooting mode, which is the basic operation, and performs still image shooting processing.

In S229, the system control unit 101 determines whether the user has changed the mode selection dial included in the operation unit 102 to a mode other than the spotlight shooting mode. If the spotlight shooting mode is to be ended, the process proceeds to S230, and if the spotlight shooting mode is not to be ended, the process returns to S222.

At S230, the subject brightness measurement process started at S221 is stopped. In S231, the subject determination process started in S220 is stopped, and the spotlight photographing mode ends.

According to the third operation described above, when the user presses the shutter button, if the brightness of the subject is sufficiently bright, an image can be obtained in the normal brightness shooting mode without disadvantages such as noise level and processing time. . Also, if the brightness of the subject is dark at the timing when the user presses the shutter button, it is possible to obtain an image with a desired exposure using the low brightness shooting mode. Then, if the user continues to press the shutter button after that, an image in the normal-luminance shooting mode can also be obtained at the timing when the subject luminance condition is satisfied.

Therefore, in a shooting environment such as a wedding ceremony or a light-up event at night, where the subject is irregularly illuminated by spotlights, an image at the shooting timing intended by the user and an image with the image quality intended by the user can be obtained. It is possible to obtain both

In the flow shown in FIG. 4, exposure control may be performed based on the luminance of a predetermined area in the image in order to more clearly determine whether the subject is illuminated with a spotlight. For example, in the processing of S223, it may be determined whether or not the brightness of the area in the image that is determined to be the main subject area is equal to or higher than a predetermined level. Further, for example, in the process of S223, it may be determined whether or not the luminance difference between the local area and other areas in the screen is equal to or greater than a predetermined level.

Although preferred embodiments of the present invention have been described above, the present invention is not limited to these, and various modifications and changes are possible within the scope of the gist. For example, the golf course shooting mode, the concert venue shooting mode, and the spotlight shooting mode have been mentioned in the above-described embodiments. may be applied to

Further, in the above-described embodiment, an example in which the mechanical shutter and the electronic shutter are separated has been described. may be For example, if the imaging apparatus cannot select the electronic shutter, the operation corresponding to the electronic shutter in the above-described embodiments may be replaced by the electronic front curtain shutter. In general, the driving sound of the electronic front curtain shutter is louder than the electronic shutter, but it is quieter than the driving sound of the mechanical shutter. Any configuration is acceptable.

(Other embodiments)
In addition, the present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads the program. It can also be realized by executing processing. It can also be implemented by a circuit (eg, ASIC) that implements one or more functions.

The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the claims are appended to make public the scope of the invention.

100: imaging device, 101: system control unit, 102: operation unit, 103: lens, 104: shutter, 105: shutter control unit, 106: image sensor, 107: imaging control unit, 108: image processing unit, 109: display Unit 110: Image memory 111: Recording unit 112: Audio input unit 113: Audio processing unit

Claims (10)

imaging means for imaging a subject;
an instruction means for a user to instruct an imaging operation;
detection means for detecting environmental sounds ;
When the imaging operation is instructed by the instructing means, and the volume of the environmental sound detected by the detecting means is less than a predetermined level, imaging is performed by a first shutter method at the timing when the imaging operation is instructed. After that, in accordance with the timing when the volume of the detected environmental sound becomes equal to or higher than the predetermined value, the imaging operation is performed by the second shutter method in which the driving sound is louder than that of the first shutter method . , a control means for driving the imaging means;
An imaging device comprising: If the volume of the environmental sound detected by the detection means is equal to or higher than the predetermined level at the time when the imaging operation is instructed by the instructing means, the control means adjusts the imaging operation to the timing when the imaging operation is instructed. 2. The image pickup apparatus according to claim 1, wherein said image pickup means is driven so as to perform an image pickup operation with said second shutter method. The detection means is capable of detecting a subject, and even if the volume of the environmental sound is less than the predetermined amount, if the subject is not performing a predetermined action , the second shutter method is used to photograph the subject. 3. The image pickup apparatus according to claim 1, wherein the image pickup apparatus performs 4. The imaging apparatus according to claim 1, wherein the first shutter method is an electronic shutter, and the second shutter method is a mechanical shutter. 5. The imaging apparatus according to any one of claims 1 to 4, wherein said detection means is capable of detecting luminance of a subject. Has multiple shooting modes,
When the first shooting mode is set, the imaging operation is instructed when the volume of the environmental sound detected by the detecting means is less than a predetermined level at the time when the imaging operation is instructed by the instructing means. After that, in accordance with the timing when the volume of the detected environmental sound becomes equal to or higher than the predetermined value, the second shutter method, in which the driving sound is louder than that of the first shutter method, is performed. 2 shutter method, and when the second photographing mode is set, when the photographing operation is instructed by the instruction means, if the luminance of the subject is less than the predetermined luminance, low-luminance photographing is performed. 6. The image pickup apparatus according to claim 5 , wherein an image pickup operation is performed in normal-luminance photographing mode in accordance with a timing when the luminance of the subject becomes equal to or higher than the predetermined luminance . 3. The image pickup operation according to the second shutter method is repeated while the user continues to instruct the image pickup operation and the volume of the environmental sound is equal to or higher than the predetermined value. 7. The imaging device according to any one of 6 . A method for controlling an imaging device comprising imaging means for imaging a subject and instruction means for instructing an imaging operation by a user, comprising:
a detection step of detecting environmental sound ;
When the imaging operation is instructed by the instructing means, and the volume of the environmental sound detected in the detecting step is less than a predetermined level, the imaging is performed by the first shutter method at the timing when the imaging operation is instructed. After that, in accordance with the timing when the volume of the detected environmental sound becomes equal to or higher than the predetermined value, the imaging operation is performed by the second shutter method with a driving sound louder than that of the first shutter method . , a control step of driving the imaging means;
A control method for an imaging device, comprising: A program for causing a computer to execute each step of the control method according to claim 8 . A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 8 .

Images