JP2015095670A - Imaging apparatus, control method thereof and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To shorten the processing time required for calculating the shake correction amount of each image, when imaging a plurality of shooting targets simultaneously.SOLUTION: Depending on the shake amount detected by a shake detection sensor 8, a shake correction amount arithmetic circuit 9 determines a first shake correction amount for performing shake correction of a first image obtained by a first imaging unit 20a. A shake correction amount conversion circuit 10 determines a second shake correction amount for performing shake correction of a second image obtained from a second imaging unit by performing conversion processing of the first shake correction amount on the basis of the imaging direction and focal length of the first imaging unit, and the imaging direction and focal length of the second imaging unit. A geometrical deformation circuit 11 corrects the first and second images, respectively, depending on the first and second shake correction amounts.

Description

  The present invention relates to an imaging apparatus, a control method thereof, and a control program, and more particularly to an imaging apparatus that generates a high-quality moving image in which camera shake is corrected.

  In general, an imaging device such as a digital camera is often used for recording a landscape of a travel destination, a building, or a person, a child's growth process, a school event, or the like. In such usage of the imaging apparatus, there is a demand for recording a photographer (user) image together with an image of a subject during photographing.

  In order to cope with such a demand, there is an imaging device that simultaneously captures imaging objects that exist in different directions, that is, an imaging device that can simultaneously image a subject and a photographer (Patent Document 1). reference).

  On the other hand, there is known an imaging apparatus that detects a camera shake of a photographer and corrects the camera shake based on the detected camera shake to generate a high-quality moving image. Then, when correcting camera shake, a method of performing image stabilization by physically moving the optical system and the image sensor in accordance with the amount of camera shake, and correcting the camera shake by geometrically converting the captured image by image signal processing. There is a method.

  Further, when detecting camera shake, the imaging device includes a gyro sensor capable of measuring acceleration in a specific direction, and the size of the camera shake (that is, the amount of camera shake) and its period based on the output signal of the gyro sensor. It is done to ask for.

JP 2005-73161 A

  By the way, when correcting camera shake by image signal processing, it is necessary to calculate a camera shake correction amount indicating the size and period of camera shake based on the output signal (detection signal) of the gyro sensor.

  When an imaging apparatus including a plurality of imaging units is used to simultaneously image a plurality of imaging objects that exist in different directions, camera shake correction is performed on each of the images obtained as a result of imaging. However, when the blur correction amount is calculated for each image in accordance with the output signal of the gyro sensor, the time required for the calculation process becomes long, and as a result, image display and the like are delayed.

  Therefore, an object of the present invention is to provide an imaging apparatus, a control method thereof, and a control program capable of reducing the processing time required to calculate the shake correction amount for each of the images when simultaneously imaging a plurality of imaging objects. It is to provide.

  In order to achieve the above object, an imaging apparatus according to the present invention is an imaging apparatus including a plurality of imaging units that capture different imaging objects, and the magnitude and direction of blur applied to the imaging apparatus is defined as a blur amount. According to the blur detection unit to detect and the blur amount detected by the blur detection unit, a blur correction amount for performing blur correction of an image obtained by a specific one of the plurality of imaging units is specified. Obtained as a correction amount, the specific blur correction amount is obtained by the remaining imaging units by performing conversion processing based on the imaging direction and focal length of the specific imaging unit and the imaging direction and focal length of the remaining imaging units. Obtained by the specific imaging unit according to the specific blur correction amount and the blur correction amount calculating means for obtaining the remaining blur correction amount for performing the blur correction of each of the obtained images Thereby correcting an image, and having and an image correction means for correcting an image obtained in the rest of the image pickup unit in accordance with the remaining shake correction amount.

  A control method according to the present invention is a method for controlling an imaging apparatus including a plurality of imaging units that capture different imaging objects, and detects the magnitude and direction of blur applied to the imaging apparatus as a blur amount by a blur detection sensor. And a shake correction amount for performing shake correction of an image obtained by a specific one of the plurality of imaging units according to the shake amount detected by the shake detection sensor. Obtained as a quantity, and the specific blur correction amount is obtained by the remaining imaging unit by performing conversion processing based on the imaging direction and focal length of the specific imaging unit and the imaging direction and focal length of the remaining imaging unit. A blur correction amount calculating step for obtaining a remaining blur correction amount for performing blur correction for each of the images; and the specific imaging unit according to the specific blur correction amount. Thereby correcting the image obtained by Tsu bets, and having and an image correcting step of correcting the image obtained by the remaining of the imaging unit in accordance with the shake correction amount of said remaining.

  The control program according to the present invention is a control program used in an imaging apparatus including a plurality of imaging units that capture different imaging objects, and a blur detection sensor is added to a computer included in the imaging apparatus. A blur detection step for detecting the magnitude and direction of the image as a blur amount, and a blur correction of an image obtained by a specific one of the plurality of imaging units according to the blur amount detected by the blur detection sensor And a specific blur correction amount is converted based on the imaging direction and focal length of the specific imaging unit and the imaging direction and focal length of the remaining imaging units. And calculating the remaining blur correction amount for performing the blur correction of each of the images obtained by the remaining imaging units. And an image for correcting the image obtained by the specific imaging unit according to the specific blur correction amount and correcting the image obtained by the remaining imaging unit according to the remaining blur correction amount. And a correction step.

  According to the present invention, a blur correction amount for correcting an image obtained by the remaining imaging units is obtained using a specific blur correction amount for correcting an image obtained by a specific imaging unit. As a result, the processing time required to calculate the shake correction amount for each image can be shortened. In other words, according to the present invention, a shake correction amount is obtained for one image, and a shake correction amount related to images taken in different directions at the same time is obtained by converting the shake correction amount obtained as a result of calculation. Since it did in this way, the time required to obtain | require all the blurring correction amounts can be shortened.

It is a block diagram which shows the structure about an example of the imaging device by embodiment of this invention. It is a figure for demonstrating the blur amount detection of the camera by the blur amount detection sensor shown in FIG. It is a flowchart for demonstrating the blurring correction process performed with the camera shown in FIG. It is a figure for demonstrating conversion of the correction direction performed by the blurring amount conversion circuit shown in FIG. 2A and 2B are diagrams for explaining image data conversion processing when shift blur occurs in the camera shown in FIG. 1, in which FIG. 1A is a diagram illustrating first image data, and FIG. 2B is a diagram illustrating second image data. FIG. 2A and 2B are diagrams for explaining the relationship between the focal length and the amount of blur correction in the camera shown in FIG. 1, in which FIG. It is a figure which shows the magnitude | size of the blurring correction amount when a focal distance is small.

  Hereinafter, an example of an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an example of an imaging apparatus according to an embodiment of the present invention.

  The illustrated imaging device is, for example, a digital camera (hereinafter simply referred to as a camera), and includes first and second imaging units 20a and 20b. The first imaging unit 20a (specific imaging unit) includes an imaging optical system 1 and an imaging element (for example, a CCD or CMOS image sensor) 3, and the second imaging unit 20b includes an imaging optical system 2 and an imaging element 4. ing. Each of the imaging optical systems 1 and 2 has a photographing lens unit (hereinafter simply referred to as a photographing lens) and a diaphragm mechanism.

  The first and second imaging units 20a and 20b are arranged in a state in which different subjects can be photographed. For example, when the first imaging unit 20a is directed to a subject that is a photographing target, the second imaging unit 20a is directed to a photographer who is a user. As a result, the camera can simultaneously photograph different subjects.

  The image sensors 3 and 4 output electrical signals (analog signals) corresponding to optical images (subject images) incident through the imaging optical systems 1 and 2, respectively. The optical system driving circuit 5 independently controls driving of the photographing optical systems 1 and 2 under the control of the system control unit 15.

  Further, the imaging system drive circuit 6 drives and controls the imaging devices 3 and 4 under the control of the system control unit 15. Hereinafter, the analog signals that are the outputs of the imaging devices 3 and 4 are referred to as a first analog signal and a second analog signal, respectively.

  An electrically erasable / recordable nonvolatile memory 16 is connected to the system control unit 15. As the nonvolatile memory 16, for example, an EEPROM is used. The nonvolatile memory 16 stores constants and programs for operating the system control unit 15.

  The program is a program for executing processing according to a flowchart described later.

  The system control unit 15 controls the entire camera. That is, the system control unit 15 executes a program stored in the above-described nonvolatile memory 16 and performs each process described later. The signal processing circuit 7 performs A / D conversion on the first and second analog signals described above, and then performs predetermined image processing to generate first image data and second image data, respectively. At this time, the signal processing circuit 7 performs, for example, pixel interpolation processing and color conversion processing as predetermined image processing.

  Further, the signal processing circuit 7 performs predetermined arithmetic processing using the first image data and the second image data, respectively. The system control unit 15 controls the optical system driving circuit 5 and the imaging system driving circuit 6 to perform exposure control and autofocus control based on the calculation result by the calculation process. The blur amount detection sensor 8 is a sensor for detecting the blur amount of the camera 100.

  FIG. 2 is a view for explaining the blur amount detection of the camera 100 by the blur amount detection sensor 8 shown in FIG.

  In FIG. 2, the horizontal direction is the X axis and the vertical direction is the Y axis. An imaging direction that is a direction orthogonal to the X axis and the Y axis is taken as a Z axis. In addition, a shake that shifts in parallel (that is, shifts) in parallel with the X axis or the Y axis of the camera 100 is called a shift shake, and a shake that rotates around the X axis, the Y axis, or the Z axis is called an angular shake. The blur amount detection sensor 8 detects the magnitude and period of the shift blur and the angle blur and outputs a blur amount detection signal.

  The blur amount detection signal is given to the blur correction amount calculation circuit 9. The shake correction amount calculation circuit 9 calculates a shake correction amount for correcting the shake amount according to the magnitude and cycle of the shake indicated by the shake amount detection signal. This blur correction amount is given to the blur correction amount conversion circuit 10, and the blur correction amount conversion circuit 10 converts the blur correction amount according to the shooting direction (imaging direction) and the focal length as described later, and converts the blur correction amount. This is the amount of image stabilization.

  As will be described later, the geometric deformation circuit 11 performs a geometric deformation process on the first image data in accordance with a blur correction amount (hereinafter referred to as a first blur correction amount), and thereby performs a first blur correction. Generated image data is generated.

  Further, the geometric deformation circuit 11 performs a geometric deformation process on the second image data based on the converted blur correction amount (hereinafter referred to as a second blur correction amount) to obtain second blur corrected image data. Is generated.

  The first shake corrected image data and the second shake corrected image data are temporarily recorded in an image memory (also simply referred to as a memory) 12. Then, the recording circuit 13 compresses the first shake-corrected image data and the second shake-corrected image data recorded in the memory 12 and records them in a recording medium 31 such as a memory card.

  Further, the first shake corrected image data and the second shake corrected image data are displayed on the display device 21 as a first shake corrected image and a second shake corrected image, respectively.

  Further, the recording circuit 13 reads and decompresses the first shake-corrected image data and the second shake-corrected image data after being recorded recorded on the recording medium 31, and performs the decompression process. The image is displayed on the display device 21 as the second image after blur correction.

  FIG. 3 is a flowchart for explaining blur correction processing performed by the camera 100 shown in FIG. Note that the processing according to the illustrated flowchart is performed under the control of the system control unit 15.

  When the blur correction process is started, the signal processing circuit 7 acquires the first and second analog signals from the first imaging unit 20a and the second imaging unit 20b, respectively, under the control of the system control unit 15 ( Step S301). Then, the signal processing circuit 7 generates first and second image data according to the first and second analog signals, respectively.

  Subsequently, the system control unit 15 detects the direction and magnitude of blurring in the camera 100 using the blur detection sensor 8, and provides a blur amount detection signal indicating the blur amount to the blur correction amount calculation circuit 9 (step S302).

  The blur correction amount calculation circuit 9 calculates a first blur correction amount indicating the blur correction amount for the first image data based on the blur amount (blur size and direction) indicated by the blur amount detection (step S303). ).

  Note that, when calculating the blur correction amount, the blur correction amount may be calculated for each blur direction of the camera 100. For example, the shake correction amount is calculated for the shift shake related to each of the X axis and the Y axis shown in FIG. 2, and the shake correction amount is calculated for the angle shake related to each of the X axis, the Y axis, and the Z axis. May be.

  Subsequently, the geometric deformation circuit 11 performs geometric deformation processing on the first image data in accordance with the first blur correction amount, and performs blur correction on the first image data (step S304). Then, the geometric deformation circuit 11 generates first shake corrected image data.

  Next, the blur correction amount conversion circuit 10 converts the correction direction (that is, the blur direction) in the first blur correction amount according to the shooting direction of the first imaging unit 20a and the second imaging unit 20b (step). S305).

  FIG. 4 is a diagram for explaining correction direction conversion performed by the shake correction amount conversion circuit 10 shown in FIG.

  Now, as shown in FIG. 4, in the camera 100, it is assumed that the imaging directions of the first imaging unit 20a and the second imaging unit 20b are opposite to each other along the Z axis. The X0 axis, the Y0 axis, and the Z0 axis indicate the blur direction of the camera 100.

  When the shooting direction is the positive direction of the Z axis, the blur direction of the first imaging unit 20a is set as the X1, Y1, and Z1 axes. Similarly, when the shooting direction is the positive direction of the Z axis, the blur direction of the second imaging unit 20b is set to the X2, Y2, and Z2 axes.

  If the blur direction of the camera 100 is a shift blur in the positive direction on the Y0 axis, the blur directions of the first imaging unit 20a and the second imaging unit 20b are both the same and positive on the Y1 axis and the Y2 axis, respectively. It will shift in the direction.

  On the other hand, if the blur direction of the camera 100 is a positive shift in the X0 axis, the first imaging unit 20a shifts in the positive direction in the X1 axis, but the second imaging unit 20b is negative in the X2 axis. Shift blur in the direction of.

  FIG. 5 is a diagram for explaining image data conversion processing when shift blur occurs in the camera 100 shown in FIG. 1. FIG. 5A shows the first image data, and FIG. 5B shows the second image data.

  As described above, assuming that shift blur occurs on the Y1 axis, the Y2 axis, the X1 axis, and the X2 axis, the blur correction amount in the first image data (first image) shown in FIG. Since the illustrated blur directions are the X-axis direction and the Y-axis direction, the blur correction amount calculation circuit 9 sets the blur correction directions indicated by the first blur correction amount as the −X axis and Y axis directions.

  On the other hand, the shake correction amount conversion circuit 10 receives the first shake correction amount, and the shake correction directions are the −X axis direction and the Y axis direction. The correction direction is changed (that is, determined) to the X-axis direction and the Y-axis direction (see FIG. 5B).

  That is, the blur correction amount conversion circuit 10 performs a conversion process for inverting the blur correction direction of the second image data with the blur correction direction of the first image data for the shift blur in the X-axis direction. On the other hand, the blur correction amount conversion circuit 10 does not perform the blur correction direction conversion process for the shift blur in the Y-axis direction, assuming that the blur correction direction of the second image data is the same as that of the first image data.

  When the camera 100 shake is an angle shake, if the angle shake is an angle shake centered on the Y0 axis, the shake directions of the first imaging unit 20a and the second imaging unit 20b coincide. On the other hand, when the angle blur is an angle blur centered on the X0 axis or an angle blur centered on the Z axis, the blur direction of the first imaging unit 20a and the second imaging unit 20b is opposite.

  Therefore, also in the case of angle blurring, the blur correction amount conversion circuit 10 performs conversion in which the blur correction direction of the second image data is opposite to the blur correction direction of the first image data in the same manner as the shift blur conversion. Process.

  Referring to FIG. 3 again, the blur correction amount conversion circuit 10 corrects the first blur correction amount (blur amount) according to the focal lengths of the first imaging unit 20a and the second imaging unit 20b. Is converted (step S306).

  Although the signal line is not shown, the blur correction amount conversion circuit 10 is given a focal length which is a result of autofocus control of the first imaging unit 20a and the second imaging unit 20b from the system control unit 15. In the following description, the focal length of the first imaging unit 20a is referred to as a first focal length, and the focal length of the second imaging unit 20b is referred to as a second focal length.

  FIG. 6 is a diagram for explaining the relationship between the focal length and the amount of blur correction in the camera 100 shown in FIG. FIG. 6A is a diagram illustrating the magnitude of the blur correction amount when the focal length is large, and FIG. 6B is a diagram illustrating the magnitude of the blur correction amount when the focal length is small.

  In the camera 100, the shootable range (hereinafter referred to as an angle of view) changes depending on the size of the focal length, and the angle of view decreases as the focal length increases. Further, as the angle of view decreases, the influence of camera 100 blur increases, and as a result, the blur correction amount increases.

  In the example shown in FIG. 5A, since the focal length is large, the angle of view is narrowed, and the amount of blur correction corresponding to the blur of the camera 100 is large. On the other hand, in the example shown in FIG. 5B, since the focal length is small, the angle of view becomes wide, and the blur correction amount corresponding to the blur of the camera 100 becomes small. That is, the blur correction amount increases in proportion to the focal length.

  The blur correction amount conversion circuit 10 multiplies the ratio of the first focal length and the second focal length by the blur correction amount (that is, the first blur correction amount in this case) to obtain the second imaging unit 20b. The amount of the shake correction amount according to the above is converted (the amount of the shake correction amount after the change is converted). Then, the blur correction amount conversion circuit 10 generates a second blur correction amount (magnitude and direction) from the first blur correction amount (magnitude and direction) by the processes of steps S305 and S306.

  Subsequently, the geometric deformation circuit 11 performs geometric deformation processing on the second image data in accordance with the second blur correction amount, and performs blur correction on the second image data (step S307). Then, the geometric deformation circuit 11 generates second shake corrected image data and ends the shake correction process.

  The first shake corrected image data and the second shake corrected image data subjected to the shake correction as described above are compressed in the recording circuit 13 and then recorded on the recording medium 31. Further, the display device 21 displays an image corresponding to the first shake corrected image data and the second shake corrected image data.

  In FIG. 3, the process of step S305 may be performed after the process of step S306.

  As described above, in the embodiment of the present invention, the first blur correction amount indicating the blur correction amount of the first imaging unit 20a is used, and the second blur correction amount indicating the blur correction amount of the second imaging unit 20b is used. A correction amount is calculated. Thereby, when simultaneously imaging a plurality of shooting objects, it is possible to shorten the processing time required for calculating the blur correction amount for each of the images.

  As is clear from the above description, in the example shown in FIG. 1, the system control unit 15, the blur correction amount calculation circuit 9, and the blur correction amount conversion circuit 10 function as blur correction amount calculation means. Further, the system control unit 15, the signal processing circuit 7, and the geometric deformation circuit 11 function as an image correction unit.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also contained in this invention. .

  For example, the function of the above embodiment may be used as a control method, and this control method may be executed by the imaging apparatus. Further, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the imaging apparatus. The control program is recorded on a computer-readable recording medium, for example.

  Each of the above control method and control program has at least a shake detection step, a shake correction amount calculation step, and an image correction step.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed.

7 Signal processing circuit 8 Shake detection sensor 9 Shake correction amount calculation circuit 10 Shake correction amount conversion circuit 11 Geometric deformation circuit 12 Memory (image memory)
DESCRIPTION OF SYMBOLS 15 System control part 16 Non-volatile memory 20a, 20b Imaging unit 21 Display part

Claims (6)

An imaging apparatus comprising a plurality of imaging units that capture different imaging objects,
Blur detection means for detecting the magnitude and direction of blur applied to the imaging device as a blur amount;
According to the blur amount detected by the blur detection unit, a blur correction amount for performing blur correction of an image obtained by a specific one of the plurality of imaging units is obtained as a specific blur correction amount, and the specific The blur correction amount of each of the images obtained by the remaining imaging units by converting the blur correction amount based on the imaging direction and focal length of the specific imaging unit and the imaging direction and focal length of the remaining imaging units. Blur correction amount calculating means for obtaining the remaining blur correction amount for performing
Image correcting means for correcting an image obtained by the specific imaging unit according to the specific blur correction amount and correcting an image obtained by the remaining imaging unit according to the remaining blur correction amount; ,
An imaging device comprising: The plurality of imaging units are a first imaging unit and a second imaging unit, wherein the first imaging unit is the specific imaging unit, and the second imaging unit is the remaining imaging unit. Has been
The blur correction amount calculation means is a blur correction direction in the first blur correction amount that is the specific blur correction amount based on the imaging direction of the first imaging unit and the imaging direction of the second imaging unit. The imaging apparatus according to claim 1, wherein the changed blur correction direction in which the change is made is a blur correction direction in a second blur correction amount that is the remaining blur correction amount.   The blur correction amount calculating unit changes a magnitude of blur correction in the first blur correction amount according to a ratio between a focal length of the first imaging unit and a focal length of the second imaging unit. The imaging apparatus according to claim 2, wherein the magnitude of the shake correction after the change is the magnitude of the shake correction in the second shake correction amount.   The blur correction amount calculating means obtains a blur direction according to the shooting direction of the second imaging unit with respect to the shooting direction of the first imaging unit, and uses the second blur correction amount based on the blur direction. The imaging apparatus according to claim 2, wherein a blur correction direction is determined. A method for controlling an imaging apparatus including a plurality of imaging units that capture different imaging objects,
A blur detection step of detecting the amount and direction of blur applied to the imaging device by a blur detection sensor as a blur amount;
According to the blur amount detected by the blur detection sensor, a blur correction amount for performing blur correction of an image obtained by a specific one of the plurality of imaging units is obtained as a specific blur correction amount, and the specific The blur correction amount of each of the images obtained by the remaining imaging units by converting the blur correction amount based on the imaging direction and focal length of the specific imaging unit and the imaging direction and focal length of the remaining imaging units. A blur correction amount calculating step for obtaining a remaining blur correction amount for performing
An image correction step of correcting an image obtained by the specific imaging unit according to the specific blur correction amount and correcting an image obtained by the remaining imaging unit according to the remaining blur correction amount; ,
A control method characterized by comprising: A control program used in an imaging apparatus including a plurality of imaging units that capture different imaging objects,
In the computer provided in the imaging device,
A blur detection step of detecting the amount and direction of blur applied to the imaging device by a blur detection sensor as a blur amount;
According to the blur amount detected by the blur detection sensor, a blur correction amount for performing blur correction of an image obtained by a specific one of the plurality of imaging units is obtained as a specific blur correction amount, and the specific The blur correction amount of each of the images obtained by the remaining imaging units by converting the blur correction amount based on the imaging direction and focal length of the specific imaging unit and the imaging direction and focal length of the remaining imaging units. A blur correction amount calculating step for obtaining a remaining blur correction amount for performing
An image correction step of correcting an image obtained by the specific imaging unit according to the specific blur correction amount and correcting an image obtained by the remaining imaging unit according to the remaining blur correction amount; ,
A control program characterized by causing

Images