JP4640080B2 - Imaging device - Google Patents

Description

  The present invention relates to a digital camera, and more particularly to a digital camera having a tilt correction function.

  Digital still cameras and digital video cameras are widely used, and it is now possible to see the scenes of presentations at various exhibitions and the scenes of various presentations taken with digital cameras.

  In such a scene, it is rare that the subject can be photographed from the front, and in many cases, the subject is forced to shoot from an oblique position. Then, if you shoot from the front, the screen or stage that should be in a rectangular shape is distorted into a trapezoid, and the far-off performer appears smaller than the closer performer.

  FIG. 3 is a diagram for explaining the relationship between the tilt angle and the image size. In FIG. 3, 30 is an image sensor, 31 is an imaging surface, 32 is an optical axis, 33 is a subject surface, 34 to 36 are subjects, 37 is an image of the subject 35, and 38 is an image of the subject 36.

In FIG. 3, the image sensor is located at 30, and a plane including the image sensor is the image pickup surface 31. The optical axis 32 is a straight line that passes through the center of the imaging device and is perpendicular to the imaging surface 31. The object surface 33 is assumed to be a screen in a presentation or the like. Here, a case where the object surface 33 is inclined with respect to the imaging surface 31 by an angle θ is considered. When subjects 34 to 36 of the same size are arranged on the subject surface 33, the size of the image mapped to the image sensor 30 is inversely proportional to the distance from the image surface 31 to each subject. Assuming that the distance to the subject 35 on the optical axis is R, the angle at which the subject 35 is viewed from the imaging element 30 with reference to the optical axis is α, and the distance from the imaging surface 31 to the subject 35 is L,
L = R + R × tan α × tan θ = R × (1 + tan α × tan θ)
Since the distance R to the subject 35 is (L ÷ R) = (1 + tan α × tan θ) times the distance L to the subject 34, the size of the image 37 of the subject 35 is (1 + tan α) of the image of the subject 34. X tan θ). If the angle at which the subject 36 on the opposite side of the subject 34 with respect to the optical axis is viewed is β, and the distance from the imaging surface 31 to the subject 36 is M,
R = M + M × tan β × tan θ = M × (1 + tan β × tan θ)
Thus, since the distance M to the subject 36 is 1 / (1 + tan β × tan θ) of the distance R to the subject 34, the size of the image 38 of the subject 36 is (1 + tan α × tan θ) times that of the subject 34. As a result, if the subject 35 and the subject 36 are the left and right sides of a rectangular screen, the length of the left side of the image is (1 + tan α × tan θ) × (1 + tan β × tan θ) times the right side. It appears as a trapezoid.

  In order to alleviate such inconvenience, there has been proposed a digital camera having a “tilt correction” function for correcting a rectangular subject so that it appears in a rectangle, and Japanese Patent Laid-Open No. 2003-283916 is introduced as a recent conventional example. FIG. 4 is a block diagram illustrating a simplified configuration of the digital camera described in Patent Document 1. In FIG. In FIG. 4, 10 is an imaging circuit, 11 is a main memory, 13 is a control circuit, 14 is an image processing circuit, 15 is a tilt correction circuit, 18 is a display circuit, 19 is an operation circuit, and 20 is a correction amount calculation circuit.

  The image pickup circuit 11 includes an image pickup device such as a CCD and an A / D converter. The image of incident light is separated into RGB three primary colors, digitized, and written in the main memory 11 as image data in RGB format. The RGB image data is converted into YC format image data suitable for display by the image processing circuit 14 and written back to the main memory 11. The display circuit 18 reads out image data in YC format from the main memory 11 and displays it on a screen such as an LCD panel.

The control circuit 13 monitors the operation circuit 19, and when the tilt correction is instructed, instructs the correction amount calculation circuit 20 to obtain the correction amount. The correction amount calculation circuit 20 reads the image data in the YC format in the main memory 11, detects the edge of the image, and obtains a correction amount that corrects the inclination of the edge horizontally or vertically. The control circuit 13 gives the correction amount acquired by the correction amount calculation circuit 20 to the correction circuit 15. Then, the tilt correction circuit 15 reads the image data from the main memory 11, performs tilt correction according to the given correction amount, and writes it back to the main memory 11. Further, the control circuit 13 supports the display circuit 18 to read the corrected YC data. As a result of this series of operations, an image subjected to tilt correction is displayed.
JP 2003-283916 A

  However, when the conventional digital camera is zoomed to change the angle of view or panned to change the direction of the camera, the tilt may not be corrected correctly.

  In FIG. 3, assuming that the subject 35 is a subject that appears on both ends of the screen of the display device 18, let us consider the case of zooming in without changing the optical axis and the case of panning the optical axis to the left and right.

  When zooming in without changing the optical axis, the above-mentioned correction amount (1 + tan α × tan θ) depends on the angle α at which the subject 35 is viewed with respect to the optical axis 32. If the angle α at which the edge of the image is viewed becomes smaller, the correction amount (1 + tan α × tan θ) at the edge of the image becomes smaller.

  When the optical axis is panned left and right, the correction amount (1 + tan α × tan θ) depends on the angle formed by the imaging surface 31 and the subject surface 33, that is, the tilt angle θ, so the optical axis 32 is panned to the left and the imaging surface When 31 is close to the direction facing the object surface 33, tan θ decreases, so the correction amount decreases. Conversely, when panning to the right, tan θ increases and the correction amount increases. When the field of view changes in this way, the correction amount needs to be updated.

  However, when zooming up, the edges of the screen, stage, etc. go out of sight, and the correction amount calculation circuit 20 may not be able to obtain the correction amount correctly because the information source is lost. Also, if the camera is panned and pointed at the object at the directly opposite position, the image may be distorted due to excessive correction. In such a case, distortion correction can be turned off in the middle of panning the camera, but the problem remains that the image becomes discontinuous before and after turning on and off.

  An imaging apparatus according to the present invention includes an imaging unit, an image processing unit that performs image processing including tilt correction, a correction amount acquisition unit that acquires a correction amount of the tilt correction, and a magnification monitoring unit that monitors an image enlargement ratio. And pan angle monitoring means for monitoring the pan angle of the optical axis, and control means for updating the correction amount in accordance with the enlargement ratio and the pan angle.

  The image pickup apparatus of the present invention has an advantage that correct tilt correction can be performed because the correction amount is updated according to the change in zoom magnification and the pan angle of the optical axis.

  Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS.

  FIG. 1 is a block diagram illustrating the configuration of a digital still camera according to the present invention. In FIG. 1, 10 is an imaging circuit, 11 is a main memory, 13 is a control circuit, 14 is an image processing circuit, 15 is a tilt correction circuit, 16 is an optical zoom device, 17 is a camera shake detection circuit, 18 is a display circuit, and 19 is a display circuit. It is an operation circuit.

  The imaging circuit 10 separates the image of incident light into RGB three primary colors and converts them into digitized RAW data in RGB format, and writes the RAW data in the main memory 11. The image processing circuit 14 reads RAW data from the main memory 11, performs color space conversion to convert the RAW data into YC data, and writes the generated YC data to the main memory 11. The tilt correction circuit 15 reads YC data from the main memory 11, performs tilt correction, and then writes it back to the main memory 11. The display circuit 18 includes an LCD panel, and reads YC data before or after tilt correction from the main memory 11 according to an instruction from the control circuit 13 and displays an image on the LCD panel.

  The optical zoom device 16 includes a zoom lens that optically enlarges or reduces an image generated in the imaging circuit 10 and a rotary encoder that detects the magnification of the optical zoom, and has a function of notifying the control circuit 13 of the optical zoom magnification. . The camera shake detection circuit 17 includes a gyro sensor and an integration circuit, calculates an angle panned by the camera by integrating an angle change amount output from the gyro sensor, and notifies the control circuit 13 of the angle. Since the purpose of the camera shake detection circuit 17 is to track the angle of the optical axis of the image pickup circuit 10, it is desirable that the camera shake detection circuit 17 be mechanically integrated with the image pickup circuit 10.

  The operation circuit 19 is an operation panel provided with switches, and gives instructions to the entire imaging apparatus. The operation circuit 19 includes an on / off switch for turning on / off the tilt correction, an automatic adjustment switch for turning on / off the automatic adjustment of the tilt correction amount, and a cross key. When the on / off switch is turned on, it is in a manual mode in which the correction amount is manually adjusted in the initial state. By operating the cross key, the horizontal and vertical tilt correction amounts can be adjusted. When the automatic adjustment switch is pressed, the automatic adjustment mode is set, and the tilt correction amount is automatically adjusted depending on the direction of the camera. In a small camera, if the operation circuit 19 is in a housing integrated with the imaging circuit 10, the optical axis may be shaken during operation. Therefore, if possible, the operation circuit 19 can be separated from the housing containing the imaging circuit 10. It is desirable to do.

  The control circuit 13 monitors the states of the optical zoom device 16, the camera shake detection circuit 17, and the operation circuit 19, and according to instructions given by the operation circuit 19, the imaging circuit 10, the image processing circuit 14, the tilt correction circuit 15, and the display circuit. 18 etc. are controlled. The control circuit 13 also has a role of calculating the tilt correction amount and giving it to the tilt correction circuit 14.

  The tilt correction circuit 14 is a circuit having a digital zoom function, and has a feature that the enlargement / reduction magnification can be continuously changed according to the pixel position in the image. The parameters of the tilt correction amount given to the tilt correction circuit 14 are a value of tan θ in the above-described correction amount formula (1 + tan α × tan θ) and a value half the angle of view. When half of the angle of view is ω, the magnification at both ends of the image is (1 + tan ω × tan θ) times and 1 / (1 + tan ω × tan θ) times the magnification at the center of the image. If the distance from the center of the image to the pixel position is A pixels and the distance from the center of the image to the edge of the image is W pixels, the magnification at the middle pixel position can be calculated as tan α = (A ÷ W) × tan ω. That's fine.

  Since the angle of view is determined by the product of the optical zoom and the digital zoom magnification, the control circuit 13 obtains the product of the optical zoom magnification notified by the optical zoom device 16 and the digital zoom magnification given to the tilt correction circuit 14, and from this, The half value ω of the angle is calculated and given to the tilt correction circuit 14. In the digital camera of the embodiment, the optical zoom is only used from 1 to 3 times, and a predetermined magnification is obtained from the optical zoom and digital zoom from 3 to 12 times, and the angle of view when the zoom magnification is 1 is 54 degrees. It is 19 degrees for 3x zoom and 5 degrees for 12x zoom. Since the necessary amount of tilt correction is determined only by the angle of view regardless of the zoom means, the half value of the angle of view corresponding to the product of the magnification of the optical zoom and the digital zoom may be given to the tilt correction circuit 14 as ω.

  The user sets an initial value of the tilt angle θ while viewing the display circuit 18, and the control circuit 13 updates the value according to the output of the camera shake detection circuit 17. FIG. 2 is a flowchart showing a tilt correction amount control algorithm. The control flow is controlled by the control circuit 13 in accordance with an instruction given by the operation circuit 19, and when the tilt correction on / off switch is pressed in the operation circuit 19, the control circuit 13 starts the flow after step S001.

First, in step S001, initialization processing is performed. In the initialization process, the control circuit 13 sets the tilt angle θ to 0 degree in the tilt correction circuit 14 and instructs the display circuit 18 to display the tilt-corrected image. In this state, the tilt correction circuit 14 is working, but since the correction amount is zero, an image as captured by the camera is displayed by the display circuit 18.
The control circuit 13 monitors the switch group of the operation circuit 19. If the right key of the cross key of the operation circuit 19 is pressed in step S002, the control circuit 13 branches to step S003 and is increased by 0.1 degree. θ is set in the tilt correction circuit 14. Otherwise, the process proceeds to step S004, and if the left key is pressed, the process branches to step S005, and the tilt angle θ reduced by 0.1 degrees is set in the tilt correction circuit 14. If neither the right key nor the left key is pressed, the process proceeds to step S006, and if the automatic adjustment switch for turning on / off the automatic adjustment of the correction amount is not pressed, the process returns to step S002. A mode in which steps 002 to 006 are repeated is a manual adjustment mode.

  In the manual adjustment mode, by repeatedly monitoring the left and right keys and increasing / decreasing the tilt angle θ, the image displayed on the display circuit 18 is displayed with the right key gradually increasing when the right key is pressed, and the left key is pressed. The left side will be displayed larger and larger. When the left and right keys are operated while the camera is capturing a rectangular object and the object initially displayed in a trapezoid is adjusted to look like a rectangle, the tilt angle is set correctly at θ. If the automatic adjustment switch is pressed here, the process branches from step 006 to step 007 to enter the automatic adjustment mode.

  In the automatic adjustment mode, a value obtained by adding the shake angle d output from the camera shake detection circuit 17 to the tilt angle θ in step 007 is set in the tilt correction circuit 14. The shake angle d is a camera shake detection circuit 17 that detects a change in the camera direction within a unit time, and d = 0 when the camera direction does not change. When the camera is panned, the tilt angle changes, and at the same time, the camera shake detection circuit 17 captures the amount of change d in the camera direction, so that the tilt angle θ can be made to follow the camera direction by the processing in step 007.

  In the next step 008, the on / off switch for tilt correction is monitored. If there is a user instruction and the correction is on, the control circuit 13 instructs the display circuit 18 to display the image after tilt correction and the process returns to step 007. . In the loop from Step 007 to Step 008 and Step 009 and returning to Step 007, the tilt angle θ can be automatically maintained at a correct value by the shake angle output from the camera shake detection circuit 17.

  If the tilt correction is instructed to be turned off in step 008, the process proceeds to step S010, and the control circuit 13 instructs the display circuit 18 to display the image before the tilt correction. In the next step 011, the automatic adjustment switch is monitored. If automatic adjustment is instructed, the process returns to step 007 to automatically update the tilt angle θ.

As described above, in response to the on / off instruction for the tilt correction, only the image displayed on the display circuit 18 is changed to the image data before correction or the image data after correction, and the camera shake detection circuit 17 outputs the image data. By adding the deflection angle d to the tilt angle θ and continuously executing the operation for updating the tilt angle θ set in the tilt correction circuit 14 regardless of the display, the tilt correction is performed even after the tilt correction is temporarily turned off. As soon as is turned on, correct tilt correction can be performed.
In step 011, when automatic adjustment OFF is instructed, the process proceeds to STOP and all the correction operations are terminated.

  As described above, since the digital camera of the present invention has a mechanism that automatically follows the change in the tilt angle, it is possible to continue correct correction even if the camera is panned or zoomed in during shooting. Even if tilt correction is temporarily turned off, there is no need to readjust the correction amount when it is turned back on.

  In the above description, in order to avoid complication, only the operation of correcting the horizontal tilt is described. However, the principle of the vertical tilt and the correction method are the same, and the tilt correction circuit 14 uses the vertical direction. However, the magnification is changed, and the camera shake detection circuit 17 also detects the vertical swing angle. In the manual adjustment mode, the vertical tilt angle is adjusted in accordance with the up / down key instruction, and in the automatic adjustment mode, the vertical tilt is detected. By updating the corners, it is possible to continue to correct the vertical tilt as well as the horizontal tilt.

  The present invention can be applied to an imaging apparatus because correct tilt correction can be performed by updating the correction amount according to the change in zoom magnification and the pan angle of the optical axis. For example, the present invention can be applied to a digital still camera, a video camera, a mobile phone terminal with a camera function, and the like.

The block diagram which shows the structure of the digital camera by this invention Flow chart showing control algorithm for tilt correction amount Schematic diagram for explaining the relationship between tilt angle and image size Block diagram showing the configuration of a conventional digital camera

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image pick-up circuit 11 Main memory 13 Control circuit 14 Image processing circuit 15 Slip correction circuit 16 Optical zoom device 17 Camera shake detection circuit 18 Display circuit 19 Operation circuit 20 Correction amount calculation circuit 30 Image sensor 31 Imaging surface 32 Optical axis 33 Subject surface 34 , 35, 36 Subject 37, 38 Image of subject

Claims (9)

Imaging means;
Image processing means for performing image processing including tilt correction;
Correction amount acquisition means for acquiring a correction amount of the tilt correction;
Pan angle monitoring means for monitoring the pan angle of the optical axis;
Control means for updating the correction amount in accordance with the pan angle;
An imaging apparatus having The imaging apparatus according to claim 1,
The pan angle monitoring means includes a gyro sensor, and the pan angle of the optical axis is acquired by an electronic angle detection means. Imaging means;
Image processing means for performing image processing including tilt correction;
Correction amount acquisition means for acquiring a correction amount of the tilt correction;
Magnification monitoring means for monitoring the magnification of the image;
Control means for updating the correction amount according to the enlargement ratio;
An imaging apparatus having The imaging apparatus according to claim 3,
The image enlargement is performed by an optical zoom unit, and the magnification monitoring unit acquires an optical zoom magnification. The imaging apparatus according to claim 3,
The image enlargement is a method in which a part of an image acquired by the imaging unit is electronically zoomed by the image processing unit, and the magnification monitoring unit acquires an electronic zoom magnification. The imaging apparatus according to claim 3,
The image enlargement is realized by both an optical zoom unit and an electronic zoom unit, and the magnification monitoring unit acquires the product of the optical zoom magnification and the electronic zoom magnification. Imaging means;
Image processing means for performing image processing including tilt correction;
Correction amount acquisition means for acquiring a correction amount of the tilt correction;
Magnification monitoring means for monitoring the magnification of the image;
Pan angle monitoring means for monitoring the pan angle of the optical axis;
Control means for updating the correction amount according to the enlargement factor and the pan angle;
An imaging apparatus having The imaging apparatus according to any one of claims 1 to 7,
An imaging apparatus comprising instruction means for instructing execution and stop of tilt correction, wherein the control means updates the correction amount even when tilt correction is stopped. The correction amount acquisition means includes display means for displaying an image;
Provided with operation means for instructing correction amount adjustment,
An image subjected to tilt correction is displayed by the display means, and a correct correction amount is obtained by a procedure for adjusting the correction amount so as to minimize distortion of the image using the operation means.
The imaging device according to claim 1.

Images