KR20040004297A - Image distortion compensation method and devices for CMOS Image Sensor - Google Patents

Abstract

PURPOSE: A method and apparatus for correcting a distortion of an image obtained using a CMOS image sensor are provided to correct an unstable image by estimating the motion of the image. CONSTITUTION: An apparatus for correcting a distortion of an image obtained by a CMOS image sensor includes a motion detector(1), a frame memory(2), a memory controller, a vertical line memory(4), a distortion correction factor calculator, a vertical interpolator and a horizontal interpolator. The motion detector detects a motion from an input image. The frame memory stores at least one input image. The memory controller calculates a range to be corrected from camera motion information from the motion detector to determine a read address of the frame memory. The vertical line memory stores images of one line. A distortion correction coefficient calculator calculators horizontal and vertical interpolation coefficients of the input image. The vertical interpolator controls sub-pixels in the vertical direction. The horizontal interpolator delays the output of the vertical interpolator by at least one pixel.

Description

CMOS image sensor distortion correction method and device {Image distortion compensation method and devices for CMOS Image Sensor}

CCD and CMOS sensors are two-dimensional sensors that capture video and still images, and play a key role in the construction of electronic cameras. In particular, the CCD sensor is superior to the CMOS sensor in terms of the image quality, but the CMOS image sensor is increasing the proportion of the market due to the shortcomings of the power and the complicated configuration, and recently, the CMOS sensor is also improving in the image quality.

In general video recording cameras using the above-described sensors, images that are unstablely shaken are often captured by vibration caused by external factors of the camera, that is, by hand shake or vehicle mounting. In order to solve such an unstable image, a device for compensating for the movement has been proposed. The motion stabilization device is divided into a motion detector and a motion compensation part.

The motion detection unit is proposed and used a method for predicting the motion of the instrument by a Gyro sensor, etc. and a method for detecting the motion part of the image every frame by the image signal processing. In addition, the image is compensated mechanically or electronically using the detected motion information. A mechanically compensated method is the method of arbitrarily refracting incident light using a refractive lens (active prism). The electronic correction method is mainly divided into the method of controlling the input position of the sensor and the method of controlling the address that is stored in the memory and then read out. The method of correcting the latter using the memory can be miniaturized for applications such as camcorders. It is widely used.

The motion stabilization device is mainly used in camcorders and has been mainly used in cameras based on CCD sensors. Such image stabilization devices are being widely used not only for camcorder cameras but also for small camera systems or mobile phone integrated cameras (aka camera phones) due to the development of miniaturization, and the CMOS sensor is also applied to small camera systems at the same time. It is becoming.

However, unlike the above-described CCD image pickup device, motion correction results for images such as CMOS image sensors show distortion such as horizontal tilt and vertical zoom. The reason for this is that in imaging an image of one frame, the CCD image pickup device captures the image at the same point in time with respect to the entire image, whereas the CMOS image sensor reads according to the position of each pixel. The exposure time of the image is sequentially delayed in order to generate a predetermined time interval between the pixel to be read first on one screen and the pixel to be read later, and the distortion of the image is caused when the camera is moved.

An example of an input image may be illustrated in FIG. 1, and an example of a distorted output image may be illustrated in FIG. 2 to explain the distortion of the image. First, it is assumed that an image of an input camera is shown in FIG. 1. The shape of the photographed subject when the camera moves downward is as shown in a) of FIG. 2. 2b of FIG. 2 is an exemplary view of a photographed image while the camera moves upward. 2 c is an exemplary view of an image photographed while the camera moves to the left. 2) is an exemplary diagram of an image captured while the camera moves to the right.

1 is an exemplary view of an input image.

FIG. 2 illustrates a distorted result of the image of FIG. 1.

2 a) is an illustration of an image when the camera moves downward,

2 b) is an example of the image when the camera moves upward,

2 c) is an illustration of an image when the camera moves to the left,

2) is an example of an image when the camera moves to the right.

3 is a block diagram showing a basic configuration of the present invention.

The present invention relates to a method and apparatus for compensating for image stabilization and horizontal tilt and vertical zoom in response to the CMOS image sensor image according to the speed of motion and the amount of change.

First, the configuration of the existing motion stabilization method will be described.

First, the motion detector inter-frame motion vector from the input imagevIs detected and the inter-frame motion vectorvTo Every framenDisplacement vector to accumulate and actually correctuCalculate This displacement vector is expressed as follows.

The image compensator controls the reading position of the image of the previous frame in the memory with reference to the displacement vector to compensate for the movement. Also, to correct more precisely, the pixel position in decimal unit is corrected. This is made possible by interpolating horizontal and vertical adjacent pixels.

In the above motion correction method, since the displacement vector is updated every frame period, the position correction vector is determined in the same way for the image in the frame. As a result, spatial distortion of the image due to time difference is inevitably output.

In order to solve the above problem, the motion caused by the time difference between the starting point and the ending point of the screen should be compensated even within the screen. This is the velocity vector between framesvThe position matrix within the screen p Compensate the position by the time difference by dot product. To compensate for this, the interpolation signal must be processed with the pixel-to-pixel value. Therefore, the horizontal register and the vertical line memory are added separately. If the above expression is expressed as an equation, the displacement vector to be corrected for each pixelu 'Is as follows.

Where the position matrix is

In addition, X, Y means the maximum number of horizontal pixels and the maximum number of vertical lines of the image, x, y means the horizontal vertical position with respect to the currently represented pixel, respectively, displacement vector u ' to be corrected for each pixel Not only n but also a function of the pixel position ( x, y ).

When the above contents are shown in a flowchart, they may be represented as shown in FIG. 3, and the configuration thereof is as follows.

1. Motion detector for detecting motion from the input of the image

2. Frame memory for storing at least one input video

3. The memory control unit which calculates the displacement to be corrected by the motion information of the camera from the motion detection unit to determine the read address of the frame memory.

4. Vertical line memory to store one line of image for interpolating each position in the vertical direction in sub-pixel units

5. Distortion correction coefficient calculation unit for calculating the interpolation coefficient in the vertical and horizontal directions according to the position of the image on the screen

6. Vertical interpolation unit for controlling the sub-pixel in the vertical direction by the output of the frame memory and the output of the vertical line memory

7. Horizontal interpolation unit for interpolating the output of the vertical interpolation unit by one pixel or more in the horizontal direction

In a CMOS sensor camera, when a moving image by a change in azimuth is viewed as a still image or a shake correction image, it is distorted according to the speed in the horizontal and vertical directions. When shooting a general video, the flow is so fast that humans cannot feel the distortion of the screen, but when viewed with still images or when image stabilization is performed, the distortion of the image is greatly increased, which is especially problematic when using a CMOS image sensor. It is becoming.

According to the present invention, a stable image output can be performed by estimating the motion of such an unstable image and correcting it not only between frames but also within a frame, thereby converting the still image or image stabilization image into a spatial distortion-free image.

In particular, the higher the resolution of the screen camera, the more effective the function is indispensable for CMOS images of more than megapixels.

Claims (3)

Apparatus and method for correcting the distortion of an image due to camera shake by internally correcting the spatial position matrix of the image and the motion vector of the camera in a camera using a CMOS sensor In order to eliminate distortion of an image due to camera shake, a motion detector which detects motion from the input of the image, a frame memory capable of storing at least one input image, and motion information of the camera from the motion detector are corrected. A memory control unit for calculating a displacement and determining a reading address of the frame memory, a vertical line memory for storing one line of image for interpolating each position in the vertical direction from the output of the memory in units of subpixels, A distortion correction coefficient calculating unit that calculates interpolation coefficients in the vertical direction and the horizontal direction according to the spatial position, a vertical interpolation unit for controlling sub-pixels in the vertical direction by the output of the frame memory and the output of the vertical line memory, and the vertical direction Horizontal direction to interpolate horizontally by delaying the output of interpolation more than one pixel Apparatus and method which is characterized by consisting of a shaft. The method according to claim 2, wherein the vertical and horizontal distortion correction coefficients are calculated according to the spatial position of the image inside the spatial position matrix of the image and the motion vector between the frames. .