KR20160087684A - Apparatus and method for forming high resolution image using optical image stabilizer - Google Patents

Abstract

An embodiment of the present invention relates to an apparatus for forming a high-resolution image using an anti-shake compensator, wherein the apparatus comprises: an anti-shake compensator for stabilizing an image by detecting the shake of a lens module; and a controller for adjusting an offset of an offset adjustment unit which adjusts an offset of an output signal of a movement sensor included in the anti-shake compensator, thereby obtaining images at each location by moving multiple different locations from a reference location of the lens module, and then forming a high-resolution image by composing images obtained at each location. Thus, a clear, unshaken high-resolution image over a fixed resolution limit that an image sensor has can be formed, and a high-quality wide-angle image can be formed.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a high-resolution image forming apparatus and method using an image stabilization apparatus,

The present invention relates to a high-resolution image forming apparatus and method using an image stabilization apparatus.

Portable electronic devices with built-in camera modules incorporate an Optical Image Stabilizer (OIS) to prevent camera shake. The camera shake correction device actively detects camera shake and moves the lens module built in the camera module to stabilize the image physically by preventing the image from shaking due to camera shake.

Even if the lens module in the camera module shakes due to camera shake due to camera shake, the user can easily take a picture without shaking, and can take a lot of light without blurring at night to take a bright picture.

In general, since the resolution of the image sensor built in the camera module is fixed, it is not easy to obtain a high resolution image that exceeds the limit resolution of the built-in image sensor. Therefore, in order to obtain a high-resolution image, a camera module equipped with a high-resolution image sensor should be used. However, the higher the resolution, the larger the size of the camera module.

The patent documents described in the following prior art documents disclose a system and method for generating a high resolution image by moving optical paths of an image sensor using an image stabilization processor and combining optical samples obtained through each optical path.

US 2014-0125825 A1

An object of the present invention is to provide a high-resolution image processing device capable of forming a clear high-resolution image having a resolution exceeding a fixed limit resolution of an image sensor and capable of forming a high-quality wide- Forming apparatus.

Another problem to be solved by one embodiment of the present invention is to provide a high-resolution image processing device capable of forming a clear high-resolution image having a resolution exceeding a fixed limit resolution of an image sensor and capable of forming a high- And a method for forming an image.

According to an aspect of the present invention, there is provided an apparatus for forming a high-resolution image using an image stabilization device, including: an image stabilization device for stabilizing an image by detecting a shake of the lens module; And an offset adjusting unit adjusting an offset of an output signal of the motion sensor included in the shake correction apparatus to move the lens module from a reference position to a plurality of different positions to obtain images at each position, And forms a high-resolution image.

When the lens module is in the shake correction mode, the controller activates the shake correction function to acquire images without shaking when acquiring images at each position.

When the lens module is in the fixed mode, the controller acquires images at each position without activating the image stabilization function when acquiring images at each position.

When the lens module is in the wide angle photographing mode, the controller moves the lens module by more than a predetermined angle to the left and right at the reference position using the camera shake correction function, To acquire images without shaking, and to synthesize the obtained left and right images to form a wide-angle image.

According to the high-resolution image forming apparatus using the camera shake correction apparatus according to an embodiment of the present invention, it is possible to form a clear high-resolution image without blurring exceeding a fixed limit resolution possessed by the image sensor and to form a high- can do.

The features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings.

Prior to this, terms and words used in the present specification and claims should not be construed in a conventional and dictionary sense, and the inventor may appropriately define the concept of a term in order to best describe its invention The present invention should be construed in accordance with the spirit and scope of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a view showing a high-resolution image forming apparatus using an image stabilization apparatus according to an embodiment of the present invention; FIG.
BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an image stabilizing apparatus.
FIGS. 3A to 3E are diagrams for explaining a method of composing images to form a high-resolution image; FIG.
4 is a diagram illustrating a method of forming a wide-angle image using an image stabilization device according to an embodiment of the present invention.
5A to 5D are diagrams for explaining a method of composing a left-side image and a right-side image to form a wide-angle image.

BRIEF DESCRIPTION OF THE DRAWINGS The objectives, specific advantages and novel features of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.

Prior to that, terms and words used in the present specification and claims should not be construed in a conventional and dictionary sense, and the inventor may properly define the concept of the term in order to best explain its invention Should be construed in accordance with the principles and the meanings and concepts consistent with the technical idea of the present invention.

It should be noted that, in the present specification, the reference numerals are added to the constituent elements of the drawings, and the same constituent elements have the same numerical numbers as much as possible even if they are displayed on different drawings.

Also, "first", "second", "one side". The terms "other" and the like are used to distinguish one element from another, and the element is not limited by the terms.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description of the present invention, detailed description of related arts which may unnecessarily obscure the gist of the present invention will be omitted.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a diagram illustrating a high-resolution image forming apparatus using an image stabilization apparatus according to an embodiment of the present invention.

1, an apparatus for forming high-resolution images using an image stabilization apparatus according to an exemplary embodiment of the present invention includes an image stabilization device for stabilizing an image by detecting a shake of a lens module 116 incorporated in a camera module And an offset adjustment unit 104 for adjusting an offset of an output signal of the gyro sensor 102 which is a motion sensor included in the shake correction apparatus 100, And a controller 120 for moving the module 116 from the reference position to a plurality of different positions to obtain images at each position, and composing the images obtained at the respective positions to form a high resolution image.

In Fig. 1, the shake correction apparatus 100 may be incorporated in a camera (not shown) or may be incorporated in a portable device such as a smart phone equipped with a camera module.

The controller 120 may be an application processor (AP) in a smart phone that controls the operation of the shake correction apparatus 100 when the shake correction apparatus 100 is mounted on the smart phone.

The controller 120 applies a control signal to the OIS controller 110 to cause the OIS controller 110 to apply an offset to the offset adjustment unit 104. However, 104, respectively.

The shake correction apparatus 100 includes a gyro sensor 102, an offset adjustment unit 104, a gyro integrator 106, a gain adjustment unit 108, an OIS controller 110, a voice coil motor (VCM) 112 And a hall sensor 114. The Hall sensor 114, as shown in FIG.

The gyro sensor 102 detects movement of the lens module 116 included in the camera module and outputs angular velocity as a motion sensor. The offset adjustment unit 104 adjusts the DC offset included in the signal output from the gyro sensor 102. The gyro integrator 106 calculates angles by multiplying the angular velocity output from the offset adjuster 104 by time and calculates the moving distance of the lens by multiplying the obtained angle by the focal distance inherent to the lens. The gain adjustment unit 108 adjusts the gain of the signal output from the gyro integrator 106 and outputs the adjusted gain.

The OIS controller 110 is a PID controller that controls the amount of current flowing through the VCM 112 so that the lens module 116 can be moved based on the output signal of the gain adjustment unit 108 and the output signal of the hall sensor 114 .

The VCM 112 serves to move the lens module 116 using a principle that a force is generated when a current flows through a coil in a magnetic field formed by a magnet.

The hall sensor 114 is an element whose voltage varies according to the intensity of a magnetic field. When a magnetic field is applied to a conductor through which a current flows, a hall sensor 114 generates a voltage in a direction perpendicular to the current and the magnetic field. The Hall sensor 114 is proportional to the intensity of the magnetic field due to the movement of the lens module 116 A voltage is generated. Since the voltage value generated at this time is weak, it is amplified and used. The amplified voltage value is input to the OIS controller 110 via an analog-to-digital converter (not shown).

The controller 120 receives the images captured by the image sensor 118 via the lens module 116 and stores them in the memory 122 and forms the high resolution image or composes the wide angle image by composing the stored images.

In the general mode, when the lens module 116 in the camera module (not shown) is shaken due to user's hand shake or the like, the camera module 100 compensates for camera shake by moving the lens module 116, .

The high-resolution image forming apparatus using the shaking motion compensating apparatus according to an embodiment of the present invention includes a shaking motion compensating apparatus 100 capable of precisely moving the lens module 116 in sub-pixel units, Acquiring an image at each of a plurality of positions shifted by a sub-pixel from a position to a different position, and composing the obtained images to form a high-resolution image.

In addition, in the high-resolution image forming apparatus using the camera-shake correction apparatus according to an embodiment of the present invention, when the lens module 116 is not fixed, it operates in the shake correction mode. When the controller 120 moves the lens module 116 to each position and acquires an image at each position in the camera shake correction mode, the controller 120 activates the shake correction function to acquire an image, It is possible to acquire missing images.

Also, in the high-resolution image forming apparatus using the shake correction apparatus according to an embodiment of the present invention, the lens module 116 is moved left and right at a predetermined angle to operate the shake correction function at each position, Obtains a shift image, and composes the obtained images to form a wide-angle image.

The offset adjustment unit 104 adjusts the DC offset in the signal output from the gyro sensor 102. However, if a predetermined DC offset is artificially applied to the offset adjustment unit 104, a signal output from the gyro sensor 102 The size of the lens module 116 can be changed, and as a result, the lens module 116 can be precisely moved in sub-pixel units.

The controller 120 artificially applies a predetermined offset to the offset adjuster 104 via the OIS controller 110 to move the lens module 116 in sub-pixel units to a desired position. In the embodiment of the present invention, the controller 120 artificially applies a predetermined offset to the offset adjustment unit 104 through the OIS controller 110. However, the controller 120 may directly control the offset adjustment unit 104, not via the OIS controller 110, A predetermined offset may be applied to the offset adjustment unit 104. [

Data relating to the relationship between the value of the offset applied to the offset adjusting section 104 and the moving direction and the moving distance of the lens module 116 is previously determined and stored in the memory 122, And is stored in a memory (not shown).

2 is a diagram illustrating a high-resolution image forming method using an image stabilization apparatus according to an embodiment of the present invention.

As shown in FIG. 2, a method of forming a high resolution according to a mode is somewhat different. In the fixed mode in which the lens module 116 is fixed, a case where the lens module 116 is fixed at a specific position rather than a direct control is advantageous for image acquisition. In this case, the lens module 116 is moved to a specific position, and the lens module 116 is moved in a different direction by 0.5 pixel at the corresponding position to obtain an image.

However, when the camera shake correction is required, since the camera shake is prevented based on the center position of the lens module 116, the camera shake correction function is activated at each position while the center position is changed by 0.5 pixel to acquire images. Since the image is obtained by activating the shake correction function in the shake correction mode, the image acquisition speed and the quality of the image may slightly differ from the fixed mode in which the image is acquired without activating the shake correction function.

Combining the four images acquired at each position in the fixed mode or camera shake correction mode can form a high resolution image that is four times the limit resolution of the image sensor.

Referring to FIGS. 1 and 2, in step S200, the controller 120 determines whether the lens module 116 is in the fixed mode or the camera shake correction mode.

The fixed mode or the shake correction mode can be set by the user or automatically set to the fixed mode when the camera (not shown) or smart phone (not shown) equipped with the lens module 116 is fixed .

If the lens module 116 is in the fixed mode, the controller 120 moves the lens module 116 to a specific position toward the position to be photographed by using the camera shake correction function in step S202.

The lens module 116 may typically be slightly offset downward by gravity. In this case, the controller 120 applies a control signal to the OIS controller 110 to apply the offset value corresponding to the movement distance for moving the lens module 116 to the reference position to the offset adjustment unit 104 , The lens module 116 is moved to the reference position toward the position to be photographed.

In step S204, the controller 120 corrects the lens module 116 by 0.5 pixels in the rightward direction, the downward direction, and the rightward diagonal direction from the reference position, as shown in FIGS. 3B to 3D, Acquires images without activating the shake correction function at each position, including the reference position, and stores the images obtained at each position in the memory 122, while moving the camera.

In the embodiment of the present invention, the lens module 116 is moved 0.5 pixels in the rightward direction, the downward direction, and the rightward diagonal direction from the reference position, but the embodiment of the present invention is not limited to this, Images may be acquired by moving the lens module 116 in units of sub-pixels less than one pixel, such as a pixel, 0.3 pixel, and so on.

 In step S204, the controller 120 applies a control signal to the OIS controller 110 to offset the offset values for moving the lens module 116 by 0.5 pixels in the rightward direction, downward direction, and rightward diagonal direction, The image is acquired at each position, including the reference position, while moving the lens module 116 by 0.5 pixels in the rightward direction, the downward direction, and the rightward diagonal direction, .

When the lens module 116 is in the fixed mode, there is almost no shaking of the lens module 116, so that it is possible to acquire an image free from blur even if an image is acquired without activating the shaking correction function in the fixed mode.

Therefore, in step S204, when the lens module 116 is moved to each position, when acquiring an image at each position, the image is acquired without activating the camera shake correction function.

In one embodiment of the present invention, when acquiring an image at each position in the fixed mode, images are acquired without activating the shake correction function, but when it is necessary to activate the shake correction function in the fixed mode, The control unit 120 may apply a control signal to the OIS controller 110 to operate the camera shake correction function in the fixed mode to acquire images in a state in which the camera shake correction function is activated when acquiring the image at each position.

In step S208, the controller 120 obtains the image 300 obtained at the reference position, the image 302 in which the lens module 116 is shifted 0.5 pixels to the right by 0.5 pixel as shown in Figs. 3A to 3E, An image 304 obtained by shifting the lens module 116 by 0.5 pixels in the downward direction and an image 306 by shifting the lens module 116 by 0.5 pixel in the lower diagonal direction are synthesized to form a high resolution image 308. [

3A to 3E show low resolution images 300, 302, 304, and 306 corresponding to one pixel to form a high resolution image 308 of one pixel for convenience of explanation.

Referring to FIGS. 3A and 3E, it can be seen that a high-resolution image 308 including four pixels is formed in an area corresponding to a conventional one pixel (horizontal (d) vertical (d))).

If the controller 120 determines that the lens module 116 is in the camera shake correction mode in step S200, the controller 120, in step S206, The camera shake correction function is activated at each position, including the reference position, while shifting the center position of the lens module 116 by 0.5 pixels in the rightward direction, the downward direction, and the rightward diagonal direction from the reference position, And stores the images obtained at each position in the memory 122. [0035]

When the camera shake correction is required, camera shake is prevented based on the center position of the lens module 116. Therefore, in the camera shake correction mode, the camera shake correction function is activated at each position while changing the center position of the lens module 116 do.

In step S206, the controller 120 applies a control signal to the OIS controller 110 to set offset values for moving the lens module 116 by 0.5 pixels in the rightward direction, the downward direction, and the rightward diagonal direction, By moving the lens module 116 by 0.5 pixels in the rightward direction, the downward direction, and the diagonal direction in the lower right direction, thereby obtaining an image at each position including the reference position, And stores it in the memory 122.

When the lens module 116 is moved to each position in the shake correction mode because the lens module 116 is not fixed in the shake correction mode, when the controller 120 acquires an image at each position, 120 acquire images without blurring at each position by acquiring images with the shake correction function activated.

In step S208, the controller 120 obtains the image 300 obtained at the reference position, the image 302 in which the lens module 116 is shifted 0.5 pixels to the right by 0.5 pixel as shown in Figs. 3A to 3E, An image 304 obtained by shifting the lens module 116 by 0.5 pixels in the downward direction and an image 306 by shifting the lens module 116 by 0.5 pixel in the lower diagonal direction are synthesized to form a high resolution image 308. [

Referring to FIGS. 3A and 3E, it can be seen that a high-resolution image 308 including four pixels is formed in an area corresponding to a conventional one pixel (horizontal (d) vertical (d))).

In step S210, the controller 120 obtains a clear high resolution image by removing the blur caused by the point spread function (PSF) in the synthesized high resolution image 308. [

5A is a reference image, FIG. 5B is a left shift image, and FIG. 5C is a view showing an enlarged view of the wide angle image in the wide angle photographing mode according to an embodiment of the present invention. 5D is a view showing a wide angle image obtained by synthesizing a left shift image and a right shift image.

5A to 5D, a wide-angle image can be formed by using a high-resolution image forming apparatus using an image stabilization apparatus according to an exemplary embodiment of the present invention.

Conventionally, a method of synthesizing images by using a super wide-angle lens or taking a photograph while moving the camera manually was used. On the other hand, in a system in which it is not easy to use an ultra-wide angle lens, it is difficult to acquire a high-quality wide-angle image due to changes in light or ambient conditions due to aberration or camera movement speed when the camera is moved.

Resolution image forming apparatus using the shake correcting apparatus according to an embodiment of the present invention can directly control the shaking motion correcting apparatus 100 to move the lens module 116 to the left and right to obtain a wide angle image by securing the angle of view .

For example, when the camera shake correction device included in the smart phone is used, the lens module 116 is moved left and right by 1.5 degrees using the camera shake correction device built in the smart phone, So that a total angle of view of 3 degrees can be additionally secured.

First, when the user selects the wide angle mode, the controller 120 applies a control signal to the OIS controller 110 in step S400 to turn on the camera shake correction function.

In step S402, the controller 120 moves the lens module 116 in the leftward direction by 0.5 degrees or more from the reference position using the camera-shake correction function.

In step S404, the controller 120 acquires the left shift image without shaking as shown in Fig. 5B while the shake correction function is activated, and stores the obtained image in the memory 122. Fig.

In step S406, the controller 120 moves the lens module 116 to the right by 0.5 degrees or more from the reference position using the shake correction function.

In step S408, the controller 120 acquires the right shift image without shaking, and stores the obtained image in the memory 122, as shown in Fig. 5C, with the shake correction function activated.

In step S410, the controller 120 synthesizes the left shift image (the image shown in Fig. 5B) stored in the memory 122 and the right shift image (the image shown in Fig. 5C) Obtain a wide-angle image.

In step S404, the controller 120 acquires the left shift image without shaking as shown in Fig. 5B while the shake correction function is activated, and stores the obtained image in the memory 122. Fig.

According to the apparatus and method for forming high resolution images using the shaking motion correction apparatus according to an embodiment of the present invention, it is possible to form a clear high resolution image without blurring exceeding a fixed limit resolution possessed by the image sensor, Can be formed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It is clear that the present invention can be modified or improved.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the appended claims.

100: Anti-shake device 102: Gyro sensor
104: offset adjustment unit 106: gyro integrator
108: Gain adjusting unit 110: OIS controller
112: VCM 114: hall sensor
116: Lens module 118: Image sensor
120: controller 122: memory

Claims (17)

An image stabilizer for detecting an image of the lens module and stabilizing the image; And
Adjusting an offset of an offset adjusting unit for adjusting an offset of an output signal of a motion sensor included in the shake correction apparatus to move the lens module from a reference position to a plurality of different positions to obtain images at each position, And a controller for synthesizing the acquired images to form a high-resolution image. The method according to claim 1,
The controller determines whether the lens module is in the camera shake correction mode,
When the lens module is in the camera shake correction mode,
Wherein the controller is configured to operate the camera shake correction function when acquiring images at the respective positions to acquire images without blurring. The method according to claim 1,
The controller determines whether the lens module is in the fixed mode,
When the lens module is in the fixed mode,
Wherein the controller acquires images without activating the shake correction function when acquiring images at the respective positions. The method according to claim 1,
Wherein the controller moves the lens module from a reference position to a plurality of different positions by a sub-pixel. The method of claim 4,
Wherein the sub-pixel comprises less than one pixel. The method according to claim 1,
The controller determines whether the lens module is in the wide angle photographing mode,
When the lens module is in the wide angle photographing mode,
Wherein the controller is configured to acquire an image by moving the lens module by a predetermined angle to the left and right at a reference position using the camera shake correction function to synthesize the obtained left and right images to form a wide angle image, Resolution image forming apparatus. The method of claim 6,
When the lens module is in the wide angle photographing mode,
Wherein the controller is configured to operate the camera shake correction function at the moved position when the lens module is moved by a predetermined angle to the left and right at the reference position to obtain an image, High resolution image forming apparatus. The method of claim 6,
Wherein the predetermined angle is 0.5 deg.. (A) adjusting an offset of an offset adjusting unit for adjusting an offset of an output signal of a motion sensor included in the camera shake correction apparatus to move the lens module from a reference position to a plurality of different positions to obtain an image at each position; And
(B) combining the images obtained at the respective positions by the controller to form a high-resolution image. The method of claim 9,
Prior to the step (A)
Further comprising the step of the controller determining whether the lens module is in the fixed mode or the camera shake correction mode. The method of claim 10,
When the lens module is in the camera shake correction mode,
Wherein the controller acquires images without shaking by activating the shake correction function when acquiring images at the respective positions. The method of claim 10,
When the lens module is in the fixed mode,
Wherein the controller acquires images at each position without activating the camera shake correction function when acquiring images at the respective positions. The method of claim 9,
In the step (A), the controller moves the lens module by a sub-pixel from a reference position to a plurality of different positions. 14. The method of claim 13,
Wherein the sub-pixel comprises less than one pixel. The method of claim 9,
Prior to the step (A)
Further comprising the controller determining whether the lens module is in the wide angle photographing mode,
When the lens module is in the wide angle photographing mode,
Wherein the controller moves the lens module by a predetermined angle to the left and right at a reference position using the camera shake correction function to obtain a left shift image and a right shift image and synthesizes the obtained left shift image and right shift image Thereby forming a wide-angle image. 16. The method of claim 15,
When the lens module is in the wide angle photographing mode,
Wherein the controller is configured to operate the camera shake correction function at the moved position when the lens module is moved by a predetermined angle to the left and right at the reference position to obtain an image, A method of forming high resolution images. 16. The method of claim 15,
Wherein the predetermined angle is 0.5 deg..

Images