WO2019098793A1 - Camera module - Google Patents

Abstract

A camera module according to an embodiment comprises an image sensor which outputs a plurality of image frames; a lens assembly which is disposed on the image sensor and includes a variable lens that adjusts a path of light which is incident on the image sensor from the outside; and a control unit which generates a control signal for controlling the variable lens; and an image synthesis unit which synthesizes a plurality of image frames to generate a composite image, wherein the composite image may have a higher resolution than a plurality of image frames, and the plurality of image frames may be each image frame generated by changing to different light paths from each other due to the variable lens.

Description

Camera module

The present invention relates to a camera module. More particularly, the present invention relates to a camera module and an optical apparatus capable of generating a super resolution image.

Users of handheld devices have a high resolution and a small size and are equipped with various shooting functions (zoom-in / zoom-out, auto-focusing, AF), anti- OIS) function and the like]. Such a photographing function can be realized by a method of directly moving the lens by combining a plurality of lenses, but when the number of lenses is increased, the size of the optical device can be increased.

The autofocus and camera shake correction functions are performed by moving a plurality of lens modules fixed to a lens holder and having optical axes aligned in a vertical direction of an optical axis or an optical axis or by tilting, Device is used. However, the power consumption of the lens driving device is high. In order to protect the lens driving device, it is necessary to add a cover glass separately from the camera module.

In addition, as the demand of high-quality images is increased, there is a demand for image sensors capable of shooting high-resolution images. However, for this purpose, the number of pixels included in the image sensor can not be increased, which increases the size of the image sensor and wastes power consumption.

That is, since the conventional camera module uses the data of a plurality of arrays as it is, there is a limit in resolution as much as the physical resolution of the image sensor. In addition, there are restrictions to use a plurality of cameras in order to generate a super-resolution image.

The present invention is to provide a camera module capable of generating a super-resolution image without increasing the number of pixels.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, unless further departing from the spirit and scope of the invention as defined by the appended claims. It will be possible.

A camera module according to an exemplary embodiment includes an image sensing unit including an image sensor outputting a plurality of image frames; A lens assembly disposed on the image sensor and forming a light path from the outside to the image sensor; A control unit for generating a control signal for adjusting at least one of the optical path of the lens assembly or the relative position of the image sensor to the lens assembly; And an image composing unit for compositing the plurality of image frames to generate a composite image, wherein the composite image has a higher resolution than the plurality of image frames, and the plurality of image frames are formed by the lens assembly, Or may be each image frame generated by changing the relative position of the image sensor with respect to the lens assembly.

The camera module according to another embodiment includes: an image sensor for outputting a plurality of image frames; A lens assembly disposed on the image sensor and including a variable lens that adjusts a light path that is incident on the image sensor from the outside; A control unit for generating a control signal for controlling the variable lens; And an image composing unit for compositing the plurality of image frames to generate a composite image, wherein the composite image has a higher resolution than the plurality of image frames, and the plurality of image frames are transmitted by different lenses And may be each image frame generated by modifying the image frame.

For example, the plurality of image frames may include a first image frame and a second image frame, and the second image frame may be an image frame shifted by a first interval with respect to the first image frame.

For example, the image sensor may generate an image frame of one of the plurality of image frames, receive a feedback signal indicating that the optical path by the variable lens is adjusted, Can be generated.

For example, the controller may receive the generation completion signal of one of the plurality of image frames, and then transmit the control signal to the variable lens to adjust the optical path.

For example, the image sensor may include a first area and a second area, and the control unit may control the light incident from the outside to pass through the variable lens to the light from the first area to the second area of the image sensor. It is possible to output a signal for adjusting the variable lens so as to change the path.

For example, the variable lens may include a liquid lens in which an interface between two liquids of different properties is deformed according to the control signal.

For example, the variable lens may be a variable prism whose angle of the lower plate is changed according to the control signal.

For example, the variable lens includes at least one lens; And an actuator for moving or tilting the at least one lens in at least one of a vertical direction and a horizontal direction according to the control signal.

For example, the image sensor may include a first area and a second area, and the control unit may control light passing through the variable lens from the outside to the second area from the first area of the image sensor, A signal for adjusting the variable lens can be output.

For example, the image sensor further includes a third region and a fourth region, and the control unit outputs a signal for adjusting the variable lens to change the light path from the second region to the third region , And outputting a signal for adjusting the variable lens to change the light path from the third region to the fourth region.

For example, the control signal may include a signal for changing the angle of view (FOV) of the lens assembly to a first direction, a signal for changing the angle of view of the lens assembly to a second direction, And a signal for changing the angle of view of the lens assembly to a fourth direction.

A camera module according to another embodiment includes an image sensing unit including an image sensor outputting a plurality of image frames; A lens assembly disposed on the image sensor and forming a light path from the outside to the image sensor; A control unit for generating a control signal for adjusting a relative position of the image sensor with respect to the lens assembly; And an image composing unit for compositing the plurality of image frames to generate a composite image, wherein the composite image has a higher resolution than the plurality of image frames, and the plurality of image frames are image- And may be each image frame generated by changing its relative position.

For example, the plurality of image frames may include a first image frame and a second image frame, and the second image frame may be an image frame shifted by a first interval based on the first image frame.

For example, the image sensing unit may further include an actuator for moving or tilting the image sensor according to the control signal in at least one of an optical axis direction and a direction perpendicular to the optical axis direction.

For example, the image sensor may include a first area and a second area, and the control unit may include a first area that receives light from the outside and that receives light passing through the lens assembly, And outputs a signal for adjusting the actuator so as to change to the area.

For example, the image sensor may further include a third region and a fourth region, and the control portion may include a region for receiving light from the outside and receiving light passing through the lens assembly from the second region to the third region And outputting a signal for adjusting the actuator such that an area that is received from the outside and receives light passing through the lens assembly changes from the third area to the fourth area have.

For example, the image synthesizer may synthesize a first super-resolution image frame using the first through fourth image frames transmitted from the image sensor, and then combine the fifth image frame output from the image sensor and the second super- The second super-resolution image frame may be synthesized using the fourth through the fourth image frames.

A camera module according to another embodiment includes: a lens assembly including a liquid lens for adjusting a light path; An image sensor for sensing a plurality of images using the lens assembly; A control unit for controlling the liquid lens; And a combining unit that combines the plurality of images to generate a composite image, and the plurality of images may include an image generated by the liquid lens changing the optical path differently.

A camera module according to another embodiment includes an image sensor for sensing a plurality of images; A lens assembly forming a light path to be incident on the image sensor; A control unit for adjusting at least one of the optical path and the position of the image sensor; And an image composing unit for compositing the plurality of images to generate a composite image, wherein the plurality of images are images generated by different optical paths by the lens assembly, or images generated at different positions of the image sensor .

According to another embodiment of the present invention, there is provided an optical apparatus comprising: the camera module; A display unit for outputting an image; A battery for supplying power to the camera module; And a housing for mounting the camera module, the display unit, and the battery.

According to another embodiment of the present invention, there is provided an image generating method comprising: outputting a first image frame; Generating a second image frame shifted by a first distance in a first direction in the first image frame; Generating a third image frame shifted by the first distance in a second direction in the second image frame; Generating a fourth image frame shifted by the first distance in a third direction in the third image frame; And synthesizing the first image frame to the fourth image frame to generate a composite image, wherein the composite image may have a higher resolution than the plurality of image frames.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, And can be understood and understood.

The effect of the device according to the present invention will be described as follows.

According to the camera module according to the embodiment of the present invention, although a high computational load is required to obtain a super-resolution image, a variable lens that changes the optical path without using a large number of cameras without increasing the number of pixels or It can be solved in hardware by using an image sensor whose relative position with respect to the lens assembly changes. That is, since a plurality of array data shifted by half (0.5 PD) of the pixel distance PD is used, an image having a super resolution higher than the physical resolution of the image sensor can be obtained.

In addition, it is possible to prevent the frame rate from lowering by continuously generating composite frames for the current frame sequentially input.

The effects obtainable by the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

1 is a block diagram illustrating a camera module according to one embodiment.

2A and 2B are views showing an embodiment of the camera module shown in FIG.

3 is a view for explaining an embodiment of an operation of changing the FOV angle of the variable lens.

4 is a view showing an embodiment of a variable lens included in the camera module shown in FIG. 2A.

5 is a view showing another embodiment of a variable lens included in the camera module shown in FIG. 2A.

FIG. 6 is a view for explaining the variable lens according to the embodiment shown in FIG. 5 in more detail.

7 is a view showing a liquid lens according to an embodiment.

8 is a block diagram illustrating a camera module according to another embodiment.

9 is a cross-sectional view of the image sensing unit shown in FIG. 8 according to an embodiment of the present invention.

10 is a plan view of another embodiment of the image sensing unit shown in FIG.

11 shows a plan view of another embodiment of the image sensing unit shown in Fig.

12A and 12B show perspective views of another embodiment of the image sensing unit shown in Fig.

13 is a diagram for explaining a method of operating a camera module according to an embodiment.

FIG. 14 is a diagram for explaining the operation method of the camera module described in FIG. 13 in more detail.

15 is a timing diagram of a method of operating a camera module according to an embodiment.

16 is a diagram illustrating an example of a frame synthesis method of a camera module according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The embodiments are to be considered in all aspects as illustrative and not restrictive, and the invention is not limited thereto. It is to be understood, however, that the embodiments are not intended to be limited to the particular forms disclosed, but are to include all modifications, equivalents, and alternatives falling within the spirit and scope of the embodiments.

The terms " first ", " second ", and the like can be used to describe various components, but the components should not be limited by the terms. The terms are used for the purpose of distinguishing one component from another. In addition, terms specifically defined in consideration of the constitution and operation of the embodiment are only intended to illustrate the embodiments and do not limit the scope of the embodiments.

In the description of the embodiments, when it is described as being formed on the "upper" or "on or under" of each element, the upper or lower (on or under Quot; includes both that the two elements are in direct contact with each other or that one or more other elements are indirectly formed between the two elements. Also, when expressed as "on" or "on or under", it may include not only an upward direction but also a downward direction with respect to one element.

It is also to be understood that the terms "top / top / top" and "bottom / bottom / bottom", as used below, do not necessarily imply nor imply any physical or logical relationship or order between such entities or elements, And may be used to distinguish one entity or element from another entity or element.

Although the camera modules 10 and 20 according to the embodiment are described using a Cartesian coordinate system, they may be described by other coordinate systems. According to the Cartesian coordinate system, the x-axis, the y-axis and the z-axis are orthogonal to each other, but the embodiment is not limited to this. That is, the x-axis, the y-axis, and the z-axis may intersect with each other.

Hereinafter, a camera module 10 according to an embodiment will be described with reference to the accompanying drawings.

1 is a block diagram illustrating a camera module 10 according to one embodiment.

Referring to FIG. 1, the camera module 10 may include a lens assembly 100A, an image sensor 200A, an image synthesizer 300, and a controller 400A.

The lens assembly 100A can transmit the optical signal to the image sensor 200A through the light incident from the outside of the camera module 10. [ The lens assembly 100A may include a variable lens 110. [ According to the embodiment, the lens assembly 100A may further include at least one lens other than the variable lens 110. [ The lenses included in the lens assembly 100A form one optical system and can be aligned around the optical axis of the image sensor 200A.

The variable lens 110 may change the optical path of the lens assembly 100A under the control of the controller 400A. The variable lens 110 may change the optical path to be incident on the image sensor 200A and may change the focal distance of the optical signal, the field of view (FOV) angle, the direction of the FOV, or the like.

According to one embodiment, the variable lens 110 may be composed of a liquid lens or a variable prism. Alternatively, the variable lens 110 does not include a liquid lens, and may include only at least one solid lens. In this case, the material of the optical element such as the variable prism has no fluidity and the refractive index can be 1 or more and 3 or less. In addition, the inner material of the variable lens 110 may be formed of two or more kinds of materials, and the interface of each material may be changed to change the optical power.

According to another embodiment, the variable lens 110 may comprise an actuator coupled with at least one lens and at least one lens. Here, at least one lens may be a liquid lens, a solid lens, or both a liquid lens and a solid lens. The actuator can control the physical displacement of at least one lens under the control of the controller 400A. That is, the actuator can adjust the distance between the at least one lens and the image sensor 200A, or adjust the angle between the at least one lens and the image sensor 200A. Or the actuator may shift at least one lens in the x-axis and y-axis directions of the plane formed by the pixel array of the image sensor 200A. Further, the actuator can change the path of the light incident on the pixel array of the image sensor 200A. For example, when at least one lens included in the variable lens 110 does not include a liquid lens, that is, when at least one lens included in the variable lens 110 is a solid lens, It is possible to move or tilt the lens in at least one of the vertical direction and the horizontal direction according to the control signal (i.e., the first signal shown in Fig. 1) output from the control unit 400A.

The image sensor 200A includes a pixel array for receiving an optical signal transmitted through the lens assembly 100A and converting the optical signal into an electrical signal corresponding to the optical signal, a driving circuit for driving a plurality of pixels included in the pixel array, And a readout circuit for reading the pixel signal. The lead-out circuit may compare the analog pixel signal with a reference signal to produce a digital pixel signal (or video signal) through an analog-to-digital conversion. Here, the digital pixel signal of each pixel included in the pixel array constitutes a video signal, and the video signal can be defined as an image frame as it is transmitted in units of frames. That is, the image sensor can output a plurality of image frames.

The image synthesizer 300 may be an image processor that receives an image signal from the image sensor 200A and processes (e.g., interpolates, frame synthesizes, etc.) the image signal. In particular, the image combining unit 300 may synthesize a video signal (high resolution) of one frame using video signals (low resolution) of a plurality of frames. The plurality of image frames may be each image frame generated by changing the optical path by the variable lens 110. The image combining unit 300 may be referred to as a post-processor. The plurality of image frames may include a first image frame and a second image frame, and the second image frame may be an image frame shifted by a first interval based on the first image frame.

The control unit 400A controls the variable lens 110 and the image sensor 200A and synchronizes with the control state of the variable lens 110 so that the image sensor 200A can generate a video signal. To this end, the control unit 400A transmits and receives a first signal to and from the variable lens 110, and transmits and receives a second signal to and from the image sensor 200A.

The first signal is generated by the control unit 400A and includes a lens control signal for optical path control of the variable lens 110 or a lens control signal for controlling the focal length or the FOV angle of the variable lens 110 can do. In particular, the lens control signal can determine the optical path of the light passing through the variable lens 110. Further, the lens control signal can determine the changing direction and the changing angle of the FOV angle of the variable lens 110. According to the embodiment, the first signal is generated by the variable lens 110, and may include a response signal indicating that the control of the variable lens 110 is completed in accordance with the lens control signal. The control unit 400A may be referred to as a variable lens driver.

The second signal may be generated by the image sensor 200A and may include a synchronization signal that directs the lens control signal to be transmitted to the variable lens 110. [ According to the embodiment, the second signal may be generated by the image sensor 200A and may include control information that is the basis of the control signal for optical path control of the variable lens 110. [ According to the embodiment, the second signal may be generated by the control unit 400A and may include a feedback signal indicating that the response signal indicating that the control of the variable lens 110 has been completed is received according to the lens control signal.

Further, the second signal may include a driving signal for driving the image sensor 200A.

Here, the signals included in the first signal and the second signal are illustrative, and may be omitted or other signals may be added if necessary.

FIG. 2A is a diagram illustrating an embodiment 10A of the camera module 10 shown in FIG.

Referring to FIG. 2A, the camera module 10A may include a lens assembly, an image sensor 200A, and a substrate 250. FIG. Here, the lens assembly includes a holder 130A, a lens barrel 140A, a first lens unit 150A, a second lens unit 160A, and an IR glass 170 that transmits or blocks IR (Infrared) And at least one of the configurations may be omitted or the up-down arrangement relationship may be changed. In addition, the lens assembly may further include a variable lens 110 capable of changing the optical path under the control of the control unit 400A.

The holder 130A may be coupled to the lens barrel 140A to support the lens barrel 140A and may be coupled to the substrate 250 to which the image sensor 200A is attached. In addition, the holder 130A may have a space to which the IR glass 170 can be attached under the lens barrel 140A. The holder 130A includes a helical structure and can be rotationally coupled with the lens barrel 140A, which also includes a helical structure. However, this is merely an example, and the holder 130A and the lens barrel 140A may be coupled through an adhesive (for example, an adhesive resin such as an epoxy) or the holder 130A and the lens barrel 140A Or may be integrally formed.

The lens barrel 140A is coupled to the holder 130A and may have a space for accommodating the first lens unit 150A, the second lens unit 160A, and the variable lens 110 therein. The lens barrel 140A may be rotationally coupled to the first lens unit 150A, the second lens unit 160A, and the variable lens 110. However, the lens barrel 140A is illustrative and may be coupled in other manners, have.

The first lens portion 150A may be disposed in front of the second lens portion 150A. The first lens unit 150A may be composed of at least one lens, or two or more lenses may be aligned with respect to the central axis to form an optical system. Here, the central axis may be the same as the optical axis of the optical system of the camera module 10 (10A). The first lens unit 150A may be formed of one lens as shown in FIG. 2A, but is not limited thereto.

The second lens unit 160A may be disposed behind the first lens unit 150A. Light incident from the outside of the camera module 10 or 10A to the first lens unit 150A may pass through the first lens unit 150A and enter the second lens unit 160A. The second lens unit 160A may be composed of at least one lens, or two or more lenses may be aligned with respect to the central axis to form an optical system. Here, the center axis may be the same as the optical axis of the optical system of the camera module 10, 10A. The second lens unit 160A may be formed of one lens as shown in FIG. 2A, but is not limited thereto.

May be referred to as a "first solid lens portion" and a "second solid lens portion" in order to distinguish the first lens portion 150A and the second lens portion 160A from the liquid lens.

As described above, the lens assembly may further include a variable lens 110, and the position of the variable lens 110 may be any one of the first position P1 to the fourth position P4. However, this is an exemplary one, and the variable lens 110 may be located at a different position depending on the presence or absence and relative positions of the first lens unit 150A, the second lens unit 160A, and the IR glass 170. [ However, the variable lens 110 may be positioned on an optical path through which light incident on the lens assembly passes, thereby changing the focal length or FOV angle.

The first position P1 corresponds to the outside of the lens barrel 140A and the second position P2 corresponds to the upper portion of the first lens unit 150A inside the lens barrel 140A. The third position P3 corresponds to the position between the first lens unit 150A and the second lens unit 160A in the lens barrel 140A and the fourth position P4 corresponds to the position between the lens barrel 140A, The second lens unit 160A is located at a position corresponding to the lower portion of the second lens unit 160A.

The IR glass 170 can filter light corresponding to a specific wavelength range with respect to light having passed through the second lens unit 160A. The IR glass 170 may be mounted and fixed to the inner groove of the holder 130A.

The image sensor 200A may be mounted on the substrate 250 and may convert light passing through the lens assembly into a video signal.

The substrate 250 may be disposed at a lower portion of the holder 130A and may include wiring for transferring electric signals between the image composing unit 300 and the control unit 400A. A connector (not shown) may be connected to the substrate 250 to electrically connect the camera module 10 to a power source or other device (for example, an application processor).

The substrate 250 may be formed of a Rigid Flexible Printed Circuit Board (RFPCB) and may be bent according to a space required for mounting the camera modules 10 and 10A. However, the embodiments are not limited thereto.

FIG. 2B is a view showing another embodiment 10B of the camera module 10 shown in FIG.

Among the components of the camera module 10B shown in Fig. 2B, the same components as those of the camera module 10A shown in Fig. 2A are denoted by the same reference numerals and the description thereof is omitted. Explain as follows. Therefore, the description of the camera module 10A shown in Fig. 2A can be applied to the respective components of Fig. 2B.

The camera module 10B may include a lens assembly, an image sensor 200A, and a substrate 250, at least one of which may be omitted, or the relationship between the upper and lower parts may be changed. The image sensor 200A and the substrate 250 shown in FIG. 2B correspond to the image sensor 200A and the substrate 250 shown in FIG. 2B, respectively, and a description thereof will be omitted. Further, the camera module 10B may further include a cover 186. Fig. The cover 186 is arranged to surround and enclose the components of the camera module 10B while exposing the upper opening of the lens assembly so as to protect each component of the camera module 10B.

The lens assembly may include a variable lens 110, a holder 130B, a lens barrel 140B, a first lens unit 150B, a second lens unit 160B, and an IR glass 170. At least one of these configurations may be omitted or the up-and-down arrangement relationship may be changed.

The variable lens 110, the holder 130B, the lens barrel 140B, the first lens unit 150B, the second lens unit 160B, and the IR glass 170 shown in FIG. 2B correspond to the variable The lens barrel 140A, the first lens unit 150A, the second lens unit 160A, and the IR glass 170, respectively. However, except for the IR glass 170, the variable lens 110, the holder 130B, the lens barrel 140B, the first lens unit 150B, and the second lens unit 160B shown in Fig. The holder 130A, the lens barrel 140A, the first lens unit 150A, and the second lens unit 160A shown in Figs. 2A and 2B.

The lens barrel 140A shown in FIG. 2A has a structure to be inserted into the holder 130A, whereas the lens barrel 140B shown in FIG. 2B can be disposed apart from the upper portion of the holder 130B.

The holder 130B can be coupled with the lens barrel 140B through the connecting portions 192 to 198 and the actuators 182 and 184 to support the lens barrel 140B. In addition, like the holder 130A shown in FIG. 2A, the holder 130B shown in FIG. 2B may have a space to which the IR glass 170 can be attached under the lens barrel 140B.

The lens barrel 140B may have a space in which a lens that can be included in the first lens unit 150B, the second lens unit 160B, and the variable lens 110 may be accommodated. The lens barrel 140B may be rotationally coupled with the lenses of the first lens unit 150B, the second lens unit 160B and the variable lens 110. However, the lens barrel 140B is illustrative, .

In addition, the first lens unit 150A shown in Fig. 2A has only one lens, whereas the first lens unit 150B shown in Fig. 2B has two lenses. Similarly, the second lens unit 160A shown in Fig. 2A has only one lens, whereas the second lens unit 160B shown in Fig. 2B has two lenses. However, the number of lenses included in each of the first and second lens units 150B and 160B may be one or three or more, as shown in FIG. 2A, unless the embodiment is limited thereto.

The variable lens 110 shown in FIG. 2B may include actuators 182 and 184, an adhesive 188, and connections 192, 194, 196, and 198. Further, the variable lens 110 may include at least one lens, and at least one lens may include only a liquid lens, only a solid lens, or both a liquid lens and a solid lens.

2B, the number of the actuators 182 and 184 included in the variable lens 110 is two. However, according to another embodiment, the number of the actuators 182 and 184 may be one. That is, the actuators 182 and 184 shown in FIG. 2B may be integrally implemented. The number of the actuators 182, 184 may be three or more. As such, embodiments are not limited to any particular number of actuators 182, 184.

The actuators 182 and 184 move or tilt at least one lens included in the lens assembly in at least one of a vertical direction and a horizontal direction in response to a control signal output from the control unit 400A, . To this end, the actuators 182 and 184 may be disposed around the outer wall of the lens barrel 140B.

The actuators 182 and 184 can control the physical displacement of the lenses 150B and 160B under the control of the controller 400A. That is, the actuators 182 and 184 adjust the distance between the at least one lens 150B and 160B and the image sensor 200A, or adjust the angle between the at least one lens 150B and 160B and the image sensor 200A Can be adjusted. Or the actuators 182 and 184 may shift at least one lens in the x and y axis directions of the plane formed by the pixel array of the image sensor 200A. Further, the actuators 182 and 184 can change the path of the light incident on the pixel array of the image sensor 200A. For example, when at least one lens included in the variable lens 110 does not include a liquid lens, that is, when at least one lens included in the variable lens 110 is a solid lens, the actuators 182, 184 transmit at least one lens included in the first and second lens units 150B and 160B and the variable lens 110 to a control signal output from the control unit 400A In at least one of the vertical direction and the horizontal direction.

As described above, if the actuators 182 and 184 are capable of moving at least one lens included in the lens assembly in the vertical direction as well as in the horizontal direction and tilting, the actuators 182 and 184 The optical path can be changed. The actuators 182 and 184 can move the lens precisely and can be driven in response to a drive signal output in the form of voltage or current from the control unit 400A. Here, the driving signal may be included in the first signal.

For example, the lens barrel 140B can be moved by a distance of 1 mm or less by the actuators 182 and 184, can be moved or tilted in the vertical direction of the optical axis or the optical axis, and the tilting angle can be 1 degree or less .

Actuator 182 and 184 may be a piezoelectric element, a voice coil motor (VCM), or a microelectromechanical systems (MEMS) Lt; RTI ID = 0.0 > 182, < / RTI >

The actuators 182 and 184 are disposed on the sides of the lens barrel 140B to horizontally move at least one lens included in the lens assembly such that the actuators 182 and 184 horizontally move the entire lens barrel 140B horizontally Direction (for example, at least one of the x-axis direction and the y-axis direction). To this end, connection portions 192 and 196 may be interposed between the actuators 182 and 184 and the lens barrel 140B. The connection portions 192 and 196 serve to connect the actuators 182 and 184 and the lens barrel 140B. In some cases, when the actuators 182 and 184 are directly connected to the lens barrel 140B, the connection portions 192 and 196 may be omitted.

The actuators 182 and 184 are disposed on the upper portion of the holder 130B so as to move the entire lens barrel 140B in the vertical direction so that the actuators 182 and 184 move the at least one lens included in the lens assembly in the vertical direction. And can be moved in the vertical direction (for example, the z-axis direction). To this end, connection portions 194 and 198 may be interposed between the actuators 182 and 184 and the holder 130B. The connecting portions 194 and 198 serve to connect the actuators 182 and 184 and the holder 130B. In some cases, when the actuators 182, 184 are directly connected to the holder 130B, the connections 194, 198 may be omitted.

The actuator may be implemented in the form of a voice coil motor (VCM). In this case, the lens barrel includes a coil disposed in the periphery of the lens barrel, and the position of the lens barrel can be adjusted by interacting with the barrel and the coil and the magnet disposed in the housing or the yoke. In this case, one end of the elastic member or the like may support the barrel and the other end may be disposed in combination with the housing or the yoke.

In addition, the variable lens 110 may further include an adhesion portion 188. [ The adhesive 188 serves to join the cover 186 and the actuators 182 and 184. The cover 186 is fixed while the actuators 182 and 184 can move or tilt in at least one of the horizontal direction or the vertical direction. To this end, the adhesive 188 may be made of a material that allows movement of the actuators 182, 184, and in some cases the adhesive 188 may be omitted. If the bonding portion 188 is omitted, the cover 186 and the actuators 182 and 184 may be disposed apart from each other.

3 is a view for explaining an embodiment of an operation of changing the FOV angle of the variable lens as a change of the optical path by the variable lens.

Referring to FIG. 3, the lens assembly 100A may have a specific field of view (FOV). The FOV means a range of incident light that the image sensor 200A can capture through the lens assembly 100A and can be defined as a FOV angle. The FOV angle of a typical lens assembly may range from 60 degrees to 140 degrees. Assuming the x axis and the y axis when viewing the lens assembly 100A from above (i.e., in a direction perpendicular to the optical axis), the FOV angle includes the first FOV angle Fx and the second FOV angle Fy . The first FOV angle Fx means the angle of the FOV determined along the x axis and the second FOV angle Fy means the angle of the FOV determined along the y axis.

The plurality of pixels included in the pixel array of the image sensor 200A may be arranged in a matrix form of NxM (where N and M are each an integer of 1 or more). That is, N pixels may be arranged along the x-axis, and M pixels may be arranged along the y-axis. An optical signal incident through the FOV corresponding to the first FOV angle Fx and the second FOV angle Fy is incident on the NxM pixel array.

The FOV of the optical path or lens assembly 100A passing through the lens assembly 100A can be changed by the lens control signal included in the first signal. The lens control signal can change the first FOV angle Fx and the second FOV angle Fy respectively and the change of the first FOV angle Fx and the second FOV angle Fy according to the lens control signal 1 angle change amount? X and the second angle change amount? Y.

The first angle variation amount? X and the second angle variation amount? Y can be defined by the following equations (1) and (2), respectively.

Here, a may be greater than 0.1 and less than 0.5, and b may be greater than 1 and less than 2, but the scope of the embodiment is not limited thereto.

At this time,? X and? Y are change angles with respect to the image generated by the image sensor 200A, and the change angle of the actual variable lens 110 may be larger or smaller. When the variable lens 110 is a transmission type, it is common that each of? X and? Y has a value smaller than 2. When the variable lens 110 is a reflection type,? X and? Y are each generally larger than 1, 100A may have a very different value depending on the configuration of the optical system.

FIG. 4 is a view showing an embodiment of a variable lens included in the camera module 10A shown in FIG. 2A.

Referring to FIG. 4, the variable lens 40 is an embodiment of the variable lens 110 shown in FIG. 1, and the variable lens 40 has a structure including a fluid 41 between upper and lower plates Lt; / RTI > When the driving voltage is applied to the liquid 41, the liquid 41 is concentrated in a specific direction, and the angle of the lower plate with respect to the image sensor can be changed. In addition, the angle of the lower plate with respect to the image sensor 200A can be increased as the driving voltage applied to the fluid 41 increases (or as the deviation of the driving voltage increases). The driving voltage applied to the variable lens 40 can be predetermined so that the first FOV angle Fx and the second FOV angle Fy change depending on the lens control signal.

The variable lens 40 may be a variable prism, but the scope of the present invention is not limited thereto.

FIG. 5 is a view showing another embodiment 50 of the variable lens 110 included in the camera module 10A shown in FIG. 2A.

1, the variable lens 50 includes two liquids 51 having different properties between the upper and lower plates. The variable lens 50 includes a plurality of liquids 51, As shown in FIG. The two liquids 51 are made of a fluid material having a refractive index of 1 to 2. When a driving voltage is applied to the two liquids 51, the interface between the two liquids 51 is deformed, so that the FOV angle of the interface can be changed. Further, as the driving voltage applied to the two liquids 51 increases (or as the deviation of the driving voltage increases), the FOV angle change of the interface can be increased. With this characteristic, the drive voltage applied to the variable lens 50 can be predetermined so that the first FOV angle Fx and the second FOV angle Fy change in accordance with the lens control signal.

FIG. 6 is a view for explaining the variable lens according to the embodiment shown in FIG. 5 in more detail.

Specifically, FIG. 6A illustrates the liquid lens 28, and FIG. 6B illustrates an equivalent circuit of the liquid lens 28. FIG. Here, the liquid lens 28 may mean the variable lens 50 of Fig.

6 (a), the liquid lens 28 whose interface is adjusted in correspondence to the driving voltage is arranged in four different directions with the same angular distance to form a plurality of electrode sectors L1 , L2, L3, and L4) and the electrode sector constituting the second electrode. When a driving voltage is applied through a plurality of electrode sectors (L1, L2, L3, L4) constituting the first electrode and an electrode sector constituting the second electrode, the conductive liquid and the nonconductive liquid The interface may be deformed. The degree and shape of the deformation of the interface between the conductive liquid and the nonconductive liquid can be controlled by a control circuit (control section 400A in Fig. 1).

6 (b), one side of the liquid lens 28 receives a voltage from the different electrode sectors L1, L2, L3 and L4 and the other side receives the voltage from the electrode sector C0 of the second electrode. And a plurality of capacitors 30 connected to the capacitors 30 and receiving a voltage.

In the present specification, four individual electrodes are described as an example, but the scope of the present invention is not limited thereto.

7 is a view showing a liquid lens 700 according to an embodiment.

Referring to FIG. 7, the liquid lens 700 may correspond to one embodiment of the cross section of the liquid lens 28 shown in FIG.

The liquid lens 700 may include a conductive liquid 72, a nonconductive liquid 73, a plate, an electrode portion, and an insulating layer 76. The plate may include a first plate 74, and may further include a second plate 77 and a third plate 71. The electrode unit may include a first electrode 75-1 and a second electrode 75-2.

The second plate 77 and the third plate 71 may be made of a transparent material. Either the second plate 77 or the third plate 71 may be arranged to receive light passing through the lens assembly 100A from the liquid lens 700 first. The third plate 71 may be disposed below the first electrode 75-1 and the second plate 77 may be disposed above the second electrode 75-2.

The conductive liquid 72 and the nonconductive liquid 73 may be filled in a cavity determined by the opening area of the first plate 74. That is, the cavity may be filled with the conductive liquid 72 of different properties and the nonconductive liquid 73, and the interface IF is formed between the conductive liquid 72 of the different nature and the nonconductive liquid 73 .

The interface IF formed by the conductive liquid 72 and the nonconductive liquid 73 can be adjusted in focal length or shape of the liquid lens 700 while bending, An area where the optical signal can pass through the interface IF may correspond to the lens area 310 described in FIG.

Here, the conductive liquid 72 may include at least one of ethylene glycol and sodium bromide (NaBr), and may be formed by mixing ethylene glycol and sodium bromide (NaBr) And the nonconductive liquid 73 may comprise a phenyl series silicone oil.

The first plate 74 may include an aperture region located between the third plate 71 and the second plate 77 and having a predetermined sloped surface (e.g., an inclined surface having an angle of about 59 degrees to 61 degrees) . That is, the first plate 74 may include an inclined surface therein, and the conductive liquid 72 and the nonconductive liquid 73 may be disposed in the cavity defined by this inclined surface. The first plate 74 is a housing structure for housing two liquids 72 and 73 having different properties in the liquid lens 700. The third plate 71 and the second plate 77 are regions in which the optical signal passes And it is possible to form the first plate (the first plate) and the second plate (the second plate) by using the same material as the first plate 74 may also be formed of a material such as glass. According to another embodiment, the first plate 74 may contain impurities such that transmission of the optical signal is not easy.

The first electrode 75-1 and the second electrode 75-2 are connected to a control circuit 400A of FIG. 1 for controlling the interface IF between the conductive liquid 72 and the nonconductive liquid 73, And a driving voltage supplied from the driving voltage generating unit. The first electrode 75-1 may be disposed on an inclined surface of the first plate 74 and the second electrode 75-2 may be disposed on the upper surface of the first plate 74. [

L1, L2, L3, and L4 and the common electrode C0 are formed on both sides of the first plate 74 adjacent to the third plate 71 and the second plate 77, And / or an electrode pattern may be included. The second electrode 75-2 is a common electrode disposed to be in contact with the conductive liquid 72 and the first electrode 75-1 is disposed so as to be close to the conductive liquid 72 with the insulating layer 76 therebetween It may be an individual electrode.

Here, the first electrode 75-1 and the second electrode 75-2 may include chromium (Cr). Chromium or chromium is a silver-colored, hard, transition metal that is easy to break, characterized by high melting point without discoloration. However, alloys containing chromium are resistant to corrosion and can be used in alloyed form with other metals. Particularly, chromium (Cr) has a characteristic of being resistant to the conductive liquid filling the cavity because of less corrosion and discoloration.

The point at which the interface IF contacts the inclined surface of the cavity can be changed according to the voltage difference between the individual electrode and the common electrode and the driving voltage is asymmetrically controlled between the individual electrodes facing each other using the same to change the predetermined FOV angle change direction and change You can change the FOV at an angle.

The insulating layer 76 is a structure for physically insulating the first electrode 75-1 from the conductive liquid 72 and the nonconductive liquid 73. [ For example, the insulating layer 76 may include parylene C and may be formed by a method such as coating, deposition, plating, and the like.

The insulating layer 76 may extend over the top of the first plate 74 and the underside of the nonconductive liquid 73, including inclined surfaces that may abut the conductive liquid 72 and the non- have. The insulating layer 76 may be disposed on the upper portion of the first electrode 75-1. The first electrode 75-1 and the second electrode 75-2 are disposed adjacent to each other on the first plate 74. The insulating layer 76 is formed such that the first electrode 75-1 is electrically connected to the conductive liquid 72 The first electrode 75-1 may be disposed so as not to be in contact with the first electrode 75-1 and at least a portion thereof may be disposed in contact with the second electrode 75-2 as shown in FIG. But is not limited thereto.

The second plate 77 may be formed of a transparent glass and constitutes a cavity together with the third plate 71 and the opening area so that the conductive liquid 72 and the nonconductive liquid 73 can be filled .

Hereinafter, a camera module 20 according to another embodiment will be described with reference to the accompanying drawings.

8 is a block diagram showing a camera module 20 according to another embodiment.

Referring to FIG. 8, the camera module 20 may include a lens assembly 100B, an image sensing unit 200B, an image synthesizing unit 300, and a control unit 400B. Here, since the image composing unit 300 performs the same function as the image composing unit 300 shown in FIG. 1, the same reference numerals are used and redundant explanations are omitted.

The lens assembly 100B is disposed on the image sensing unit 200B and can transmit the optical signal to the image sensor 230 included in the image sensing unit 200B through the light incident from the outside of the camera module 20. [ have. That is, the lens assembly 100B can form an optical path to be incident on the image sensing unit 200B.

The lens assembly 100B may have a configuration as shown in Figs. 2A and 2B. At this time, the lens assembly 100B may include the variable lens 110, or may not include the variable lens 110. [

If the lens assembly 100B does not include the variable lens 110, the configuration in which the variable lens 110 is omitted from the camera modules 10A and 10B shown in FIG. 2A or 2B is applied to the lens assembly 100B, And thus redundant description thereof will be omitted.

The image sensing unit 200B may include an image sensor 230 and an actuator 240. [ The image sensing unit 200B can adjust the relative position of the image sensor 230 with respect to the lens assembly 100B under the control of the controller 400B. To this end, the actuator 240 may adjust the physical displacement of the image sensor 230.

In the case of the camera module 10 according to the embodiment shown in FIG. 1, a variable lens 110 is used to adjust the optical path from the outside to the image sensor 200A.

8, by adjusting the relative position of the image sensor 230 with respect to the lens assembly (i.e., the lens), the optical path incident on the image sensor 230 A controlled effect can be obtained.

When the lens assembly 100B shown in FIG. 8 includes the variable lens 110 like the lens assembly 100A shown in FIG. 1, The optical path is primarily adjusted by the variable lens 110, and can be adjusted by the actuator 240 in a secondary order.

When the lens assembly 100B includes the variable lens 110, the control unit 400B can control the lens assembly 100B and the image sensor 230 in the same manner as the control unit 400A. In this case, the third signal shown in Fig. 8 may include a first signal, and the fourth signal may include a second signal. However, when the lens assembly 100B does not include the variable lens 110, the third signal can be omitted.

The controller 400B may control the actuator 240 of the image sensing unit 200B to adjust the relative position of the image sensor 230 with respect to the lens assembly 100B. For this, the control unit 400B can transmit and receive the fourth signal to and from the image sensing unit 200B. That is, in response to the fourth signal, the actuator 240 rotates the image sensor 230 in at least one of the optical axis direction (or the direction parallel to the optical axis) (e.g., the z axis direction) Direction or tilting. To this end, the actuator 240 may be a piezoelectric element, a voice coil motor (VCM), a MEMS (MEMS), but embodiments are not limited to any particular form of the actuator 240.

The actuator 240 moves the image sensor 230 itself in at least one direction of the optical axis direction (or in the direction parallel to the optical axis) (e.g., the z axis direction) or in the horizontal direction perpendicular to the optical axis direction Or may move the substrate 250 on which the image sensor 230 is disposed, move the package of the image sensor 230, or move the image sensor 230, as shown in FIG. 2A or 2B, And the embodiment is not limited to this.

Hereinafter, various embodiments 200B1 to 200B4 of the image sensing unit 200B shown in FIG. 8 will be described below with reference to the accompanying drawings, but the embodiments are not limited thereto. For example, The configuration of the camera module 20 according to the first embodiment of the present invention is not limited to the substrate 250 and the image sensor 200A in the camera modules 10A and 10B shown in Figs. And image sensing units 200B1 and 200B4, and the variable lens 110 shown in FIGS. 2A and 2B may be omitted.

FIG. 9 shows a cross-sectional view of an image sensing unit 200B shown in FIG. 8 according to an embodiment 200B1.

Referring to FIG. 9, the image sensing unit 200B1 may include an actuator 240A, a substrate 250A, and an image sensor 230A. Here, the image sensor 230A and the actuator 240A correspond to one embodiment of the image sensor 230 and the actuator 240 shown in FIG. 8, respectively.

The substrate 250A may be disposed on the actuator 240A and the image sensor 230A may be disposed on the substrate 250A. That is, the actuator 240A may be disposed under the substrate 250 shown in FIG. 2A or 2B, as shown in FIG. The controller 400B moves the actuator 240A in at least one of an optical axis direction (or a direction parallel to the optical axis) (e.g., a z-axis direction) or a horizontal direction perpendicular to the optical axis direction Tilting can be performed. The substrate 250A and the image sensor 230A can move together in the same direction as the direction in which the actuator 240A moves. For example, when the actuator 240A moves in the arrow directions AR1 and AR2, the substrate 250A and the image sensor 230A can move together in the same direction.

Fig. 10 shows a plan view by another embodiment 200B2 of the image sensing unit 200B shown in Fig.

Referring to FIG. 10, the image sensing unit 200B2 may include an actuator 240B, a substrate 250B, elastic members 262 to 268, and an image sensor 230B. Here, the image sensor 230B and the actuator 240B correspond to different embodiments of the image sensor 230 and the actuator 240 shown in FIG. 8, respectively.

The substrate 250B and the image sensor 230B perform the same functions as the substrate 250 and the image sensor 200A shown in Figs. 2A and 2B, respectively. The substrate 250B has a ring-shaped planar shape surrounding the image sensor 230B. The image sensor 230B is disposed inside the annular substrate 250B and may be disposed at a predetermined distance from the inner circumferential surface of the substrate 250B.

The elastic members 262 to 268 are disposed between the outer edge of the image sensor 230B and the inner edge of the substrate 250B so that the position of the image sensor 230B whose physical displacement is changed by the actuator 240B . The actuator 240B may be disposed between the image sensor 230B and the substrate 250B, and may be a MEMS.

When the actuator 240B moves or tilts in one of the horizontal direction and the vertical direction, the image sensor 230B can also move in the same direction as the direction in which the actuator 240B moves.

Fig. 11 shows a plan view of another embodiment 200B3 of the image sensing unit 200B shown in Fig.

Referring to FIG. 11, the image sensing unit 200B3 may include actuators AR1 to AR4 (242 to 248) and an image sensor 230C. Here, the image sensor 230C and the actuators 242 to 248 correspond to another embodiment of the image sensor 230 and the actuator 240 shown in FIG. 8, respectively.

The actuators AC1, AC2, AC3 and AC4 242 to 248 are disposed opposite to the four sides of the image sensor 230C and apply pressure to the image sensor 230C in the arrow directions AR3 to AR6, The sensor 230C can be moved in the horizontal direction. That is, the actuator AC1 242 can move the image sensor 230C in the + x-axis direction by applying pressure in the arrow direction AR3 to the first one of the four sides of the image sensor 230C. The actuator AC2 244 can move the image sensor 230C in the -x axis direction by applying pressure in the arrow direction AR4 to the second side of the four sides of the image sensor 230C. The actuator AC3 246 can move the image sensor 230C in the + y axis direction by applying pressure in the arrow direction AR5 to the third side of the four sides of the image sensor 230C. The actuator AC4 248 can move the image sensor 230C in the -y axis direction by applying pressure in the arrow direction AR6 to the fourth side of the four sides of the image sensor 230C.

11, the image sensing portion 200B3 includes four actuators AC1, AC2, AC3, and AC4 (242 to 248), since the image sensor 230C has a rectangular planar shape. It is not limited. That is, the number of the actuators 242 to 248 may be more or less than four, depending on the plane shape of the image sensor 230C.

11, actuators are additionally disposed on the upper and lower portions of the image sensor 230C to apply pressure to the upper and lower portions of the image sensor 230C, It may be moved in the vertical direction.

As described above, the actuator that applies pressure to the image sensor 230C may be implemented as a piezoelectric element, and the controller 400B may generate a fourth signal that drives the corresponding piezoelectric element among the plurality of piezoelectric elements.

Figs. 12A and 12B show a perspective view of another embodiment 200B4 of the image sensing unit 200B shown in Fig.

Referring to FIG. 12B, the image sensing unit 200B4 may include an image sensor 230D and an actuator 240C. Here, the image sensor 230D and the actuator 240C correspond to another embodiment of the image sensor 230 and the actuator 240 shown in Fig. 8, respectively.

The image sensor 230D may be disposed on the MEMS type actuator 240C shown in Fig. 12A.

When the actuator 240C moves or tilts in at least one of the horizontal direction and the vertical direction under the control of the control unit 400B, the image sensor 230D can also move in the same direction as the direction in which the actuator 240C moves.

The distance by which the image sensors 230A, 230B, 230C and 230D are moved by the actuators 240A, 240B, 242 to 248 and 240C may be 1 mm or less and the tilting angle may be 1 degree or less.

8 may be an image processor that receives an image signal from the image sensor 230 and processes (e.g., interpolates, synthesizes, etc.) the image signal. In particular, the image combining unit 300 may synthesize a video signal (high resolution) of one frame using video signals (low resolution) of a plurality of frames. The plurality of image frames may be each image frame generated by changing the relative position of the image sensor 230 with respect to the lens assembly 100B. The image combining unit 300 may be referred to as a post-processor. The plurality of image frames may include a first image frame and a second image frame, and the second image frame may be an image frame shifted by a first interval based on the first image frame.

Hereinafter, an operation method according to an embodiment of the camera module 10, 20 will be described with reference to the accompanying drawings.

FIG. 13 is a diagram for explaining an operation method of the camera module 10, 20 according to an embodiment. FIG. 14 is a diagram for explaining the operation method of the camera module 10, 20 explained in FIG. 13 in more detail.

13, a schematic diagram of a method for obtaining a super resolution image by changing the optical path incident on the image sensor 200A or changing the relative position of the image sensor 230 with respect to the lens assembly Respectively.

The pixel array of image sensors 200A, 230 may comprise a plurality of pixels arranged in a matrix form of NxM. In the following description, for convenience of description, it is assumed that the pixel array includes a plurality of pixels A1 to A4 arranged in a matrix of 2x2 as in Fig.

Each pixel A1 to A4 generates image information (i.e., an analog pixel signal corresponding to the optical signal) for each pixel scene PS1 to PS4 through an optical signal transmitted through the lens assemblies 100A and 100B .

When the distance between neighboring pixels in the x-axis direction (or the y-axis direction) (for example, the distance between the pixel centers) is 1 PD (pixel distance), half thereof corresponds to 0.5 PD. Here, the first to fourth pixel shifts A to D will be defined.

The first pixel shift A means shifting each pixel A1 to A4 by 0.5 PD to the right along the + x axis direction, and the pixels after the first pixel shift A is completed are B1 to B4.

The second pixel shift B means shifting each pixel B1 to B4 by 0.5 PD downward along the + y axis direction and the pixel after the second pixel shift B is completed is C1 to C4.

The third pixel shift C means shifting each pixel C1 to C4 by 0.5 PD to the left along the -x axis direction and the pixel after the third pixel shift C is completed is D1 to D4.

The fourth pixel shift D means moving each pixel D1 to D4 by 0.5 PD upward along the -y axis direction, and the pixel after the fourth pixel shift D is completed is A1 to A4.

Here, the pixel shifting does not move the physical location of the pixel in the pixel array, but rather the variable pixel (e.g., B1) between two pixels (e.g., A1 and A2) Refers to an operation of adjusting at least one of the lens 110 and the actuator 240 to adjust the path of the transmitted light or the relative position of the image sensor 230 with respect to the lens assembly.

Referring to Fig. 14, each pixel A1 to A4 acquires a pixel scene S1, and the image sensors 200A and 230 generate a first frame F1 from the pixel signals of the respective pixels A1 to A4 .

According to the lens control signal for changing the optical path or FOV of the lens assembly 100A to the right by the first angle variation amount? X for the first pixel shift A, the variable lens 110 is moved to the lens assembly 100A The first pixel shift A can be performed by changing the optical path or FOV of the first pixel shift right by the first angle change amount [theta] x. Alternatively, according to a fourth signal that causes the relative position of the image sensor 230 relative to the lens assembly 100B to move to the right by the first angle variation amount? X for the first pixel shift A, the actuator 240 The first pixel shift A may be performed by changing the relative position of the image sensor 230 to the right by the first angle variation amount X. Each of the pixels B1 to B4 may then acquire the pixel scene S2 and the image sensors 200A and 230 may generate the second frame F2 from the pixel signals of the respective pixels B1 to B4.

In accordance with the lens control signal that causes the optical path or FOV of the lens assembly 100A to change by the second angle variation amount? Y for the second pixel shift B, the variable lens 110 moves the lens assembly 100A The second pixel shift B can be performed by changing the optical path or the FOV of the second pixel shift down by the second angle change amount [theta] y. Or a fourth signal that causes the relative position of the image sensor 230 relative to the lens assembly 100B to change by the second angle variation amount? Y for the second pixel shift B, the actuator 240 The second pixel shift B can be performed by changing the relative position of the image sensor 230 by the second angle variation amount? Y. Each of the pixels C1 to C4 may then acquire the pixel scene S3 and the image sensors 200A and 230 may generate the third frame F3 from the pixel signals of the respective pixels C1 to C4.

In accordance with the lens control signal for changing the optical path or FOV of the lens assembly 100A to the left by the first angle variation amount? X for the third pixel shift C, the variable lens 110 is moved to the lens assembly 100A The third pixel shift C can be performed by changing the optical path or FOV of the first pixel shift amount? Alternatively, according to a fourth signal that causes the relative position of the image sensor 230 relative to the lens assembly 100B to change to the left by the first angle variation amount? X for the third pixel shift C, the actuator 240 The third pixel shift C can be performed by changing the relative position of the image sensor 230 to the left by the first angle variation amount? X. Each pixel D1 through D4 may then acquire a pixel scene S4 and the image sensors 200A and 230 may generate a fourth frame F4 from the pixel signals of each pixel D1 through D4.

The variable lens 110 is moved to the lens assembly 100A in accordance with the lens control signal for changing the optical path or FOV of the lens assembly 100A by the second angle variation amount? The fourth pixel shift D can be performed by changing the optical path or FOV of the first pixel shift by the second angle change amount [Theta] y. Or the fourth signal that causes the relative position of the image sensor 230 relative to the lens assembly 100B to change by the second angle variation amount? Y for the fourth pixel shift D, The fourth pixel shift D may be performed by changing the relative position of the image sensor 230 by the second angle variation amount? Y. Each pixel A1 to A4 may then acquire a pixel scene S1 and the image sensors 200A and 230 may generate a fifth frame F5 from the pixel signals of each pixel A1 to A4. Thereafter, the pixel movement and the frame generation operation through the moved pixel can be repeatedly performed.

Here, information corresponding to the degree of optical path change or the degree of change of the relative position of the image sensor 230 is stored so that the pixel shift can be performed by 0.5 PD for each of the first angle change amount? X and the second angle change amount? And may be calculated and stored in advance based on the first FOV angle Fx and the second FOV angle Fy (for example, the image sensor 200A or 230 or the image synthesizer 300 or the controller 400A , 400B).

The image sensor 200A includes a first area and a second area and the control part 400A controls the light from the outside to pass through the variable lens 110 from the first area to the second area of the image sensor 200A It is possible to output the first signal for adjusting the variable lens 110 to change the path of the light. Alternatively, the image sensor 230 may include a first area and a second area, and the control unit 400B may include a first area of the image sensor 230 that receives light from the outside and receives light passing through the lens assembly 100B, And the fourth signal for adjusting the actuator 240 to change from the first region to the second region.

The image sensor 200A further includes a third area and a fourth area and the control part 400A controls the variable lens 110 to change the light path from the second area to the third area of the image sensor 200A. And outputs a first signal for adjusting the variable lens 110 to change the light path from the third region to the fourth region. Alternatively, the image sensor 230 may further include a third area and a fourth area, and the control unit 400B may include a region for receiving light from the outside and receiving light passing through the lens assembly 100B, A fourth signal for adjusting the actuator 240 so as to change from the second area to the third area and a fourth signal for adjusting the actuator 240 so that the area where light is received from the third area to the fourth area is changed, Can be output.

The first signal is a signal for changing the angle of view (FOV) of the lens assembly 100A in the first direction, a signal for changing the angle of view of the lens assembly 100A in the second direction, a signal for changing the angle of view of the lens assembly 100A in the third direction And a signal for changing the angle of view of the lens assembly 100A to a fourth direction. Alternatively, the fourth signal may be a signal for changing the angle of view (FOV) of the image sensor 230 in the first direction, a signal for changing the angle of view of the image sensor 230 in the second direction, A signal for changing the angle of view of the image sensor 230 in the fourth direction, and a signal for changing the angle of view of the image sensor 230 in the fourth direction.

The image synthesizer 300 may be an image processor that receives an image signal from the image sensor 200A and processes (e.g., interpolates, frame synthesizes, etc.) the image signal. In particular, the image combining unit 300 may synthesize a video signal (high resolution) of one frame using video signals (low resolution) of a plurality of frames. The plurality of image frames may be each image frame generated by changing the optical path by the variable lens 110. The image combining unit 300 may be referred to as a post-processor. The plurality of image frames may include a first image frame and a second image frame, and the second image frame may be an image frame shifted by a first interval based on the first image frame.

The image synthesizer 300 may generate the image obtained by the 2Nx2M pixel array rather than the NxM pixel array by composing the first through fourth frames. The method of synthesizing the first through fourth frames by the image composition unit 300 may include simply merging the first through fourth frames according to the positions of the respective pixels (for example, pixel signals of A1 in the first row, B1 (For example, C1) by using a method of arranging the pixel signal of the pixel A 1, the pixel signal of the pixel A 2, the pixel signal of the pixel B 2, and the pixel signal of the pixel B 2, A method of correcting the pixel value of the pixel by using pixel signals of adjacent pixels (e.g., A1, B1, A2, D1, D2, A3, B3, A4) may be used. However, A generation method can be used. The post-processor synthesizes the first super resolution image frame using the first through fourth image frames transmitted from the image sensors 200A and 230, The second super resolution image frame may be synthesized using the fifth image frame and the second through fourth image frames output from the sensors 200A and 230. [

According to the operation method of the camera modules 10 and 20 shown in FIGS. 13 and 14, an image having a resolution of four times can be generated by combining a plurality of frames obtained through pixel shifting.

15 is a timing diagram of an operation method of a camera module 10, 20 according to an embodiment.

15, in accordance with a lens control signal for changing the optical path or FOV of the lens assembly 100A by the second angle variation amount? Y, the variable lens 110 is moved in the optical path of the lens assembly 100A Or a fourth pixel shift (D) that changes the FOV by the second angle variation amount? Y. According to the embodiment, the controller 400A may transmit a feedback signal to the image sensor 200A indicating that the fourth pixel shift (D) by the variable lens 110 is completed according to the first signal. At this time, the controller 400A may determine that the fourth pixel movement (D) is completed through a response signal from the variable lens 110 or a separate timer. Each pixel A1 to A4 of the image sensor 200A that has received the feedback signal acquires the pixel scene S1 and the image sensor 200A receives the pixel signal of each pixel A1 to A4 from the first frame F1, Can be generated.

Alternatively, in accordance with a fourth signal that causes the image sensor 230 to be changed by the second angle variation amount? Y, the actuator 240 changes the image sensor 230 by a second angle variation amount? A four-pixel shift (D) can be performed. According to an embodiment, the control unit 400B may transmit a feedback signal to the image sensor 230 indicating that the fourth pixel movement (D) by the actuator 240 has been completed according to the fourth signal. At this time, the controller 400B may determine that the fourth pixel movement (D) is completed through a response signal from the actuator 240 or a separate timer. Each pixel A1 to A4 of the image sensor 230 that has received the feedback signal acquires the pixel scene S1 and the image sensor 230 generates the first frame F1 from the pixel signals of each pixel A1 to A4, Can be generated.

The variable lens 110 changes the optical path or the FOV of the lens assembly 100A to the first optical path or the FOV of the lens assembly 100A in accordance with the first signal for changing the optical path or the FOV of the lens assembly 100A to the right by the first angle variation amount [ It is possible to perform the first pixel shift (A) which shifts rightward by the angle change amount? X. According to the embodiment, the controller 400A may transmit a feedback signal to the image sensor 200A indicating that the first pixel shift (A) by the variable lens 110 is completed according to the first signal. At this time, the controller 400A may determine that the first pixel movement (A) is completed through a response signal from the variable lens 110 or a separate timer. Each of the pixels B1 to B4 of the image sensor 200A that has received the feedback signal acquires the pixel scene S2 and the image sensor 200A receives the pixel signals of the pixels B1 to B4 from the second frame F2, Can be generated.

Alternatively, in accordance with the fourth signal for changing the image sensor 230 to the right by the first angle variation amount? X, the actuator 240 changes the image sensor 230 to the right by the first angle variation amount? X (A) < / RTI > According to an embodiment, the controller 400B may transmit a feedback signal to the image sensor 230 indicating that the first pixel shift (A) by the actuator 240 has been completed according to the fourth signal. At this time, the controller 400B may determine that the first pixel movement (A) is completed through a response signal from the actuator 240 or a separate timer. Each pixel B1 to B4 of the image sensor 230 that has received the feedback signal acquires the pixel scene S2 and the image sensor 230 receives the pixel signal of each pixel B1 to B4 from the second frame F2, Can be generated.

According to the lens control signal for changing the optical path or FOV of the lens assembly 100A by the second angle variation amount? Y, the variable lens 110 changes the optical path or FOV of the lens assembly 100A to the second It is possible to perform a second pixel shift (B) in which the pixel shift is changed by an angle change amount? Y. According to the embodiment, the control unit 400A may transmit a feedback signal to the image sensor 200A indicating that the second pixel shift (B) by the variable lens 110 has been completed in accordance with the lens control signal. At this time, the controller 400A can determine that the second pixel movement (B) is completed through the response signal from the variable lens 110 or a separate timer. Each pixel C1 to C4 of the image sensor 200A that has received the feedback signal acquires the pixel scene S3 and the image sensor 200A receives the pixel signal of each pixel C1 to C4 from the third frame F3, Can be generated.

Alternatively, in accordance with the fourth signal for changing the image sensor 230 by the second angle variation amount? Y, the actuator 240 changes the image sensor 230 by the second angle variation amount? Y To perform a second pixel shift (B). According to an embodiment, the control unit 400B may transmit a feedback signal to the image sensor 230 indicating that the second pixel shift (B) by the actuator 240 has been completed according to the fourth signal. At this time, the controller 400B may determine that the second pixel movement (B) is completed through a response signal from the actuator 240 or a separate timer. Each pixel C1 to C4 of the image sensor 230 that has received the feedback signal acquires the pixel scene S3 and the image sensor 230 acquires the pixel signal of each pixel C1 to C4 from the third frame F3, Can be generated.

The variable lens 110 changes the optical path or the FOV of the lens assembly 100A to the first optical path or the FOV in accordance with the lens control signal for changing the optical path or FOV of the lens assembly 100A to the left by the first angle variation amount [ A third pixel shift (C) for shifting to the left by an angle change amount? X can be performed. According to the embodiment, the control unit 400A may transmit a feedback signal to the image sensor 200A indicating that the third pixel shift (C) by the variable lens 110 has been completed in accordance with the lens control signal. At this time, the controller 400A may determine that the third pixel shift (C) is completed through a response signal from the variable lens 110 or a separate timer. Each pixel D1 to D4 of the image sensor 200A that has received the feedback signal acquires the pixel scene S4 and the image sensor 200A receives the pixel signal of each pixel D1 to D4 from the fourth frame F4, Can be generated.

Alternatively, in accordance with the fourth signal for changing the image sensor 230 to the left by the first angle variation amount? X, the actuator 240 changes the image sensor 230 to the left by the first angle variation amount? X A third pixel shift (C) may be performed. According to an embodiment, the control unit 400B may transmit a feedback signal to the image sensor 230 indicating that the third pixel shift (C) by the actuator 240 has been completed according to the fourth signal. At this time, the controller 400B may determine that the third pixel shift (C) is completed through a response signal from the actuator 240 or a separate timer. Each of the pixels D1 to D4 of the image sensor 230 that has received the feedback signal acquires the pixel scene S4 and the image sensor 230 generates the fourth frame F4 from the pixel signals of the pixels D1 to D4, Can be generated.

Thereafter, in accordance with the lens control signal for changing the optical path or FOV of the lens assembly 100A by the second angle variation amount? Y, the variable lens 110 adjusts the optical path or the FOV of the lens assembly 100A A fourth pixel shift (D) in which the pixel shift is changed by two angle change amounts (? Y). According to the embodiment, the control unit 400A may transmit a feedback signal to the image sensor 200A indicating that the fourth pixel movement (D) by the variable lens 110 has been completed in accordance with the lens control signal. At this time, the controller 400A may determine that the fourth pixel movement (D) is completed through a response signal from the variable lens 110 or a separate timer. Each pixel A1 to A4 of the image sensor 200A that has received the feedback signal acquires the pixel scene S1 and the image sensor 200A receives the pixel signal of each pixel A1 to A4 from the fifth frame F5, Can be generated. Thereafter, the pixel movement and the frame generation operation through the moved pixel can be repeatedly performed.

Alternatively, in accordance with a fourth signal that causes the image sensor 230 to be changed by the second angle variation amount? Y, the actuator 240 changes the image sensor 230 by a second angle variation amount? A four-pixel shift (D) can be performed. According to an embodiment, the control unit 400B may transmit a feedback signal to the image sensor 230 indicating that the fourth pixel movement (D) by the actuator 240 has been completed according to the fourth signal. At this time, the controller 400B may determine that the fourth pixel movement (D) is completed through a response signal from the actuator 240 or a separate timer. Each of the pixels A1 to A4 of the image sensor 230 that has received the feedback signal acquires the pixel scene S1 and the image sensor 230 generates the fifth frame F5 from the pixel signals of the pixels A1 to A4, Can be generated. Thereafter, the pixel movement and the frame generation operation through the moved pixel can be repeatedly performed.

15, the transmission of the lens control signal of the control unit 400A is performed by transmitting a synchronization signal for instructing the generation of the image frame by the image sensor 200A to transmit the lens control signal to the variable lens 110, (200A) is transmitted. That is, sequential operations of pixel movement, frame generation, and subsequent pixel movement can be performed by synchronizing through transmission and reception of the first signal and the second signal.

16 is a diagram showing an example of a frame synthesis method of the camera module 10, 20 according to the embodiment.

16, it is assumed that the image sensors 200A and 230 sequentially generate first to seventh frames F1 to F7 according to sequential pixel shifts A to D, respectively.

The image synthesizer 300 sequentially receives the frames and generates a synthesized frame that is a super-resolution image by synthesizing a super-resolution (SR) image.

At this time, as shown in FIG. 16, the image synthesizer 300 may generate the first synthesis frame F1 'by receiving the first to fourth frames F1 to F4. Thereafter, the image combining unit 300 may generate the second combined frame F2 'by receiving the second through fifth frames F2 through F5. Thereafter, the image combining unit 300 may receive the third through sixth frames F3 through F6 to generate the third combined frame F3 '. The image combining unit 300 receives the fourth through seventh frames F4 through F7 and generates a fourth combined frame F4 'through a super-resolution image generating algorithm.

Here, the image combining unit 300 sequentially receives the first through seventh frames F1 through F7 from the image sensors 200A and 230, and stores three frames immediately before the current input frame for generating a composite frame . According to an embodiment, the buffer storing the frames may have a storage capacity capable of storing at least three frames.

If a composite frame is generated using the first to fourth frames and then a composite frame is generated using the fifth to eighth frames, the frame rate is reduced to 1/4 of the original frame rate. However, it is possible to prevent a decrease in the frame rate by continuously generating synthesized frames using the current frame sequentially input and the three frames immediately before the current frame as in the system according to the embodiment.

In the present specification, a method of generating a super-resolution image having a resolution of four times by moving the pixel four times has been described. However, the scope of the present invention is not limited thereto, Images can be generated.

A camera module according to another embodiment includes: a lens assembly including a liquid lens for adjusting a light path; An image sensor for sensing a plurality of images using the lens assembly; A control unit for controlling the liquid lens; And a combining unit that combines the plurality of images to generate a composite image, and the plurality of images may include an image generated by the liquid lens changing the optical path differently.

A camera module according to another embodiment includes an image sensor for sensing a plurality of images; A lens assembly forming a light path to be incident on the image sensor; A control unit for adjusting at least one of the position of the optical path and the image sensor; And an image combining section for combining the plurality of images to generate a composite image, wherein the plurality of images may be images generated by different optical paths by the lens assembly, or may include images generated at different positions of the image sensor .

Hereinafter, a super-resolution image generation method performed by the camera modules 10 and 20 will be described as follows.

A method for generating a super-resolution image, comprising: outputting a first image frame; generating a second image frame shifted by a first distance in a first direction in a first image frame; Creating a third image frame shifted by one distance, generating a fourth image frame shifted by a first distance in a third direction in the third image frame, and And generating a composite image. A composite image generated through this method can have a higher resolution than a plurality of image frames.

According to the camera modules 10 and 20 according to the embodiments, a high computation amount required for obtaining a super-resolution image can be solved in hardware by using the variable lens 110 or the actuator 240 for changing the FOV angle .

In addition, it is possible to prevent the frame rate from lowering by continuously generating composite frames for the current frame sequentially input.

The camera module 10 described above includes a lens assembly including a liquid lens mounted on a housing and at least one solid lens that can be disposed on a front surface or a rear surface of the liquid lens, And a control circuit for supplying a driving voltage to the liquid lens. At this time, the lens assembly of the camera module 20 may not include a liquid lens but may include only a solid lens.

The above contents can also be applied to a distance or depth measuring apparatus using time of flight (ToF). In particular, the ToF sensor generally has a lower resolution than a general image sensor. Therefore, using the ToF sensor to implement facial recognition, object recognition, depth extraction, contour recognition, and the like, a remarkably improved effect can be obtained by using the super resolution image implementation method described above.

While only a few have been described above with respect to the embodiments, various other forms of implementation are possible. The technical contents of the embodiments described above may be combined in various forms other than the mutually incompatible technologies, and may be implemented in a new embodiment through the same.

For example, an optical device (optical device) including the camera module 10, 20 including at least one of the above-described liquid lens or solid lens can be implemented. Here, the optical device may include a device capable of processing or analyzing an optical signal. Examples of optical devices may include camera / video devices, telescope devices, microscope devices, interferometer devices, photometer devices, polarimeter devices, spectrometer devices, reflectometer devices, autocollimator devices, lens meter devices, The present embodiment can be applied to an optical device that can include at least one of the above-described optical elements. In addition, the optical device can be implemented as a portable device such as a smart phone, a notebook computer, and a tablet computer. Such an optical apparatus may include a camera module, a display unit for outputting an image, and a main body housing for mounting the camera module and the display unit. The optical device may further include a memory unit in which a communication module capable of communicating with other devices can be mounted on the body housing and can store data.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

The mode for carrying out the invention has been fully described in the above-mentioned " Best Mode for Carrying Out the Invention ".

The camera module according to the embodiment can be used for a camera / video device, a telescope device, a microscope device, an interferometer device, a photometer device, a polarimeter device, a spectrometer device, a reflectometer device, an auto collimator device,

Claims (10)

1. An image sensor for outputting a plurality of image frames;

   A lens assembly disposed on the image sensor and including a variable lens that adjusts a light path that is incident on the image sensor from the outside;

   A control unit for generating a control signal for controlling the variable lens; And

   And an image composing unit for compositing the plurality of image frames to generate a composite image,

   Wherein the composite image has a higher resolution than the plurality of image frames,

   Wherein the plurality of image frames are each image frames generated by changing the optical path by the variable lens.
2. 2. The apparatus of claim 1, wherein the image sensor generates an image frame of one of the plurality of image frames, receives a feedback signal indicating that the optical path by the variable lens is adjusted, A camera module that generates image frames.
3. 2. The image sensor of claim 1, wherein the image sensor comprises a first region and a second region,

   Wherein,

   And outputs a signal for adjusting the variable lens to change the light path from the outside to the second area from the first area of the image sensor, through the variable lens.
4. 2. The camera module according to claim 1, wherein the variable lens includes a liquid lens whose interface is deformed by two liquids of different properties according to the control signal.
5. 2. The zoom lens according to claim 1, wherein the variable lens

   At least one lens; And

   And an actuator for moving or tilting the at least one lens in at least one of a vertical direction and a horizontal direction according to the control signal.
6. The method according to claim 1,

   Wherein the image sensor comprises a first region and a second region,

   Wherein,

   And outputs a signal for adjusting the variable lens to change the light path from the outside to the second area from the first area of the image sensor, through the variable lens.
7. The method according to claim 6,

   Wherein the image sensor further comprises a third region and a fourth region,

   Wherein,

   A signal for adjusting the variable lens to change the light path from the second region to the third region,

   And outputs a signal for adjusting the variable lens to change the light path from the third region to the fourth region.
8. The method according to claim 1,

   Wherein the image synthesizing unit comprises:

   Synthesizing the first super resolution image frame using the first through fourth image frames transmitted from the image sensor,

   And then synthesizing a second super-resolution image frame using a fifth image frame output from the image sensor and the second through fourth image frames.
9. A lens assembly including a liquid lens for adjusting a light path;

   An image sensor for sensing a plurality of images using the lens assembly;

   A control unit for controlling the liquid lens; And

   And a combining unit for combining the plurality of images to generate a composite image,

   Wherein the plurality of images comprises an image generated by the liquid lens having different optical paths.
10. An image sensor for sensing a plurality of images;

    A lens assembly forming a light path to be incident on the image sensor;

    A control unit for adjusting at least one of the optical path and the position of the image sensor; And

    And an image composing unit for compositing the plurality of images to generate a composite image,

    Wherein the plurality of images comprises images generated by different light paths by the lens assembly or images generated at different locations of the image sensor.

Images