WO2020159160A1

WO2020159160A1 - Optical device including liquid lens and having chromatic aberration correction function, and image processing method

Info

Publication number: WO2020159160A1
Application number: PCT/KR2020/001163
Authority: WO
Inventors: 정재욱
Original assignee: 엘지이노텍(주)
Priority date: 2019-01-30
Filing date: 2020-01-23
Publication date: 2020-08-06
Also published as: KR20200094509A

Abstract

An optical device of an embodiment comprises: a lens unit including a liquid lens; an image sensor that receives information of light passing through the lens unit via an R channel that receives a red wavelength range, a G channel that receives a green wavelength range, and a B channel that receives a blue wavelength range; and an image processing unit that shifts information of light, received through at least one correction channel from among the R channel, the G channel, and the B channel, by a preset movement amount corresponding to driving information of the liquid lens and field information on the image sensor of the correction channel.

Description

Optical device and image processing method with liquid lens and chromatic aberration correction

An embodiment relates to an optical device and an image processing method including a liquid lens and having chromatic aberration correction.

In the conventional optical device, a camera lens including a solid lens is designed such that chromatic aberration is suppressed as much as possible, so that a separate chromatic aberration correction is not required in an image process step after taking an image. Alternatively, if necessary, the chromatic aberration can be corrected easily because the user can directly remove chromatic aberration through candidate correction, the chromatic aberration amount is very small, and the shape in which the chromatic aberration is generated is symmetrical up/down/left/right.

However, in the case of a camera lens including a liquid lens, chromatic aberration exhibits the same aspect as a camera lens including a solid lens in normal shooting, that is, in a normal state, in which an auto-focusing (AF) function is performed. On the other hand, when a camera lens including a liquid lens performs image stabilization or image stabilization (OIS), chromatic aberration is largely generated and the amount of chromatic aberration varies depending on the OIS driving frequency, but OIS function is performed. It will increase approximately 5 to 8 times compared to when it is not. Here, the OIS driving frequency means a frequency for the shaking of a user who uses a camera lens.

In addition, in order to perform the OIS function, unlike the conventional method of using a VCM (Voice Coil Motor) that transverses in the x-axis and y-axis directions on the xy plane perpendicular to the optical axis, a camera lens including a liquid lens Since the use of tilting of the interface between the two liquids, the shape of chromatic aberration is also asymmetrical up/down/left/right. Due to this, there is a problem that the image quality of the photographed image is significantly deteriorated.

An embodiment includes a liquid lens capable of correcting chromatic aberration and providing improved image quality, and provides an optical device and an image processing method having a chromatic aberration correction function.

An optical device according to an embodiment includes a lens unit including a liquid lens; An image sensor receiving information of light passing through the lens unit through an R channel receiving a red wavelength range, a G channel receiving a green wavelength range, and a B channel receiving a blue wavelength range; And shifting information of light received through at least one correction channel among the R channel, the G channel, and the B channel by a preset movement amount corresponding to the driving information of the liquid lens and field information on the image sensor of the correction channel. It may include an image processing unit.

For example, among the R channel, the G channel, and the B channel, the correction channel may be a B channel and an R channel, and the G channel may be a reference channel.

For example, the image processing unit maps and stores a correction angle of the liquid lens for each of a plurality of compensation angles, and stores the first movement amount of each field and the R channel of the image sensor of the B channel for each of the correction angles. A look-up table that maps and stores the second movement amount for each field on the image sensor; The first movement amount for each field of the B channel corresponding to the compensation angle corresponding to the driving information of the liquid lens, and the second movement amount for each field of the R channel corresponding to the compensation angle are output from the lookup table. Control unit; And shifting information of the light received on the B channel by the first movement amount for each field output from the lookup table by field, and receiving information on the light received on the R channel by the second movement amount for each field output from the lookup table. An image shifting unit may be shifted for each field.

For example, the image processing unit may include a correction angle determining unit corresponding to a compensation angle corresponding to the driving information of the liquid lens and obtaining a preset correction angle of the liquid lens; A movement amount determination unit corresponding to the correction angle and determining a first movement amount for each field on the image sensor of the B channel and a second movement amount for each field on the image sensor of the R channel; And an image shifting unit shifting the information of the light received through the B channel by the first movement amount for each field, and shifting the information of the light received by the R channel by the second movement amount for each field.

For example, the optical device may generate a corrected final image by synthesizing information of light received on the G channel, information of light received on the B channel and shifted, and information on light received on the R channel and shifted. A synthetic part may be further included.

For example, the optical device may further include a hand shake detection unit that converts user hand shake information corresponding to the driving information of the liquid lens into a compensation angle.

For example, the image stabilization unit may include a gyro sensor that senses the degree of image stabilization of the user and converts the driving information of the liquid lens with respect to the sensed image amount to a compensation angle.

According to another embodiment, a lens unit including a liquid lens; An image sensor that receives information of light passing through the lens unit; And an image processing unit for correcting the information of the light, comprising: converting driving information of the liquid lens into a compensation angle; Obtaining a correction angle of the preset liquid lens corresponding to the compensation angle; Corresponding to the correction angle, the field information on the image sensor of at least one correction channel among the R channel receiving the red wavelength range, the G channel receiving the green wavelength range, and the B channel receiving the blue wavelength range. Determining a corresponding preset movement amount; And shifting information of light received through the correction channel by the determined amount of movement for each field on the image sensor.

For example, among the R channel, the G channel, and the B channel, the correction channel may be an R channel and a B channel, and the G channel may be a reference channel.

For example, the determining of the movement amount may include determining a first movement amount for each field on the image sensor of the B channel that corresponds to the correction angle; And

The method may include determining a second movement amount for each field on the image sensor of the R channel corresponding to the correction angle.

For example, the shifting step may include shifting information of light received through the B channel by the first movement amount for each field on the image sensor; And shifting the information of the light received through the R channel by the second movement amount for each field on the image sensor.

For example, the image processing method may generate a corrected final image by synthesizing information of light received on the G channel, information of light received on the R channel and shifted, and information of light received on the B channel and shifted. It may further include a step.

The optical device and the image processing method including a liquid lens according to an embodiment and having a chromatic aberration correction function are improved by correcting chromatic aberration by shifting information of light received through a compensation channel by a preset amount of movement for each field according to the degree of hand movement of the user It can provide image quality.

In addition, the effects obtainable in the present embodiment are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description. There will be.

1 is a block diagram of an optical device according to an embodiment.

FIG. 2 is a cross-sectional view of the liquid lens unit including the liquid lens illustrated in FIG. 1.

3 shows an implementation example according to an embodiment of the optical device shown in FIG. 1.

4 shows a planar shape according to the embodiment of the image sensor shown in FIG. 1.

5 shows the relationship between the amount of chromatic aberration and the field of the image sensor when the interface of the liquid lens is not tilted.

6 shows the relationship between the aberration amount and the field of the image sensor when the interface of the liquid lens is tilted.

7 is a block diagram according to an embodiment of the image processing unit illustrated in FIG. 1.

8 shows an example of data stored in the LUT.

9 is a block diagram according to another embodiment of the image processing unit shown in FIG. 1.

10 is a flowchart illustrating an image processing method according to an embodiment.

11 is a view for explaining chromatic aberration correction according to an embodiment.

12 is a diagram for describing an image in which chromatic aberration is corrected by an optical device and an image processing method according to an embodiment.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

However, the technical spirit of the present invention is not limited to some described embodiments, and may be implemented in various different forms, and within the scope of the technical spirit of the present invention, one or more of its components between embodiments may be selectively selected. It can be used by bonding and substitution.

In addition, the terms used in the embodiments of the present invention (including technical and scientific terms), unless specifically defined and described, can be generally understood by those skilled in the art to which the present invention pertains. It can be interpreted as meaning, and commonly used terms, such as predefined terms, may interpret the meaning in consideration of the contextual meaning of the related technology.

In addition, the terms used in the embodiments of the present invention are for describing the embodiments and are not intended to limit the present invention. In the present specification, the singular form may also include the plural form unless specifically stated in the phrase, and is combined with A, B, C when described as “at least one (or more than one) of A and B, C”. It can contain one or more of all possible combinations.

In addition, in describing the components of the embodiments of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only for distinguishing the component from other components, and the term is not limited to the nature, order, or order of the component.

And, when a component is described as being'connected','coupled' or'connected' to another component, the component is not only directly connected, coupled or connected to the other component, but also to the component It may also include the case of'connected','coupled' or'connected' due to another component between the other components.

In addition, when described as being formed or disposed in the “top (top) or bottom (bottom)” of each component, the top (top) or bottom (bottom) is not only when two components are in direct contact with each other, but also one It also includes a case in which another component described above is formed or disposed between two components. In addition, when expressed as “up (up) or down (down)”, it may include the meaning of the downward direction as well as the upward direction based on one component.

The variable lens may be a variable focus lens. Also, the variable lens may be a lens whose focus is adjusted. The variable lens may be at least one of a liquid lens, a polymer lens, a liquid crystal lens, a VCM type, and an SMA type. The liquid lens may include a liquid lens including one liquid and a liquid lens including two liquids. The liquid lens including one liquid may change the focus by adjusting the membrane disposed at a position corresponding to the liquid, and for example, the focus may be changed by pressing the membrane by the electromagnetic force of the magnet and the coil. The liquid lens including two liquids may control the interface formed by the conductive liquid and the non-conductive liquid by using a voltage applied to the liquid lens, including the conductive liquid and the non-conductive liquid. The polymer lens can change the focus of the polymer material through a driving unit such as a piezo. The liquid crystal lens can change the focus by controlling the liquid crystal by electromagnetic force. The VCM type can change the focus by adjusting the solid lens or the lens assembly including the solid lens through an electromagnetic force between the magnet and the coil. The SMA type may use a shape memory alloy to control a solid lens or a lens assembly including the solid lens to change focus.

Hereinafter, the optical device having the chromatic aberration correction function according to the embodiment will be described as including a liquid lens as a variable lens, and the image processing method according to the embodiment will be described as being performed at an optical device including a liquid lens as a variable lens. , The embodiment is not limited thereto. That is, the following description of the optical device can be applied to an optical device including a variable lens other than a liquid lens, and the description of the following image processing method is performed by an optical device including a variable lens other than a liquid lens It can also be applied to the video processing method.

Hereinafter, an optical device including a liquid lens according to an embodiment and having a chromatic aberration correction function will be described with reference to the accompanying drawings.

1 is a block diagram of an optical device 100 according to an embodiment, and includes a lens unit (or lens assembly) 110, an image sensor 120, an image processing unit 130, and an image synthesis unit 140. Can.

The lens unit 110 may include a liquid lens 110A.

Although the general liquid lens 110A is described with reference to FIG. 2 attached, the embodiment is not limited to the liquid lens 110A having a specific structure.

2 is a cross-sectional view of a liquid lens unit including the liquid lens 110A shown in FIG. 1.

The liquid lens unit illustrated in FIG. 2 may include a liquid lens 110A and first and second connecting

substrates

116 and 118.

The liquid lens 110A includes a plurality of different types of liquids LQ1, LQ2, first to third plates P1, P2, P3, first and second electrodes E1, E2, and an insulating layer 119. It may include.

A plurality of liquids (LQ1, LQ2) is accommodated in a cavity (CA:cavity), the first liquid having a conductivity (LQ1) and non-conductive and can be embodied as a non-conductive material such as oil (oil) 2 liquid (or insulating liquid) (LQ2). The first liquid LQ1 and the second liquid LQ2 do not mix with each other, and an interface BO may be formed at a contact portion between the first and second liquids LQ1 and LQ2. The first liquid LQ1 may be disposed on the second liquid LQ2, or the second liquid LQ2 may be disposed on the first liquid LQ1.

The inner surface of the first plate P1 may form a side wall i of the cavity CA. The first plate P1 may include upper and lower openings having a predetermined inclined surface. That is, the cavity CA may be defined as an area surrounded by an inclined surface of the first plate P1, a first opening contacting the second plate P2, and a second opening contacting the third plate P3.

The diameter of the wider opening among the first and second openings may vary depending on the field of view (FOV) required by the liquid lens 110A or the role of the liquid lens 110A. The interface BO formed by the two liquids may move along the inclined surface of the cavity CA by the driving voltage.

The first liquid LQ1 and the second liquid LQ2 may be filled, received, or disposed in the cavity CA of the first plate P1. Further, the cavity CA is a site through which light passes. Therefore, the first plate P1 may be made of a transparent material, or may contain impurities so that light transmission is not easy.

First and second electrodes E1 and E2 may be disposed on one surface and the other surface of the first plate P1, respectively. The plurality of first electrodes E1 are spaced apart from the second electrode E2 and may be disposed on the upper surface, side surfaces, and lower surfaces of the first plate P1. The second electrode E2 is disposed in at least a portion of the lower surface of the first plate P1 and may directly contact the first liquid LQ1.

As described above, the plurality of first electrodes E1 may correspond to individual electrodes that can be electrically separated from each other, and the plurality of second electrodes E2 may correspond to common electrodes that may not be electrically separated from each other. have. A portion of the second electrode E2 disposed on the other surface of the first plate P1 may be exposed to the first liquid LQ1 having conductivity.

In addition, the second plate P2 may be disposed on the upper surface of the first electrode E1 and the cavity CA. The third plate P3 may be disposed under the second electrode E2 and under the cavity CA. The second plate P2 and the third plate P3 may be disposed to face each other with the first plate P1 therebetween. Also, at least one of the second plate P2 or the third plate P3 may be omitted.

Each of the second and third plates P2 and P3 is a region through which light passes, and may be made of a translucent material. For example, each of the second and third plates P2 and P3 may be made of glass, and may be formed of the same material for convenience of processing.

The second plate P2 may have a configuration that allows the incident light to proceed into the cavity CA of the first plate P1. The third plate P3 may have a configuration that allows light passing through the cavity CA of the first plate P1 to be emitted. The third plate P3 may directly contact the first liquid LQ1.

The insulating layer 119 may be disposed while covering a part of the lower surface of the second plate P2 in the upper region of the cavity CA. That is, the insulating layer 119 may be disposed between the second liquid LQ2 and the second plate P2. In addition, the insulating layer 119 may be disposed while covering at least a portion of the first electrode E1 forming the sidewall of the cavity CA. In addition, the insulating layer 119 may be disposed on the lower surface of the first plate P1, covering a portion of the second electrode E2 and the first plate P1 and the first electrode E1. Due to this, the contact between the first electrode E1 and the first liquid LQ1 and the contact between the first electrode E1 and the second liquid LQ2 may be blocked by the insulating layer 119.

The insulating layer 119 covers one electrode (eg, the first electrode E1) of the first and second electrodes E1 and E2, and the other electrode (eg, the second electrode E2). )) to expose a portion of the first liquid (LQ1) having conductivity so that electrical energy is applied.

The first connecting substrate 116 and the second connecting substrate 118 serve to supply voltage to the liquid lens 110A. To this end, the first electrode E1 may be electrically connected to the first connection substrate 116, and the second electrode E2 may be electrically connected to the second connection substrate 118.

When a driving voltage is applied to the first and second electrodes E1 and E2 through the first connecting substrate 116 and the second connecting substrate 118, between the first liquid LQ1 and the second liquid LQ2 Deformation of the interface BO of at least one of the shape or focal length, such as the curvature of the liquid lens 110A, may be changed (or adjusted). For example, the focal length of the liquid lens 110A may be adjusted while at least one of the bending or inclination of the interface BO formed in the liquid lens 110A changes in response to the driving voltage. When the deformation, curvature radius, tilting angle, etc. of the interface BO is controlled, the optical device including the liquid lens 110A has an Auto-Focusing (AF) function or image stabilization or image stabilization (OIS: Optical) Image Stabilizer) function.

3 shows an implementation example according to an embodiment of the optical device 100 shown in FIG. 1.

The lens unit 110 of the optical device 100 illustrated in FIG. 3 may include a first lens unit 150, a liquid lens 110A, and a second lens unit 160. In addition, the optical device 100 may further include a filter 170 and an image sensor 120.

Since the liquid lens 110A and the image sensor 120 shown in FIG. 3 correspond to the liquid lens 110A and the image sensor 120 shown in FIG. 1, respectively, the same reference numerals are used and overlapping descriptions are omitted. . Accordingly, the imaging surface 120S illustrated in FIG. 3 may correspond to the imaging surface of the image sensor 120 illustrated in FIG. 1.

Light incident from the outside to the first lens unit 150 may pass through the liquid lens 110A and enter the second lens unit 160. Unlike the liquid lens 110A, each of the first lens unit 150 and the second lens unit 160 is a solid lens, and may be implemented as glass or plastic, but the embodiment is the first lens unit 150 and the first 2 It is not limited to each specific material of the lens unit 160. Also, at least one of the first lens unit 150 or the second lens unit 160 may be omitted. In addition, in the case of FIG. 3, although the liquid lens 110A is illustrated between the first lens unit 150 and the second lens unit 160, unlike the liquid lens 110A, the first lens unit ( It may be disposed on the left side of the 150) or the right side of the second lens unit 160.

For example, as illustrated in FIG. 3, the first lens unit 150 may include two

lenses

152 and 154, but may also include fewer or more than two lenses. In addition, as illustrated in FIG. 3, the second lens unit 160 may include three

lenses

162, 164, and 166, but may also include fewer or more lenses than three.

The filter 170 may filter light corresponding to a specific wavelength range for light passing through the first lens unit 150, the liquid lens 110A, and the second lens unit 160. The filter 170 may be an infrared (IR) blocking filter or an ultraviolet (UV) blocking filter, but embodiments are not limited thereto. The filter 170 may be disposed between the lens unit 110 and the imaging surface 120S of the image sensor 120. In some cases, the filter 170 may be omitted.

Referring to FIG. 1 again, the image sensor 120 converts light passing through the lens unit 110 into an image, and outputs the converted image to the image processing unit 130. That is, the image sensor 120 has a lens through an R channel receiving a red wavelength range, a G channel receiving a green wavelength range, and a B channel receiving a blue wavelength range. Information of the light passing through the unit 110 may be received.

The image processing unit 130 serves to correct chromatic aberration of the image output from the image sensor 120. Here, the term “chromatic aberration” may mean the degree to which information (or an image) of light received through the correction channel is shifted compared to information (or an image) of light received through the reference channel. The meaning of chromatic aberration will be described later in detail.

The'correction channel' means a channel to be corrected among R channels, G channels and B channels, and the'reference channel' means a channel that is a reference for correcting the correction channels among R channels, G channels and B channels. do. That is, the degree to which the image received by the correction channel is shifted based on the image received by the reference channel corresponds to chromatic aberration. Hereinafter, the correction channels are described as B channels and R channels, and G channels are reference channels, but embodiments are not limited thereto.

The image processing unit 130 is configured to correspond to field information on the image sensor 120 of the driving information of the liquid lens 110A and the correction channel of the information of the light received through at least one correction channel of the R channel, the G channel, and the B channel. Chromatic aberration can be corrected by shifting by a set amount of movement. Here, the driving information of the liquid lens 110A may be information about the degree of hand movement of the user (hereinafter, referred to as “hand shake information”). At this time, the image processing unit 130 shifts the information (for example, an image) of the light received through the correction channel by the amount of movement for each field.

4 shows a planar shape according to the embodiment 120A of the image sensor 120 shown in FIG. 1, the horizontal axis represents the position in the x-axis direction, and the vertical axis represents the y-axis position perpendicular to the x-axis.

The aforementioned “field” indicates a position relative to the optical axis on the imaging surface 120S of the image sensor 120. That is, the field indicates a relative position of the image sensor 120 in the radial direction with respect to the center of the image sensor 120. Referring to FIG. 4, when the z-axis perpendicular to each of the x-axis and the y-axis is an optical axis, a field at the center of the image sensor 120 is 0.0f, and the entire area of the image sensor 120 is radially based on the center. The field of 50% of the point is ±0.5f, and the edge field of the image sensor 120 is ±1.0.

FIG. 5 shows the relationship between the amount of chromatic aberration (hereinafter referred to as'aberration amount') and the field of the image sensor 120 when the interface BO of the liquid lens 110A is not tilted, and FIG. 6 shows the liquid lens ( Shows the relationship between the aberration amount and the field of the image sensor 120 when the interface BO of 110A) is tilted. In each graph, the horizontal axis represents the aberration amount, and the vertical axis represents the physical position of the image sensor 120. That is, a value obtained by normalizing the physical position of the vertical axis corresponds to the field. In addition, in each drawing,'BR' represents the amount of chromatic aberration between the B channel and the R channel, and'BG' represents the amount of chromatic aberration between the B channel and the G channel.

5 and 6, the sign of the field is (+) on the upper side and the sign of the field is (-) on the lower side with respect to the axis representing the aberration amount.

In the general case, that is, when the interface BO is not tilted, the sign (ie, direction) of this field is meaningless because the camera lens is rotationally symmetric about the optical axis. For example, referring to FIG. 5, since the amount of aberration between the positive (+) field and the negative (−) field is symmetrical with respect to the optical axis (ie, the center of the image sensor 120), The sign of the field, ie the direction of the field, is not taken into account.

However, when the interface BO of the liquid lens 110A shown in FIG. 2 is tilted, the sign (ie, direction) of this field must be considered because the camera lens is not rotationally symmetric about the optical axis. For example, referring to FIG. 6, since the amount of aberration between the positive sign (+) field and the negative sign (−) field is asymmetric with respect to the optical axis (ie, the center of the image sensor 120), Consider the sign of the field, ie the direction of the field.

Meanwhile, the amount of movement for each field required to correct chromatic aberration in the image processing unit 130 may be determined in advance for each driving information (for example, hand-shake information) of the liquid lens 110A. This will be described as follows with reference to FIGS. 7 to 9.

7 is a block diagram according to an embodiment 130A of the image processing unit 130 shown in FIG. 1.

The image processing unit 130A may include a control unit 134, a look-up-table (LUT) 136 and an image shifting unit 138.

The LUT 136 maps and stores the correction angles of the liquid lens 110A for a plurality of compensation angles in advance, and stores the first movement amount and R channel for each field on the image sensor 120 of the B channel for each of the correction angles. The second movement amount for each field on the sensor 120 may be mapped and stored in advance.

The correction angle stored in the LUT 136 may be applied to the liquid lens 110A and replaced with a driving voltage that controls the interface of the liquid lens 110A.

8 shows an example of data stored in the LUT 136.

For example, the LUT 136 shown in FIG. 8 maps and stores the compensation angle and the compensation angle, and stores the compensation angle and the first and second movement amounts for each field. The unit of each of the first and second movement amounts illustrated in FIG. 8 is the number of pixels. In the case of FIG. 8, only fields having a positive (+) sign are stored in the LUT 136, but fields having a negative (-) sign may also be stored in the LUT 136.

Referring again to FIG. 7, the control unit 134 looks up the first movement amount for each field of the B channel corresponding to the compensation angle corresponding to the image stabilization information and the second movement amount for each field of the R channel corresponding to the compensation angle ( 136). That is, using the compensation angle corresponding to the hand-shake information as the address of the LUT 136, the control unit 134 determines the first and second movement amounts for each field of the B and R channels stored in the corresponding address of the LUT 136. It can be output to the image shifting unit 138.

The optical device according to the embodiment may further include a camera shake detection unit 50. The camera shake detection unit 50 may convert camera shake information corresponding to driving information of the liquid lens 110A into a compensation angle, and output the converted compensation angle to the controller 134. To this end, the camera shake detection unit 50 may include a gyro sensor 52. The gyro sensor 52 senses the degree of the user's hand-shake, converts the hand-shake information about the sensed hand-shake to a compensation angle, and outputs the converted result to the controller 134. It is not limited to the specific implementation examples of unit 50.

For example, referring to FIG. 8, when the compensation angle is “0”, there is no first and second movement amount for each field. However, when the compensation angle is '0.3 degrees', the compensation angle is 0.42, and the first movement amount for each field in fields 0, 0.2, 0.4, 0.6, 0.8, and 1 is -0.1, 0.5, 0.4, -0.2, -0.4, respectively. And 0.7, and the second movement amount per field in fields 0, 0.2, 0.4, 0.6, 0.8, and 1 is 0, 0.4, 0.6, 0.6, 0.8, and 1.9, respectively.

In this case, in the case of FIG. 8, the spacing between fields is 0.2, but this is only an example, and may be larger or smaller than 0.2. As the distance between fields is smaller, the amount of movement is further subdivided, so that chromatic aberration can be accurately corrected.

The image shifting unit 138 shifts the information (for example, an image) of light received in the B channel for each field by the first movement amount for each field output from the LUT 136, and images the shifted result through the output terminal OUT2 Output to the synthesis unit 140. In addition, the image shifting unit 138 shifts the information (for example, an image) of light received on the R channel by the second movement amount per field output from the LUT 136 for each field, and outputs the shifted result to the output terminal OUT2. Output to the image synthesis unit 140 through. To this end, the image shifting unit 138 receives the information of the light received from the image sensor 120 through the input terminal IN1 in the B channel, and receives the information of the light received from the image sensor 120 through the input terminal IN2 through the R channel. Can receive Alternatively, the image shifting unit 138 receives information of light including both the information of the light received from the image sensor 120 through the input terminal IN1 or IN2 in the B channel and the information of the light received through the R channel, and from the received light information. It is also possible to separate information of light received on the B channel and information of light received on the R channel.

For example, when the compensation angle is 0.3 degrees as described above, the image shifting unit 138 displays information of the light received through the B channel in fields 0, 0.2, 0.4, 0.6, 0.8, and 1, respectively, -0.1, 0.5, and 0.4. , -0.2, -0.4 and 0.7 pixels, and the information of the light received through the R channel is 0, 0.4, 0.6, 0.6, 0.8 and 1.9 pixels in fields 0, 0.2, 0.4, 0.6, 0.8 and 1, respectively. Shift it.

9 is a block diagram according to another embodiment 130B of the image processing unit 130 shown in FIG. 1.

The image processing unit 130B may include a correction angle determination unit 133, a movement amount determination unit 135, and an image shifting unit 138. In FIG. 9, the same reference numerals are used for the same components as in FIG. 7, and overlapping descriptions are omitted.

The correction angle determination unit 133 obtains a correction angle of the preset liquid lens 110A corresponding to the compensation angle output from the image stabilization unit 50 and outputs the correction angle to the movement amount determination unit 135. To this end, referring to FIG. 8, the correction angle determination unit 133 may be implemented in the form of a look-up table 133A using a compensation angle as an address and a correction angle as data stored at each address. To this end, the correction angle determining unit 133 may map a plurality of correction angles for each compensation angle in advance and store them in the form of a lookup table 133A. In this case, the correction angle determination unit 133 may output the correction angle mapped to the compensation angle output from the image stabilization unit 50 to the movement amount determination unit 135.

Also, the correction angle determined by the correction angle determination unit 133 may be output to the liquid lens driving unit (not shown) through the output terminal OUT3. The liquid lens driving unit generates a driving voltage for driving the liquid lens 110A in response to the correction angle output from the correction angle determination unit 133, thereby tilting the angle θ of the interface BO of the liquid lens 110A. Can be adjusted.

The movement amount determination unit 135 corresponds to the correction angle output from the correction angle determination unit 133, and the first movement amount per field on the image sensor 120 of the preset B channel and each field on the image sensor 120 of the R channel. The second movement amount may be determined, and the determined result may be output to the image shifting unit 138.

To this end, referring to FIG. 8, the movement amount determination unit 135 may be implemented in the form of a look-up table 135A that uses correction angles as addresses and first and second movement amounts for each field as data stored in each address. have. To this end, the movement amount determination unit 135 may map a plurality of correction angles and first and second movement amounts for each field in advance and store them in the form of a look-up table 135A. In this case, the movement amount determination unit 135 may output first and second movement amounts for each field mapped to the correction angle output from the correction angle determination unit 133 to the image shifting unit 138.

Referring again to FIG. 1, the image synthesizing unit 140 synthesizes information of light received on the G channel, information of light received on the B channel and shifted, and information on light received on the R channel and shifted, and synthesized results. The final image with corrected chromatic aberration can be output through the output terminal OUT1. To this end, the image synthesizing unit 140 extracts the information of the light received through the G channel from the information of the light output from the image sensor 120, the information of the light received and shifted by the B channel, and the received and shifted by the R channel. The information of the light may be received from the image processing unit 130, and information of the light received through the extracted G channel, information of the light received and shifted through the B channel, and information of the light received and shifted through the R channel may be synthesized.

The final image output through the output terminal OUT1 may be displayed on a display (not shown), but the embodiment is not limited thereto.

The above-described image processing unit 130 and the image synthesis unit 140 may be respectively implemented in separate configurations, or may serve as the above-described image processing unit 130 and the image synthesis unit 140 in one control unit.

Hereinafter, an image processing method for correcting chromatic aberration according to an embodiment will be described with reference to the accompanying drawings.

10 is a flowchart illustrating an image processing method 200 according to an embodiment.

The image processing method 200 illustrated in FIG. 10 may be performed by the optical device 100 illustrated in FIG. 1, or may also be performed by an optical device having a configuration different from the configuration illustrated in FIG. 1. Alternatively, the optical device 100 illustrated in FIG. 1 may perform the image processing method 200 illustrated in FIG. 10 or may perform another image processing method illustrated in FIG. 10.

Hereinafter, for ease of understanding, it will be described that the image processing method 200 illustrated in FIG. 10 is performed by the optical device 100 illustrated in FIG. 1.

First, information on the hand-shake, which is information on the degree of the user's hand-shake, is converted into a compensation angle (step 210). Step 210 may be performed by the image stabilization unit 50 shown in FIGS. 7 and 9.

After the step 210, a correction angle of the preset liquid lens 110A corresponding to the compensation angle is obtained (step 220). The step 220 may be performed by the control unit 134 and the LUT 136 illustrated in FIG. 7, or may be performed by the correction angle determination unit 133 illustrated in FIG. 9.

After the step 220, a predetermined amount of movement for each field corresponding to the correction angle and corresponding to field information on the image sensor 120 of at least one correction channel among R channels, G channels, and B channels is determined (230 and 240) step).

Steps

230 and 240 may be performed by the control unit 134 and the LUT 136 illustrated in FIG. 7, or may be performed by the movement amount determining unit 135 illustrated in FIG. 9.

As described above, among the R, G, and B channels, the correction channels are R and B channels, and the G channel can be a reference channel. In this case, after the step 220, the first movement amount for each field on the image sensor 120 of the preset B channel corresponding to the correction angle may be determined (step 230). After the step 230, a second movement amount for each field on the image sensor 120 of the preset R channel corresponding to the correction angle may be determined (step 240). In the case of FIG. 10, after operation 230 is performed, operation 240 is performed, but the embodiment is not limited thereto. That is, after the 240th step is performed, the 230th step may be performed, and the 230th and 240th steps may be simultaneously performed.

After the 240th step, the information of the light received through the correction channel is shifted by the amount of movement for each field on the image sensor (steps 250 and 260).

Steps

250 and 260 may be performed by the image shifting unit 138 illustrated in FIGS. 7 and 9.

If, among R channels, G channels, and B channels, the correction channels are R channels and B channels, and the reference channel is G channels, after the 240th step, image information (eg, an image) of the light received through the B channel is imaged. The first movement amount may be shifted for each field on the sensor 120 (step 250).

After the 250th step, information (eg, an image) of the light received through the R channel may be shifted by a second amount of movement for each field on the image sensor 120 (Step 260). In the case of FIG. 10, after the 250th step is performed, the 260th step is illustrated, but the embodiment is not limited thereto. That is, after step 260 is performed, step 250 may be performed, and steps 250 and 260 may be performed at the same time. Alternatively, step 250 may be performed after step 230 is performed, and step 260 may be performed after step 240 is performed.

As described above, steps 220 to 260 illustrated in FIG. 10 may be performed by the image processing unit 130.

After the step 260, information of the light received through the G channel (eg, an image), information of the light received through the R channel and shifted (eg, an image), and information of the light received through the B channel and shifted (eg, an image) , Image) may be synthesized, and the synthesized result may be generated as a final image in which chromatic aberration is corrected (step 270 ). Step 270 may be performed by the image synthesis unit 140 illustrated in FIG. 1.

Hereinafter, an optical device and an image processing method according to an embodiment will be described in detail as follows. For convenience of description, the information of the light received through the G channel is referred to as an'G channel image', the information of the light received through the R channel is referred to as an'R channel image', and the information of the light received through the B channel is' B channel image.

The image of which chromatic aberration is suppressed can be obtained by increasing and decreasing the size of the image by the amount of chromatic aberration of the lens for each R channel, G channel and B channel of the image having chromatic aberration. To help understanding, referring to FIG. 11, the image IM1 output from the image sensor 120 has chromatic aberration. Here, the chromatic aberration may mean a difference in size of an image for each wavelength. Based on the G-channel image (GIM), the first size difference obtained by subtracting the size of the G-channel image from the R-channel image size is referred to as'α', and the B-channel image size Let's suppose the second size difference subtracting the size of the image is'β'. At this time, the result of subtracting the first size difference (α) from the size of the image of the R channel (RIM), the result of subtracting the second size difference (β) from the size of the image of the B channel (BIM), and the G channel When synthesizing the image GIM of, an image IM2 with corrected (or suppressed) chromatic aberration may be obtained.

Among the differences between the size of the image of the R channel having a wavelength band of 656.2 nm, the size of the image of the B channel having a wavelength band of 436.1 nm, and the size of the image of the G channel having a wavelength band of 587.6 nm, the largest value is the chromatic aberration. It corresponds. For example, a chromatic aberration of 10 μm means that the largest size difference among image size differences of the R channel, the G channel, and the B channel is 10 μm. As described above, in the case of the embodiment, by increasing or decreasing the size of the image of the R channel and the B channel, so that the size of the image of the R channel and the B channel is different from that of the G channel, based on the image of the G channel, Chromatic aberration can be corrected (or suppressed).

However, the chromatic aberration of the image generated from the optical signal passing through the lens unit 110 in the state where the interface BO of the liquid lens 110A is tilted has a different size between the images of the R channel, the G channel, and the B channel. Instead of appearing, the images of R channel, G channel and B channel appear in shifted form. For example, referring to FIG. 4, when the G channel is the reference channel among the R channels, the G channels, and the B channels, the R channel image R1 is shifted based on the G channel image GIM. Chromatic aberration appears. For this reason, referring to FIG. 6, as the amount of chromatic aberration in a field with a positive sign (+) increases, the amount of chromatic aberration in a field with a negative sign (-) decreases. Accordingly, the optical device 100 and the image processing method 200 according to the above-described embodiment may correct chromatic aberration by shifting the image of each of the correction channel, for example, the B channel and the R channel according to image stabilization information.

12 is a diagram for describing an image in which chromatic aberration is corrected by the optical device 100 and the image processing method 200 according to the embodiment. Here, 0.9f to -0.9f denote a field, and 0 degrees and 0.6 degrees denote compensation angles. The compensation angle of '0 degree' is a steady state without shaking, and as shown in FIG. 2, there is no tilting at the interface BO. The compensation angle of '0.6 degrees' may mean that the shaking motion is maximized, and as illustrated in FIG. 2, the inclined angle θ of the interface BO is 0.6 degrees.

Referring to FIG. 12, as shown in the embodiment, the image of the chromatic aberration corrected is obtained by shifting and synthesizing the image of the R channel and the image of the B channel by a predetermined amount of movement for each field according to the compensation angle obtained according to the image stabilization information. Can be seen.

Meanwhile, the optical device according to the above-described embodiment may include a device capable of processing or analyzing an optical signal. Examples of the optical device may include a camera/video device, a telescope device, a microscope device, an interferometer device, a photometer device, a polarimeter device, a spectrometer device, a reflectometer device, an autocollimator device, and a lens meter device.

Further, the optical device may be implemented as a portable device such as a smart phone, a notebook computer, and a tablet computer. Such an optical device may include a display unit for outputting an image (not shown), a battery for supplying power to each unit (not shown), and a main body housing for mounting each unit and the battery. The optical device may further include a communication module capable of communicating with other devices and a memory unit capable of storing data. The communication module and the memory unit may also be mounted in the body housing.

Meanwhile, a lens unit including a liquid lens; An image sensor that receives information of light passing through the lens unit; And an image processing unit for correcting the information of light, a recording medium recording a program for executing an image processing method performed in an optical device, the function of converting information on the degree of hand shake of a user into a compensation angle and corresponding to the compensation angle At least, the function to obtain a predetermined correction angle of the liquid lens, and at least one of the R channel corresponding to the correction angle and the R channel receiving the red wavelength range, the G channel receiving the green wavelength range, and the B channel receiving the blue wavelength range. Records a program that implements a function of determining a preset amount of movement corresponding to field information on an image sensor of one correction channel and a function of shifting information of light received by a correction channel by a determined amount of movement for each field on an image sensor, and the computer The recording medium can be read.

In addition, the function of determining the amount of movement implemented by the program recorded on the computer-readable recording medium corresponds to the correction angle, and the function of determining the first amount of movement for each field on the preset B channel image sensor and the correction angle And it may include a function for determining the second movement amount for each field on the preset R channel image sensor.

In addition, the shifting function implemented by a program recorded on a computer-readable recording medium is a function of shifting the information of light received on the B channel by a first movement amount for each field on the image sensor and the information of light received on the R channel. A function of shifting by the second movement amount for each field on the sensor may be included.

In addition, the computer-readable recording medium further includes a function of synthesizing the information of the light received on the G channel, the information of the light received on the R channel and shifted, and the information of the light received on the B channel and shifted to generate a corrected final image. You can record the program you implement.

The computer-readable recording medium includes any kind of storage device that stores data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. The computer-readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the image processing method can be easily inferred by programmers in the technical field to which the present invention pertains.

The embodiments have been mainly described above, but this is merely an example, and the present invention is not limited thereto, and those skilled in the art to which the present invention pertains are not exemplified above in the range that does not depart from the essential characteristics of the present embodiment. It will be appreciated that various modifications and applications are possible. For example, each component specifically shown in the embodiment can be implemented by modification. And differences related to these modifications and applications should be construed as being included in the scope of the invention defined in the appended claims.

Modes for carrying out the invention have been fully described in the above-mentioned "Best Mode for Invention".

The optical device and the image processing method including the liquid lens according to the embodiment and having chromatic aberration correction function include a camera/video device, a telescope device, a microscope device, an interferometer device, a photometer device, a polarimeter device, a spectrometer device, a reflectometer device, and an autocollimator It can be used in devices, lens meter devices, smartphones, notebook computers, tablet computers, and the like.

Claims

A lens unit including a liquid lens;

An image sensor receiving information of light passing through the lens unit through an R channel receiving a red wavelength range, a G channel receiving a green wavelength range, and a B channel receiving a blue wavelength range; And

An image for shifting information of light received through at least one correction channel among the R channel, the G channel, and the B channel by a preset movement amount corresponding to the driving information of the liquid lens and field information on the image sensor of the correction channel. Optical device comprising a processing unit.
The optical device of claim 1, wherein the correction channel among the R channel, the G channel, and the B channel is a B channel and an R channel, and the G channel is a reference channel.
The method of claim 2, wherein the image processing unit

The correction angle of the liquid lens is mapped and stored for each of the compensation angles, and the first movement amount for each field on the image sensor of the B channel and the second movement amount for each field on the image sensor of the R channel for each of the correction angles Lookup table to store the mapping;

The first movement amount for each field of the B channel corresponding to the compensation angle corresponding to the driving information of the liquid lens, and the second movement amount for each field of the R channel corresponding to the compensation angle are output from the lookup table. Control unit; And

The information of the light received on the B channel is shifted for each field by the first movement amount for each field output from the lookup table, and the information of the light received on the R channel by the second movement amount for each field output from the lookup table is fielded. An optical device including an image shifting unit that is not very shifting.
The method of claim 2, wherein the image processing unit

A correction angle determination unit corresponding to a compensation angle corresponding to the driving information of the liquid lens and obtaining a preset correction angle of the liquid lens;

A movement amount determination unit corresponding to the correction angle and determining a first movement amount for each field on the image sensor of the B channel and a second movement amount for each field on the image sensor of the R channel; And

And an image shifting unit for shifting the information of the light received on the B channel by the first movement amount for each field, and shifting the information of the light received on the R channel by the second movement amount for each field.
The method of claim 3 or 4,

And an image synthesizing unit for synthesizing the information of the light received on the G channel, the information of the light received on the B channel and shifted, and the information of the light received on the R channel and shifted to generate a corrected final image.
The optical device according to claim 3 or 4, further comprising a camera shake detection unit that converts a user's camera shake information corresponding to the driving information of the liquid lens into a compensation angle.
A lens unit including a liquid lens; An image sensor that receives information of light passing through the lens unit; And an image processing unit for correcting the information of the light,

Converting driving information of the liquid lens into a compensation angle;

Obtaining a correction angle of the preset liquid lens corresponding to the compensation angle;

Corresponding to the correction angle, the field information on the image sensor of at least one correction channel among the R channel receiving the red wavelength range, the G channel receiving the green wavelength range, and the B channel receiving the blue wavelength range. Determining a corresponding preset movement amount; And

And shifting the information of the light received through the correction channel by the determined amount of movement for each field on the image sensor.
8. The method of claim 7, wherein the correction channel among the R channel, the G channel and the B channel is an R channel and a B channel, and the G channel is a reference channel,

Determining the amount of movement is

Determining a first movement amount for each field on the image sensor of the preset B channel corresponding to the correction angle; And

And determining a second movement amount for each field on the image sensor of the R channel that corresponds to the correction angle.
The method of claim 8, wherein the shifting step

Shifting the information of the light received through the B channel by the first movement amount for each field on the image sensor; And

And shifting the information of the light received through the R channel by the second movement amount for each field on the image sensor.
8. The method of claim 7, wherein the correction channel among the R channel, the G channel and the B channel is an R channel and a B channel, and the G channel is a reference channel,

And generating a corrected final image by synthesizing the information of the light received on the G channel, the information of the light received on the R channel and shifted, and the information of the light received on the B channel and shifted.