KR101327035B1 - Camera module and image processing method - Google Patents

Abstract

An image processing method of a camera module according to the embodiment of the present invention comprises a step of receiving an image; a step of analyzing the image by obtaining information about the brightness of the image; a step of determining a gain value for correcting the image by using the analysis result; and a step of correcting the image using the gain value determined. [Reference numerals] (110) Lens;(120) Input image processing unit;(130) Image sensor;(140) Output image processing unit;(150) Display unit;(160) Storage unit;(170) User input unit;(180) Control unit

Description

Camera module and its image processing method {CAMERA MODULE AND IMAGE PROCESSING METHOD}

The embodiment relates to a camera module, and more particularly, to a camera module and an image processing method thereof for preventing flare by adding an image processing block to an input terminal of an image sensor.

Closed Circuit Televisions (CCTVs), which are installed outside general homes, as well as indoors of department stores, banks, exhibition halls and factories, are widely used to prevent theft and to determine the operation status of the machine, process flow or situation.

Surveillance cameras have been used for the purpose of remotely monitoring all the situation in the place installed in a particular place, for this purpose includes a display unit for receiving a signal transmitted from the image transmitter and the image transmitter to provide to the display device. .

On the other hand, in general, digital cameras are similar in that they use the optics and mechanisms of ordinary cameras.However, digital cameras accept images with an image sensor called CCD (Charge Coupled Device) instead of film, and convert the signal into digital data. The difference is that the file is stored in memory as a file.

Such a digital camera can immediately check the captured image on the display screen, and can process various data such as editing and output through a computer, and if necessary, can be immediately printed on the printer without complicated film development and printing process. Utilization is expanding.

1 is a view for explaining the operation of the image sensor according to the prior art.

Referring to FIG. 1, generally, there is an acceptable input limit in an image sensor, and when bright light above the input limit is incident, the brightness of the incident light is reduced according to the input limit.

However, in the case where bright light above the input limit acceptable to the image sensor is incident as described above, severe flare occurs in most camera modules.

The flare phenomenon means reflection and scattering of light generated in the lens and the image sensor.

Such flare phenomenon becomes more severe as the lens is manufactured incorrectly, the brightness of the light source is bright, or the average brightness of the image is low.

Accordingly, there is a need for a technique of reducing the flare phenomenon generated as described above to prevent deterioration of image quality.

The embodiment provides a camera module and an image processing method thereof by adding a histogram calculating block and a gain adjusting block to an input terminal of an image sensor, thereby preventing flare caused when very bright light is incident on the camera module. .

An image processing method of a camera module according to an embodiment includes receiving an image; Analyzing the characteristics of the image by obtaining brightness information of the received image; Determining a gain value for compensating the image using the analyzed characteristic of the image; And modulating the received image by applying the determined gain value.

The acquiring the brightness information may include calculating a histogram for each frame in the received image, and analyzing the frequency number for each brightness level of the pixels included in the frame using the calculated histogram. Steps.

The determining of the gain value may include determining a frequency count for each of the analyzed brightness levels, and using the checked frequency number, a frequency number corresponding to a brightness level of each pixel in the frame. Determining a gain value to be applied to each pixel.

The determining of the gain value may include: expressing a histogram graph of the frequency number for each brightness level, connecting the frequency number corresponding to each brightness level included in the histogram graph, and the frequency. Determining a graph representing the connection of numbers as a gamma curve for the compensation of the image.

The determined gain value increases or decreases in inverse proportion to the frequency number of the brightness levels of the respective pixels.

The gamma curve is a curve for increasing a gain value of a pixel having a high frequency brightness level in the frame and reducing a gain value of a pixel having a low frequency brightness level.

Also, the image is an image of a subject photographed through a lens, and the receiving, analyzing, determining, and modulating step is performed in an image sensor.

The method further includes converting the modulated image into an electrical signal.

On the other hand, the camera module according to the embodiment includes a lens; An input image processor which first processes an image input through the lens; An image sensor converting the first processed image into an electrical signal; An output image processor for second-processing the image converted into the electrical signal; And a controller configured to control the image input through the lens to be primarily processed through the input image processor, wherein the input image processor acquires brightness information of the input image and analyzes the characteristics thereof. The received image is first processed by applying a gain value adjusted according to the characteristic.

The input image processor may further include a histogram analyzer configured to analyze the histogram of the input image to determine characteristics of the image, and a gain value to be applied to the compensation of the image according to the characteristics of the image determined by the histogram analyzer. And a gain setting unit configured to set the control unit, and an input image adjusting unit modulating the received image by applying a gain value set through the gain setting unit.

The histogram analyzer may analyze the frequency number of each brightness level of the pixels included in each frame of the image by using the histogram of the image.

In addition, the gain setting unit determines the frequency number for each of the analyzed brightness levels and determines a frequency number corresponding to the brightness level of each pixel in the frame as a gain value to be applied to each pixel.

In addition, the gain setting unit, in the histogram graph expressed in the frequency number for each brightness level for the image, determines the graph represented by connecting the frequency number corresponding to each brightness level as a gamma curve for the compensation of the image. .

The determined gain value increases or decreases in inverse proportion to the frequency number of the brightness levels of the respective pixels.

The gamma curve is a curve for increasing a gain value of a pixel having a high frequency brightness level in the frame and reducing a gain value of a pixel having a low frequency brightness level.

The input image processor may acquire brightness information for each frame of the input image, analyze a characteristic, and first process the image by applying a gain value that is differently adjusted for each frame according to the analyzed characteristic. .

In addition, the input image processor is configured at an input terminal in the image sensor.

According to the embodiment, by analyzing the image input through the lens, by performing the histogram and gain processing before the image is input to the image sensor, it is possible to prevent the flare occurring in the general camera module, thereby Providing images can improve user satisfaction.

1 is a view for explaining the operation of the image sensor according to the prior art.
2 is a diagram illustrating a camera module according to an exemplary embodiment.
3 is a diagram illustrating a detailed configuration of the input image processor 120 shown in FIG. 2.
4 is a diagram illustrating a histogram graph detected according to an exemplary embodiment.
5 is a view illustrating an operation of the gain setting unit 122 according to an embodiment.
6 is a diagram illustrating a gamma curve according to an embodiment.
7 is a diagram illustrating a luminance compensation image according to an exemplary embodiment.

The following merely illustrates the principles of the invention. Thus, those skilled in the art will be able to devise various apparatuses which, although not explicitly described or shown herein, embody the principles of the invention and are included in the concept and scope of the invention. Furthermore, all of the conditional terms and embodiments listed herein are, in principle, only intended for the purpose of enabling understanding of the concepts of the present invention, and are not to be construed as limited to such specifically recited embodiments and conditions do.

It is also to be understood that the detailed description, as well as the principles, aspects and embodiments of the invention, as well as specific embodiments thereof, are intended to cover structural and functional equivalents thereof. It is also to be understood that such equivalents include all elements contemplated to perform the same function irrespective of the currently known equivalents as well as the equivalents to be developed in the future, i.e., the structure.

Thus, for example, the block diagrams herein should be understood to represent a conceptual view of example circuitry embodying the principles of the invention. Similarly, all flowcharts, state transition diagrams, pseudo code, and the like are representative of various processes that may be substantially represented on a computer-readable medium and executed by a computer or processor, whether or not the computer or processor is explicitly shown .

The functions of the various elements shown in the figures, including the functional blocks depicted in the processor or similar concept, may be provided by use of dedicated hardware as well as hardware capable of executing software in connection with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

Also, the explicit use of terms such as processor, control, or similar concepts should not be interpreted exclusively as hardware capable of running software, and may be used without limitation as a digital signal processor (DSP) (ROM), random access memory (RAM), and non-volatile memory. Other hardware may also be included.

In an embodiment, the image input through the lens is analyzed and histogram and gain processing is performed before the image is input to the image sensor, thereby preventing a flare phenomenon occurring in a general camera module. Disclosed is a camera module and an image processing method thereof that can improve user satisfaction by providing a.

2 is a diagram illustrating a camera module according to an exemplary embodiment.

Referring to FIG. 2, the camera module 100 includes a lens 110, an input image processor 120, an image sensor 130, an output image processor 140, a display unit 150, a storage unit 160, and a user input unit. 170 and the controller 180.

The lens 110 is an optical system OPS and may further include a filter. The lens 110 optically processes light of an image to be photographed.

The lens 110 allows the optical image of the subject to be formed on the image sensor 130.

The lens 110 may include a zoom lens (not shown) and a focus lens (not shown) that is movable in the optical axis direction to focus the optical image formed on the image sensor 130.

The lens 110 acquires an image of a subject to be photographed by a user.

Specifically, the lens 110 includes a first lens group composed of a concave lens composed of diffractive optical elements on at least one surface thereof, and a second lens group composed of convex lenses composed of diffractive optical elements on at least one surface thereof. Can be.

The first lens group is a concave lens having a negative power to have a wide viewing angle and a sufficient back focal length (BFL), one side of which is an aspherical surface and at least one side of the diffractive optical element Since the dispersion of the plane on which the diffractive optical element is formed has a negative sign, it is possible to easily correct the chromatic aberration on the axis and to bear a part of the power, thereby bringing the shape of the lens to some extent.

In addition, the second lens group of the convex lens shape has positive power, at least one surface is an aspherical surface, and at least one surface is composed of diffractive optical elements to be incident on the first lens group of the concave lens shape. Converged video information.

The input image processor 120 performs line processing on the image acquired through the lens 110.

The image sensor 130 converts the preprocessed image through the input image processor 120 into electrical data and outputs the converted electrical data.

2 illustrates an embodiment in which the input image processor 120 and the image sensor 130 are separately provided, but the scope of the embodiment is not limited thereto.

That is, the input image processor 120 may be included in the image sensor 130 to perform a corresponding function.

Hereinafter, the configurations of the input image processor 120 and the image sensor 130 will be described separately.

The input image processor 120 analyzes an image acquired through the lens 110 and performs line processing on the image according to the analysis result of the image.

In particular, the input image processor 120 obtains brightness information of the obtained image.

The input image processor 120 checks the gray level distribution of each frame of the image using the obtained brightness information.

In addition, when the gray distribution is confirmed, the input image processor 120 sets a gain value for compensating the image according to the checked gray distribution.

In addition, when the gain value is set, the input image processor 120 processes the image for each frame by using the set gain value.

A detailed operation of the input image processor 120 will be described in more detail below.

The image sensor 130 receives the preprocessed image through the input image processor 120, converts the received image into electrical data, and outputs the converted image.

The image sensor 130 may be configured of a complementary metal-oxide semiconductor (CMOS) or a charge coupled device (CCD).

The image sensor 130 has a form in which a plurality of photo detectors are integrated as each pixel, and converts image information of a subject into electrical data and outputs the converted electrical data.

The image sensor 130 accumulates the input light amount and outputs the image captured by the lens 110 according to the accumulated light amount in accordance with the vertical synchronization signal.

Image acquisition is performed by the image sensor 130 that converts light reflected from a subject into an electrical signal.

In order to obtain a color image using the image sensor 130, a color filter is required, and a filter (not shown) called a CFA (Color Filter Array) is mostly adopted. CFA passes only light representing one color per pixel, has a regularly arranged structure, and has various forms depending on the arrangement structure.

The output image processor 140 may generally be referred to as an image signal processor (ISP), and processes an image output through the image sensor 130 in units of frames.

In this case, the output image processor 140 may include a lens shading compensator (not shown).

The lens shading compensator is a block for compensating for a lens shading phenomenon that appears differently in the amount of light of the center and the edge region of the image. The lens shading compensator receives the lens shading setting value from the controller 180 to be described later, and adjusts the color of the center and the edge region of the image. To compensate.

In addition, the lens shading compensator may receive a shading variable set differently according to the type of illumination and process the lens shading of the image according to the received parameter. Accordingly, the lens shading compensator may perform the lens shading process by applying a different degree of shading according to the type of illumination.

Meanwhile, the lens shading compensator receives a shading variable differently set according to an automatic exposure weight applied to a specific area of the image to prevent saturation occurring in the image, and adjusts the lens shading of the image according to the received variable. It can also be processed.

More specifically, the lens shading compensator compensates for the brightness change occurring in the edge region of the image signal as the auto exposure weight is applied to the center region of the image signal.

That is, when saturation of the video signal occurs by illumination, the intensity of light decreases from the center to the outer in the form of concentric circles, so that the lens shading compensator amplifies the edge signal of the video signal to compensate for the brightness of the center contrast. do.

The display unit 150 displays an image captured by the control of the controller 180 to be described later, and displays a setting screen necessary for taking a picture or a screen for selecting an operation of a user.

In addition, according to an exemplary embodiment, the display unit 150 displays a preview screen when a preview key is input, and displays a pop-up screen or a preset animation or image that is popped up when a shooting key is input.

The storage unit 160 stores data necessary for the camera module 100 to operate.

In addition, according to an embodiment, the storage unit 160 stores a pop-up screen, an animation, or an image to be displayed through the display unit 150 when a photographing key is input.

The storage unit 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory) RAM, ROM (EEPROM, etc.), and the like.

The user input unit 170 receives a user's input and transmits it to the controller 180.

When the display unit 150 is implemented as a touch screen, the display unit 150 may perform the image display function and operate as the input unit 170.

According to an embodiment, the input unit 170 may further include a photographing key for performing photographing and a preview key for displaying a preview screen.

The controller 180 controls each component of the camera module.

In an embodiment, when an image is acquired through the lens 110, the controller 180 controls the input image processor 120 to be preprocessed before the acquired image is input to the image sensor 130. .

In addition, the controller 180 allows the input image processor 120 to acquire the brightness information of the obtained image, and performs a gain compensation for removing a flare phenomenon based on the obtained brightness information.

3 is a diagram illustrating a detailed configuration of the input image processor 120 shown in FIG. 2.

Hereinafter, an operation of the input image processor 120 and a process of removing flare phenomenon by the operation will be described with reference to FIG. 3.

Referring to FIG. 3, the input image processor 120 includes a histogram analyzer 121, a gain setting unit 122, and an input image adjusting unit 124.

The histogram analyzer 121 detects a histogram of each frame in the image input through the lens 110.

At this time, the histogram analyzes the gray level for each pixel, and includes the frequency number, peak number, peak distance, and peak width of each gray level.

In general, a histogram shows a distribution of contrast values for pixels in an image. When a light pixel and a dark pixel are distributed, the histogram expresses a range and a value, and this graph is called a histogram graph.

4 is a diagram illustrating a histogram graph detected according to an exemplary embodiment.

As shown in FIG. 4, the range of the contrast value (gradation) in the 256 gray level image has a value ranging from 0 to 255, and the frequency of each contrast value (level) is represented by the height of the graph.

The histogram contains various information of the image, which is used as a condition for performing image processing.

The gain setting unit 122 sets a gain value applied for each frame by using the histogram for each frame of the image analyzed by the histogram analyzer 121.

In this case, the set gain value may be represented by a gamma curve indicating a relationship between an input gray level and an output gray level of each pixel.

Here, the gain setting unit 122 checks the frequency number of each gray level in the characteristics of the histogram analyzed by the histogram analyzer 121, and uses the checked frequency number to obtain a gain value to be applied to the image, again. In other words, determine the gamma curve.

In this case, the determined gain value is closely related to the frequency included in the histogram.

In other words, the frequency of each grayscale included in the histogram graph may be determined as a gain value to be applied to the grayscale.

5 is a view illustrating an operation of the gain setting unit 122 according to an embodiment.

Referring to FIG. 5, the gain setting unit 122 receives a histogram graph detected through the histogram analyzer 121.

Thereafter, the gain setting unit 122 connects the frequency numbers of the gray levels with each other in the received histogram graph.

In this case, the gradation included in the histogram graph does not appear as the height of 256 gradations corresponding to the gradation level of 0 to 255, grouping a gradation level of a predetermined range in a predetermined step, and detecting the frequency number in the grouping step.

For example, when the gradation level is divided into 10 levels, the first level includes gradation levels in the range of 0 to 25, the second level includes gradation levels in the range of 26 to 50, and in this manner, the last 10 levels The gray level may range from 226 to 256.

The histogram analyzer 121 detects a frequency number of the gradation level included in each step, and detects a histogram graph in the form of a bar graph as shown in FIG. 4 by using the detected frequency number. do.

In this case, the histogram graph includes a bar graph indicating the frequency of each gray level corresponding to the gray level divided into the ten levels, and the bar graph has a number corresponding to the step. For example, when the gradation level is divided into ten levels, the histogram graph includes ten bar graphs.

The gain setting unit 122 connects the frequency number (bar graph) corresponding to each step in the histogram graph.

The gain setting unit 122 checks a graph formed by connecting the frequency numbers to each other, and determines a gamma curve to be applied to each frame in the image based on the checked graph.

6 is a diagram illustrating a gamma curve according to an embodiment.

In other words, a graph generated by concatenating the frequency of each gray level in the histogram graph is determined as a gamma curve to be applied to each frame in the image.

Accordingly, the gain value is determined as a value corresponding to the frequency of the corresponding gradation level.

For example, when the frequency of 250 gray levels is 100, the gain value of 250 gray levels may be set to 100.

In conclusion, the gain setting unit 122 sets the gain value using the histogram on the same principle as that of the black level expansion (BLE) and the white level expansion (WLE).

In other words, the gain setting unit 122 finds the characteristics of such a frame based on the histogram graph in the case of a frame in which the entire area is dark and the area is very bright, and thus the gain value of the dark area is increased and the bright area is increased. Lower the gain value of to eliminate the play phenomenon.

That is, since the bright area in the frame is part of the area, the frequency frequency according to the gradation level of the bright area will be low, and thus the gain value applied to the bright area is also low.

On the contrary, since the dark area in the frame corresponds to most of the entire area, the frequency of the dark area will be high, and thus the gain value applied to the dark area will also be high.

The input image adjusting unit 124 compensates the luminance of the input image by applying a gain value set through the gain setting unit 122, that is, a gamma curve.

7 is a diagram illustrating a luminance compensation image according to an exemplary embodiment.

Referring to FIG. 7, in the prior art, an image obtained through a lens is flared around a light source.

However, through the histogram and gain block processing according to the embodiment it is possible to efficiently remove the flare phenomenon generated around the light source.

8 is a view for explaining an image processing method according to an embodiment step by step.

Referring to FIG. 8, in the image processing method, an image is first acquired through the lens 110, and the obtained image is input to the image sensor 130 (step 101).

In operation 102, the input image processor 120 configured at the input terminal of the image sensor 130 obtains brightness information by using the input image.

In other words, the input image processor 120 analyzes the histogram based on the input image (step 102).

Thereafter, the input image processor 120 determines a gain value applied to the corresponding frame, that is, a gamma curve, by using the analyzed histogram (step 103).

That is, the input image processor 120 connects the frequency numbers of the gray levels in the histogram graph to each other, and determines the graph formed by the frequency connection as the gamma curve.

In operation 104, the input image processor 120 processes the input image by applying the set gain (gamma curve).

When the image processing is completed, the processed image is input to the image sensor 130.

According to the embodiment, by analyzing the image input through the lens, by performing the histogram and gain processing before the image is input to the image sensor, it is possible to prevent the flare occurring in the general camera module, thereby Providing images can improve user satisfaction.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: Camera module
110: lens
120: input image processing unit
121: histogram analysis unit
122: gain setting section
123: input image adjusting unit
130: image sensor
140: output image processing unit
150:
160:
170: user input unit
180:

Claims (17)

Analyzing the characteristics of the image by acquiring brightness information of an image input through a lens;
Determining a gain value for compensating the image using the analyzed characteristic of the image;
First processing the image by applying the determined gain value;
Converting the first processed image into an electrical signal; And
Secondary processing the image for display of the image converted into the electrical signal. The method of claim 1,
The brightness information,
Calculate a histogram for each frame in the image,
The image processing method obtained by analyzing the frequency number of each brightness level for the pixels included in the frame using the calculated histogram. 3. The method of claim 2,
Determining the gain value,
Checking the frequency count for each of the analyzed brightness levels;
And determining the frequency number corresponding to the brightness level of each pixel in the frame as the gain value to be applied to each pixel by using the identified frequency number. 3. The method of claim 2,
Determining the gain value,
Expressing a histogram graph of the frequency number of each brightness level;
Linking frequency numbers corresponding to respective brightness levels included in the histogram graph to each other;
And determining a graph representing the connection of the frequency numbers as a gamma curve for compensating the image. The method of claim 3, wherein
The determined gain value is
And an inverse increase and decrease in inverse proportion to the frequency of the brightness level of each pixel. 5. The method of claim 4,
The gamma curve is,
And a curve for increasing a gain value of a pixel having a high frequency brightness level in the frame and reducing a gain value of a pixel having a low frequency brightness level. The method of claim 1,
Characterization of the image, gain value determination and primary processing
Image processing method performed within the image sensor. delete lens;
An input image processor which first processes an image input through the lens;
An image sensor converting the first processed image into an electrical signal;
An output image processor for second-processing the image converted into the electrical signal; And
And a controller configured to control the image input through the lens to be primarily processed through the input image processor.
The input image processor,
Acquiring brightness information of the input image to analyze a characteristic, and applying the gain value adjusted according to the analyzed characteristic to first process the input image. The method of claim 9,
The input image processor,
A histogram analyzer for analyzing a histogram of the input image to determine characteristics of the image;
A gain setting unit for setting a gain value to be applied to the compensation of the image according to the characteristics of the image identified through the histogram analyzer;
And an input image adjusting unit configured to modulate the input image by applying a gain value set through the gain setting unit. The method of claim 10,
The histogram analysis unit,
A camera module for analyzing the frequency number of each brightness level for the pixels included in each frame of the image using the histogram of the image. 12. The method of claim 11,
The gain setting unit,
And checking the analyzed frequency number for each brightness level, and determining a frequency number corresponding to the brightness level of each pixel in the frame as a gain value to be applied to each pixel. 12. The method of claim 11,
The gain setting unit,
And a histogram graph represented by the frequency level for each brightness level of the image, and determining a graph represented by connecting the frequency numbers corresponding to each brightness level as a gamma curve for compensating the image. 13. The method of claim 12,
The determined gain value is
And a camera module that increases or decreases in inverse proportion to a frequency of brightness levels of each pixel. The method of claim 13,
The gamma curve is,
And a curve for increasing a gain value of a pixel having a high frequency of brightness level and reducing a gain value of a pixel having a low frequency of brightness level in the frame. The method of claim 9,
The input image processor,
Acquiring the brightness information for each frame of the input image to analyze the characteristics, and the camera module to process the image first by applying a gain value adjusted differently for each frame according to the analyzed characteristics. The method of claim 9,
The input image processor,
A camera module configured at an input in the image sensor.

Images