KR100602705B1 - Apparatus for processing video signal in handset with camera function and method thereof - Google Patents

Abstract

The present invention not only improves the data processing speed by setting the preview resolution lower than the imaging resolution and sets the high resolution at the time of imaging, but also controls the camera sensor when switching from the low resolution mode to the high resolution mode. The present invention relates to an image processing apparatus and a method of a terminal having a camera function for converting a reference value for a through a linear equation so that there is no delay when switching to a shooting operation while performing a preview function.

The present invention provides a camera sensor for acquiring analog image data having different resolutions for each mode; And switching the modes, converting the analog image data into digital image data, setting a plurality of reference values with reference to analog image data having a reference resolution, and converting the plurality of reference values into a plurality of linear equations, respectively, during mode switching. And a plurality of image processing units to obtain a plurality of adjustment reference values, wherein the image processing unit controls the amplification gain, white balance, and exposure of the camera sensor with reference to the plurality of reference values or the plurality of adjustment reference values.

Resolution, handset, white, balance, exposure, camera, image, shot

Description

An image processing apparatus and method in a terminal having a camera function {APPARATUS FOR PROCESSING VIDEO SIGNAL IN HANDSET WITH CAMERA FUNCTION AND METHOD THEREOF}             

1 is a block diagram showing an image processing apparatus in a terminal having a conventional camera function;

2 is a block diagram illustrating an image processing apparatus in a terminal having a camera function according to an embodiment of the present invention;

3 is a flowchart illustrating an image processing method in a terminal having a camera function according to an embodiment of the present invention.

Explanation of symbols on the main parts of the drawings

210: camera sensor 220: image processing unit

230: display unit 240: storage unit

The present invention relates to an image processing apparatus and a method for a terminal having a camera function, and more particularly, to a portable terminal equipped with a high pixel camera module, and an image in a terminal having a camera function capable of implementing a fast preview function. A processing apparatus and method.

Today's portable terminals have added various functions with improved communication speeds and increased internal memory and processing speeds. Among them, the enlargement of the display device and the improvement of the display level make it possible to view not only pictures but also videos.

Accordingly, the camera function among the functions of the portable terminal has been expanded to the application area of the conventional digital camera as the resolution thereof is improved.

In other words, the resolution of the camera sensor mounted in the portable terminal exceeds the VGA (Video Graphic Array) level and is progressing to the SXGA level. Here, VGA and SXGA is a standard for displaying computer-processed data on a terminal screen, and VGA is a standard and device for realizing a video screen with a resolution of 640 × 480 pixels (300,000 pixels or more). SXGA, a more advanced standard, is a standard and device that realizes a resolution of 1,280 × 960 pixels (more than 1.3 million pixels), which is much higher than conventional TVs.

1 is a block diagram illustrating an image processing apparatus in a terminal having a conventional camera function.

The lens 110 determines a physical magnification in capturing an object, and serves to block contamination from the outside. In addition, the camera sensor 120 converts an external image input through the lens 110 into analog image data. Here, the camera sensor 120 refers to a CMOS sensor or a CCD sensor that converts an input image into an electronic signal.
Meanwhile, the digital signal processor (DSP) 130 converts analog image data input from the camera sensor 120 into digital image data. At this time, the DSP 130 calculates reference values for AEC (Automatic Exposure Control), AGC (Automatic Gain Control) and AWB (Automatic White Balance) according to the external environment change.
In addition, the main processor 140 receives digital image data from the DSP 130, adjusts the camera sensor 120 according to reference values for AEC, AGC, and AWB calculated by the DSP 130, and digital image. The data is processed to generate image data for display. Meanwhile, the display unit 150 receives the display image data from the main processor 140 and displays the image for recognition by the user.

The operation of the image processing apparatus in the terminal having the conventional camera function described above is as follows.

First, the camera sensor 120 captures an external object and acquires an image frame, which is electrical analog image data. The frame includes a plurality of pixels, and each pixel includes 18 bits of RGB information. can do. For example, in the case of a frame having 160 × 128 pixels, 18 × 160 × 128 bits of information may be included. Thereafter, the image frame is processed by the DSP 130 into digital image data. At this time, each information bit is processed according to an internal clock.

Here, the frequency of the internal clock generation element employed in the general portable terminal is 24MHz ~ 27MHz, the higher the frequency, the digital image data processing speed increases. At this time, in the case of a VGA class sensor in which a frame having 640 × 480 pixels is input under a frequency environment of 24 MHz to 27 MHz, the frame processing speed in the DSP 130 is about 30 frames / second, and 1280 × 960. In the case where an SXGA-class sensor into which a frame having pixels is input is employed, the frame processing rate in the DSP 130 is about 7.5 frames / second. For reference, as the number of frames processed per second increases, the motion of the acquired video becomes more natural, and the number of frames inserted per second in a cartoon movie theater is about 30 frames. It is known as about 20 frames.

That is, according to the conventional technology described above, when performing a video operation such as preview by driving the camera of the portable terminal, when performing the preview function of the VGA class, the frame processing speed is about 30 frames / sec. Provides a high quality preview video, but when the SXGA-class preview function is performed, the frame processing speed is about 7.5 frames per second, so there is a problem that the user is too slow preview video.

In particular, when performing a video operation such as preview by driving the camera of the portable terminal at night, since the number of frames to be processed per second drops by half, there is a problem that is quite inconvenient in performing the preview function.

The present invention has been made to solve the above problems, in the terminal equipped with a camera module, the preview resolution is set differently from the imaging resolution, and by setting a high resolution only at the time of imaging, thereby increasing the data processing speed In addition to improving, when switching from a low resolution mode to a high resolution mode, by converting the reference value for controlling the camera sensor through a linear equation, there is a terminal having a camera function so that there is no delay when switching to the shooting operation during the preview function An object of the present invention is to provide an image processing apparatus and method.

In order to achieve the above object, a terminal having a camera function of the present invention, comprising: a camera sensor for obtaining analog image data having different resolutions for each mode; And switching the modes, converting the analog image data into digital image data, setting a plurality of reference values with reference to analog image data having a reference resolution, and converting the plurality of reference values into a plurality of linear equations, respectively, during mode switching. And a plurality of image processing units to obtain a plurality of adjustment reference values, wherein the image processing unit controls the amplification gain, white balance, and exposure of the camera sensor with reference to the plurality of reference values or the plurality of adjustment reference values.

In order to achieve the above object, an image processing method in a terminal having a camera function of the present invention includes a camera sensor, a digital signal processing unit, a main processing unit, a memory, and a display unit. Calculating an exposure control reference value, an amplification gain control reference value, and a white balance reference value for controlling the camera sensor in a mode; When the mode is changed from the normal mode to the high pixel mode, an exposure control adjustment reference value is calculated using the exposure control reference value and the exposure control slope value, and an amplification gain control adjustment reference value is calculated using the amplification gain control reference value and the amplification gain slope value. Calculating a white balance adjustment reference value using the white balance reference value; And a third step of the main processor adjusting the camera sensor using the exposure control adjustment reference value calculated in the second step, the amplification gain control adjustment reference value and the white balance adjustment reference value in the high pixel mode.

Hereinafter, the most preferred embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art can easily implement the technical idea of the present invention. .

First, FIG. 2 is a block diagram illustrating an image processing apparatus in a terminal having a camera function according to an embodiment of the present invention. In the terminal having a camera function of the present invention, the image processing apparatus includes a camera sensor 210. , An image processor 220, a display 230, and a storage 240.

The camera sensor 210 acquires analog image data having different resolutions for each mode. That is, the camera sensor 210 converts an external image input through a lens (not shown) into analog image data. Here, the camera sensor 210 may be a CMOS sensor or a CCD sensor that converts an input image into an electronic signal, but is not limited thereto.

In addition, the image processing unit 220 switches the mode, converts analog image data input from the camera sensor 210 into digital image data, and converts a plurality of reference values with reference to analog image data having a reference resolution. In the mode switching, the plurality of reference values are applied to a plurality of linear equations to obtain a plurality of adjustment reference values. Here, the image processor 220 may control the amplification gain, white balance, and exposure of the camera sensor 210 with reference to the plurality of reference values or the plurality of adjustment reference values. In this case, the image processor 220 will be described in detail as follows.

The digital signal processor 221 mounted in the image processor 220 converts the analog image data input from the camera sensor 210 into digital image data, and the main processor 223 which describes the converted digital image data later. It prints in). At this time, the digital signal processor 221 calculates the plurality of reference values which are reference values for AEC (Automatic Exposure Control), AGC (Automatic Gain Control) and AWB (Automatic White Balance) according to the external environment change.

The reference value calculator 222 mounted in the image processor 220 receives the plurality of reference values from the digital signal processor 221 and substitutes the plurality of reference values into preset linear equations to adjust the plurality of adjustment reference values. Serves to obtain.

In this case, the plurality of linear equations include a linear equation for exposure control, and the linear equation for exposure control is as shown in Equation 1 below.

는 상기 기준값 중 노출 제어용 기준값이고,

는 상기 조정 기준값 중 노출 제어용 조정 기준값이며, a는 노출 제어 기울기값이고, b는 노출 제어 오프셋값이다. 또한, 상기 a 및 상기 b는 상기 카메라 센서에 입력되는 빛의 양을 변화시켜 복수개의 상기 노출 제어용 기준값 및 상기 노출 제어용 조정 기준값 쌍을 구하여 획득한 그래프에 의해 결정될 수 있으나 이에 한정되지 않는다.here,

Is a reference value for exposure control among the reference values,

Is an adjustment reference value for exposure control among the adjustment reference values, a is an exposure control inclination value, and b is an exposure control offset value. The a and b may be determined by a graph obtained by obtaining a plurality of pairs of the exposure control reference values and the exposure control adjustment reference values by changing the amount of light input to the camera sensor, but are not limited thereto.

In addition, the plurality of linear equations include a linear equation for amplification gain, and the linear equation for amplification gain is expressed by Equation 2 below.

는 상기 기준값 중 증폭 이득용 기준값이고,

는 상기 조정 기준값 중 증폭 이득용 조정 기준값이며, c는 증폭 이득 기울기값이다. 또한, 상기 c는 상기 일반 모드에서 획득되는 아날로그 영상 데이터의 해상도에 대한 상기 고화소 모드에서 획득되는 아날로그 영상 데이터의 해상도의 비의 증가에 따라 증가할 수 있으나, 이에 한정되지 않는다.here,

Is an amplification gain reference value among the reference values,

Is an amplification gain adjustment reference value among the adjustment reference values, and c is an amplification gain slope value. In addition, the c may increase as the ratio of the resolution of the analog image data obtained in the high pixel mode to the resolution of the analog image data obtained in the normal mode is not limited thereto.

In addition, the plurality of linear equations include a linear equation for white balance, and the linear equation for white balance is as shown in Equation 3 below.

는 상기 기준값 중 화이트밸런스용 기준값이고,

는 상기 조정 기준값 중 화이트밸런스용 조정 기준값이다. 즉, 상기 조정 기준값 중 화이트밸런스용 조정 기준값은, 상기 기준값 중 화이트밸런스용 기준값을 그대로 사용하는 것이 바람직하나, 이에 한정되지 않는다.here,

Is a reference value for white balance among the reference values,

Is an adjustment reference value for white balance among the adjustment reference values. That is, the white balance adjustment reference value among the adjustment reference values is preferably used as the white balance reference value of the reference value as it is, but is not limited thereto.

Meanwhile, the main processor 223 mounted in the image processor 220 performs the mode switching control, receives digital image data from the digital signal processor 221, and receives the digital image data by the digital signal processor 221. The camera sensor according to the calculated exposure control reference value, the amplification gain control reference value and the white balance reference value or the exposure control adjustment reference value calculated by the reference value calculator 222, the amplification gain control adjustment reference value and the white balance adjustment reference value. 210, and processes digital image data to generate display image data or storage image data.

In addition, the display unit 230 receives the image data for display from the image processor 220 and displays the image for recognition by the user. Here, the reference resolution may be a resolution (128 × 160) applied to the display window of the display unit 230, but is not limited thereto.

Meanwhile, the storage unit 240 receives and stores the image data for storage from the image processor 220 and provides the stored data to the image processor 220 under the control of the image processor 220. Do it.

The operation of the image processing apparatus in the terminal having the camera function of the present invention described above is as follows.

First, the camera function is activated, the camera sensor 210 converts an external image input through the lens into analog image data, and then the digital signal processing unit 221 is input from the camera sensor 210. Convert analog video data into digital video data. Here, the camera sensor 210, when activated for the first time, may operate as a VGA class or lower sensor into which a frame having 640x480 pixels is input.

At this time, the digital signal processor 221 calculates reference values for amplification gain, white balance, and exposure control of the camera sensor 210 with reference to the digital image data of one frame unit.

Thereafter, the main processor 223 adjusts the camera sensor 210 according to reference values for amplification gain, white balance, and exposure control of the camera sensor 210 calculated by the digital signal processor 221.

On the other hand, during shooting, the reference value calculator 222 substitutes a reference value for amplification gain, white balance, and exposure control of the camera sensor 210 calculated by the digital signal processor 221 into a linear equation to provide a sensor of VGA class or higher ( For example, an adjustment reference value for amplification gain, white balance, and exposure control, which is suitable for a SXGA class sensor into which a frame having 1,280 x 960 pixels is inputted, is calculated.

Thereafter, the main processor 223 adjusts the camera sensor 210 according to an adjustment reference value for amplification gain, white balance, and exposure control of the camera sensor 210 calculated by the reference value calculator 222.

3 is a flowchart illustrating an image processing method in a terminal having a camera function according to an embodiment of the present invention.

First, a camera function is activated and a plurality of reference values are set with reference to analog image data having a resolution for a normal mode (S301). That is, the camera sensor 210 converts an external image input through the lens into analog image data, and then the digital signal processing unit 221 converts the analog image data input from the camera sensor 210 into a digital image. While converting the data, reference values for amplification gain, white balance, and exposure control of the camera sensor 210 are calculated with reference to the digital image data of one frame unit. At this time, the main processor 223 adjusts the camera sensor 210 according to a reference value for amplification gain, white balance, and exposure control of the camera sensor 210 calculated by the digital signal processor 221.

Thereafter, one of the normal mode and the high pixel mode is selected (S302). Here, the high pixel mode may be selected only while the photographing operation is performed, but is not limited thereto. The input image in the high pixel mode preferably has a larger resolution than the normal mode resolution.

If the normal mode is selected, the input image is processed according to the plurality of reference values set at the time of camera activation initialization (S303). That is, the normal mode is selected when the preview operation is performed so that the image frame of the high pixel is not required, thereby increasing the processing speed.

On the other hand, when the high pixel mode is selected, after processing the input image according to the plurality of adjustment reference values obtained by applying the plurality of reference values to a linear equation, and proceeds to the mode selection step (S302) (S304). In other words, the camera module operation can be continuously implemented since the adjustment reference value for the high pixel mode is calculated immediately by substituting the linear equation without disabling the reference value in the high pixel mode.

In this case, the linear equation for exposure control is the same as that of Equation 1, the amplification gain linear equation is the same as Equation 2, and the linear equation for white balance is the same as Equation 3 described above.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited to the drawings shown.

According to the present invention, in a terminal equipped with a camera module, the preview resolution is set differently from the imaging resolution, and the high resolution is set only at the time of imaging, thereby not only improving data processing speed but also low resolution mode. By converting the reference value for controlling the camera sensor through the linear equation when switching from the high resolution mode to, there is an advantage that there is no delay when switching to the shooting operation during the preview function.

Claims (13)

A camera sensor for acquiring analog image data having different resolutions for each mode; And Switch the mode, convert the analog image data into digital image data, set a plurality of reference values with reference to analog image data having a reference resolution, and apply the plurality of reference values to a plurality of linear equations, respectively, during mode switching. A plurality of image processing units to obtain a plurality of adjustment reference values And the image processor is a terminal having a camera function for controlling amplification gain, white balance, and exposure of the camera sensor with reference to the plurality of reference values or the plurality of adjustment reference values. The method of claim 1, The image processor, Digital signal processing means for converting analog image data input from the camera sensor into digital image data and calculating an exposure control reference value, an amplification gain control reference value and a white balance reference value; When switching from the normal mode to the high pixel mode, the exposure control adjustment reference value is calculated using the exposure control reference value and the exposure control inclination value, and the amplification gain control adjustment reference value is calculated using the amplification gain control reference value and the amplification gain inclination value, Reference value calculating means for calculating a white balance adjustment reference value using the white balance reference value; And Perform mode switching control between the normal mode and the high pixel mode, and adjust the camera sensor according to an exposure control reference value, an amplification gain control reference value and a white balance reference value input from the digital signal processing means, or from the reference value calculation means. And main processing means for adjusting the camera sensor according to an input exposure control adjustment reference value, an amplification gain control adjustment reference value, and a white balance adjustment reference value, and processing digital image data input from the digital signal processing means. An image processing apparatus according to a terminal having a camera function. The method of claim 2, And the main processing means switches a mode to the high pixel mode during a photographing operation. The method of claim 2, The reference value calculating means, And the exposure control offset value is calculated by multiplying the exposure control reference value by the exposure control inclination value, and then calculating the exposure control adjustment reference value. The method of claim 2, The reference value calculating means, And multiplying the amplification gain control reference value by the amplification gain slope value to calculate the amplification gain control adjustment reference value. The method of claim 2, The reference value calculating means, And the white balance reference value is used as the white balance adjustment reference value. delete In the image processing method in a terminal comprising a camera sensor, a digital signal processing unit, a main processing unit, a memory and a display unit, A first step of calculating an exposure control reference value, an amplification gain control reference value, and a white balance reference value for controlling the camera sensor in a normal mode; When the mode is changed from the normal mode to the high pixel mode, an exposure control adjustment reference value is calculated using the exposure control reference value and the exposure control slope value, and an amplification gain control adjustment reference value is calculated using the amplification gain control reference value and the amplification gain slope value. Calculating a white balance adjustment reference value using the white balance reference value; And And a third step of the main processor adjusting the camera sensor using the exposure control adjustment reference value calculated in the second step, the amplification gain control adjustment reference value and the white balance adjustment reference value in the high pixel mode. Image processing method. delete delete The method of claim 8, The exposure control adjustment reference value in the second step is And multiplying the exposure control reference value by the exposure control inclination value and then adding an exposure control offset value. The method of claim 8, The amplification gain control adjustment reference value in the second step is And multiplying the amplification gain control reference value by the amplification gain slope value. The method of claim 8, The white balance adjustment reference value in the second step is the same as the white balance reference value.

Images