CN101009851B - Image processing method and its device - Google Patents

Abstract

The image process method comprises: A. multiplying these linear-calculation matrixes, including the transition matrix to transfer pixel RGB data into color space with separated brightness and chroma into a unique processing matrix; and B. multiplying the said unique matrix to pixel EGB data to obtain the pixel point data with separated brightness and chroma, then taking Gamma correction for brightness component. This invention increases processing speed, reduces difficulty on chip design and production, and decreases power consumption and cost.

Description

A kind of image processing method and device thereof

Technical field

The present invention relates to a kind of image processing apparatus and processing method thereof.

Background technology

Digital image pick-up is treated to rgb color space in the view data with the imageing sensor collection pixel number according to after; usually also need to carry out colour correction, color space conversion, tone adjustment, saturation adjustment and Y gamma rear end color treatments such as (Gamma), to obtain more clear, bright-coloured image.As shown in Figure 1, comprise following process:

1) colour correction: the pixel RGB data that will obtain the view data processing (as various interpolation processing) that image sensor is gathered are proofreaied and correct, and obtain the standard RGB data of pixel, to obtain the more bright-coloured image of color; This operation realizes by the color correction matrix that will import data and multiply by one 3 * 3;

2) RGB gamma: by carrying out table lookup operation, R, G, B component data are carried out gamma correction respectively, with the nonlinear characteristic of compensation display;

3) RGB to YUV: the pixel number certificate of rgb color space be multiply by one 3 * 3 color space transition matrix, obtain the pixel number certificate of YUV color space; Because a lot of image compression algorithms carry out on the YUV color space as JPEG (Joint Photographic Experts Group, JPEG (joint photographic experts group)) algorithm, so need be with pixel number according to being transformed into the YUV color space from rgb color space;

4) tone adjustment: color component U and V to the pixel number certificate handle, and carry out the adjustment of tone;

5) saturation adjustment: color component U and V to the pixel number certificate handle, and carry out the saturation adjustment;

6) yuv data after output is handled.

The rear end color processing method of above-mentioned pixel number certificate is in step 2) need carry out table lookup operation respectively according to R, G, the B component of each pixel to pixel number, and in step 1), 3), 4), 5) all need to carry out repeatedly multiplication and add operation, processing speed is slow, and the logic that corresponding picture processing chip takies is many, chip area is big, so power consumption and design production cost are all higher.

Summary of the invention

The technical problem to be solved in the present invention is, overcomes the deficiency of image processing method in the prior art, proposes a kind of processing speed faster, image display effect better rear end color processing apparatus and processing method thereof after handling.

In order to address the above problem, the invention provides a kind of image processing method, this method comprises following steps:

Steps A: with merging into a single processing array behind the used a plurality of matrix multiples that the image pixel point data carried out linear operation in the image processing, comprising the color space transition matrix that the RGB data transaction of pixel is separated to YC;

Step B: the RGB data of each pixel of image be multiply by described single processing array, obtain the pixel number certificate that YC separates, and then carry out the gamma correction of luminance component.

In addition, described a plurality of matrixes that the image pixel point data is carried out linear operation also comprise color correction matrix used when the RGB adjustment of data with pixel is standard RGB data.

In addition, described a plurality of matrixes that the image pixel point data is carried out linear operation also comprise colourity processing array used when the chromatic component of the pixel number certificate of the color space that YC is separated is handled, this colourity processing array is that tone is adjusted matrix, or be saturation adjustment matrix, or form by tone adjustment matrix and the merging of saturation adjustment matrix.

In addition, described a plurality of matrixes that the image pixel point data is carried out linear operation comprise that also matrix was adjusted in used brightness when luminance component to the pixel number certificate carried out brightness and adjusts.

In addition, described color space transition matrix is for being transformed into the pixel number certificate in the matrix of YUV or YCbCr or YIQ color space.

In addition, described color space transition matrix is for being transformed into the pixel number certificate in the approximate color space conversion matrix of LAB color space.

The present invention also provides a kind of image processing apparatus, comprising:

The integrated treatment unit, disposed in the image processing image pixel point data is carried out merging the single processing array that obtains behind a plurality of matrix multiples of linear operation, after receiving the RGB data of each pixel of image, it be multiply by described single processing array, obtain pixel number certificate and output that YC separates;

The luminance component gammate is used for the luminance component of the pixel number certificate of integrated treatment unit output is carried out gamma correction.

In addition, the single processing array of described integrated treatment configuration of cells is formed by following linear matrix merging:

The color space transition matrix is the used matrix of color space that the RGB data transaction of pixel is separated to YC; And one or combination in any in the following matrix:

Used color correction matrix when the RGB adjustment of data of pixel is standard RGB data;

Matrix was adjusted in used brightness when the luminance component of pixel number certificate was carried out brightness and adjusts;

Used colourity processing array when the chromatic component of the pixel number certificate of the color space that YC is separated is handled.

In addition, described colourity processing array is that tone is adjusted matrix, or is saturation adjustment matrix, or is formed by tone adjustment matrix and the merging of saturation adjustment matrix.

In addition, described color space transition matrix is the matrix that the pixel number certificate is transformed into YUV or YCbCr or YIQ color space, or for the pixel number certificate being transformed into the approximate color space conversion matrix of LAB color space.

As from the foregoing, owing to use the Y gamma operation to replace the RGB gamma operation, and use single processing array that the pixel number certificate is handled, the rear end color treatments speed and the treatment effect of pixel number certificate have been accelerated, reduce the difficulty of image chip design and production, reduced the power consumption and the cost of equipment.

Description of drawings

Fig. 1 is the flow chart of image rear end color treatments in the prior art;

Fig. 2 is the substep process of analysis figure of image processing method of the present invention;

Fig. 3 is the flow chart of image processing method of the present invention;

Fig. 4 is the structural representation of image processing apparatus of the present invention.

Embodiment

Basic ideas of the present invention are to use single processing array that the pixel number certificate is carried out linear process, and adopt the Y gamma operation to replace the RGB gamma operation, and the rear end color treatments speed of image is significantly improved.

As mentioned above, image back-end processing of the prior art need carried out the RGB gamma operation to pixel number usually according to after carrying out colour correction, this is operating as non-linear table lookup operation, just can carry out linear operations such as RGB to YUV, tone adjustment after this operation is finished.Because the relative colorimetric data, human eye is more responsive to brightness, not only can reduce the number of times of tabling look-up so above-mentioned RGB gamma operation is replaced to the Y gamma operation, and image processing effect is also better.

Below in conjunction with drawings and Examples image processing method of the present invention is described in detail.

Fig. 2 is the substep process of analysis figure of present embodiment image processing method.This flow process is for principle of the present invention is described, the actual flow process of carrying out please be joined the flow process among Fig. 3 and Fig. 4.As shown in Figure 2, the image processing method that adopts the Y gamma operation to replace the RGB gamma operation comprises following processing procedure:

Step 1: colour correction; To handle the pixel RGB data that obtain to view data and proofread and correct, obtain standard RGB (sRGB) data of pixel, to obtain the more bright-coloured image of color.The color correction matrix C that the RGB component of the pixel number certificate of this operation by will input multiply by one 3 * 3 in step realizes:

$\left[\begin{array}{c}r\\ g\\ b\end{array}\right]=C\×\left[\begin{array}{c}r\_\mathrm{in}\\ g\_\mathrm{in}\\ b\_\mathrm{in}\end{array}\right];$ Wherein $C=\left[\begin{array}{ccc}c11& c12& c13\\ c21& c22& c23\\ c31& c32& c33\end{array}\right];$

Above-mentioned matrix is by the chromatic characteristic decision of image sensor, and for the image sensor with same color characteristic, this Matrix C is identical constant matrix.

Aforesaid operations is to the RGB component of a pixel, need carry out 9 multiplication and 6 sub-additions.

Step 2:RGB to YUV: the RGB component of the pixel number certificate of rgb color space be multiply by one 3 * 3 color space transition matrix D, obtain the pixel number certificate of YUV color space:

$\left[\begin{array}{c}y0\\ u0\\ v0\end{array}\right]=D\×\left[\begin{array}{c}r\\ g\\ b\end{array}\right];$ Wherein $D=\left[\begin{array}{ccc}d11& d12& d13\\ d21& d22& d23\\ d31& d32& d33\end{array}\right];$

Above-mentioned matrix D is the color space transition matrix of RGB to YUV, as:

$D=\left[\begin{array}{ccc}0.299& 0.587& 0.114\\ -0.148& -0.289& 0.437\\ 0.615& -0.515& -0.100\end{array}\right].$

Aforesaid operations is to the RGB component of a pixel, need carry out 9 multiplication and 6 sub-additions.

In addition, step 2 also can convert the RGB component to the color space of other YC separation, as YCbCr or YIQ color space.Compare with converting the YUV color space to, converting YCbCr or YIQ color space to only is color space transition matrix difference.

If to the LAB color space, the color space that also can use an approximate color space conversion matrix to be similar to is changed with the RGB data transaction.

Step 3: the tone adjustment, the chromatic component U and the V of pixel number certificate carried out following processing:

u1＝u0×ec+v0×ef；

v1＝-u0×ef+v0×ec；

Wherein, ec=cos (θ); Ef=sin (θ); Wherein, θ is hue angle, 0 °≤θ≤360 °; That is :-1≤ec≤1;-1≤ef≤1.

Aforesaid operations can be expressed as:

$\left[\begin{array}{c}y1\\ u1\\ v1\end{array}\right]=\left[\begin{array}{ccc}1& 0& 0\\ 0& \mathrm{ec}& \mathrm{ef}\\ 0& -\mathrm{ef}& \mathrm{ec}\end{array}\right]\left[\begin{array}{c}y0\\ u0\\ v0\end{array}\right];$ Wherein, $\left[\begin{array}{ccc}1& 0& 0\\ 0& \mathrm{ec}& \mathrm{ef}\\ 0& -\mathrm{ef}& \mathrm{ec}\end{array}\right]$ For tone is adjusted matrix.

Aforesaid operations is to the YUV component of a pixel, need carry out 4 multiplication and 2 sub-additions.

Step 4: the saturation adjustment, the color component U and the V of pixel number certificate carried out following processing:

u2＝k×u1；

v2＝k×v1；

Wherein, k is the saturation coefficient, 0≤k≤2; The k value is big more, and the saturation of image is high more.

Aforesaid operations can be expressed as:

$\left[\begin{array}{c}y2\\ u2\\ v2\end{array}\right]=\left[\begin{array}{ccc}1& 0& 0\\ 0& k& 0\\ 0& 0& k\end{array}\right]\left[\begin{array}{c}y1\\ u1\\ v1\end{array}\right];$ Wherein, $\left[\begin{array}{ccc}1& 0& 0\\ 0& k& 0\\ 0& 0& k\end{array}\right]$ For saturation is adjusted matrix.

Aforesaid operations need carry out multiplication 2 times to the YUV component of a pixel.

Above-mentioned steps 3 and step 4 can be merged into one step:

u2＝k×ec×u0+k×ef×v0；

v2＝-k×ef×u0+k×ec×v0；

That is:

$\left[\begin{array}{c}y2\\ u2\\ v2\end{array}\right]=\left[\begin{array}{ccc}1& 0& 0\\ 0& k\×\mathrm{ec}& k\×\mathrm{ef}\\ 0& -k\×\mathrm{ef}& k\×\mathrm{ec}\end{array}\right]\×\left[\begin{array}{c}y0\\ u0\\ v0\end{array}\right];$

Wherein, $\left[\begin{array}{ccc}1& 0& 0\\ 0& k\×\mathrm{ec}& k\×\mathrm{ef}\\ 0& -k\×\mathrm{ef}& k\×\mathrm{ec}\end{array}\right]$ Be the colourity processing array, k * ec and k * ef are that colourity is handled coefficient.

Step 5:Y gamma carries out gamma correction to the Y component, to compensate the nonlinear characteristic of display:

This operation is carried out table lookup operation to the Y component of pixel number certificate, and output pixel point data { y_out, u_out, v_out}; Wherein y_out is the gained result that tables look-up according to the y2 value, u_out=u2; V_out=v2.

As seen from the above description, because step 1 all is linear operation to the image slices vegetarian refreshments to step 4, so can use single processing array TotalMatrix combined statement to be shown:

$\left[\begin{array}{c}y2\\ u2\\ v2\end{array}\right]=\mathrm{TotalMatrix}\×\left[\begin{array}{c}r\_\mathrm{in}\\ g\_\mathrm{in}\\ b\_\mathrm{in}\end{array}\right];$

Wherein,

$\mathrm{TotalMatrix}=\left[\begin{array}{ccc}1& 0& 0\\ 0& k& 0\\ 0& 0& k\end{array}\right]\×\left[\begin{array}{ccc}1& 0& 0\\ 0& \mathrm{ec}& \mathrm{ef}\\ 0& -\mathrm{ef}& \mathrm{ec}\end{array}\right]\×\left[\begin{array}{ccc}d11& d12& d13\\ d21& d22& d23\\ d31& d32& d33\end{array}\right]\×\left[\begin{array}{ccc}c11& c12& c13\\ c21& c22& c23\\ c31& c32& c33\end{array}\right]=$

$\left[\begin{array}{ccc}t11& t12& t13\\ t21& t22& t23\\ t31& t32& t33\end{array}\right];$ (formula 1)

Wherein:

t11＝d11×c11+d12×c21+d13×c31；

t12＝d11×c12+d12×c22+d13×c32；

t13＝d11×c13+d12×c23+d13×c33；

t21＝k×ec×(d21×c11+d22×c21+d23×c31)+k×ef×(d31×c11+d32×c21+d33×c31)；

t22＝k×ec×(d21×c12+d22×c22+d23×c32)+k×ef×(d31×c12+d32×c22+d33×c32)；

t23＝k×ec×(d21×c13+d22×c23+d23×c33)+k×ef×(d31×c13+d32×c23+d33×c33)；

t31＝-k×ef×(d21×c11+d22×c21+d23×c31)+k×ec×(d31×c11+d32×c21+d33×c31)；

t32＝-k×ef×(d21×c12+d22×c22+d23×c32)+k×ec×(d31×c12+d32×c22+d33×c32)；

t33＝-k×ef×(d21×c13+d22×c23+d23×c33)+k×ec×(d31×c13+d32×c23+d33×c33)

In addition, among another embodiment, before above-mentioned steps 5, also can add brightness adjustment operation:

Y '=y * brightness; And keeping u, v is constant.

When needs increase image brightness, make brightness＞1;

When needs reduce image brightness, make 0＜brightness＜1; Wherein, brightness is that coefficient is adjusted in brightness.

Aforesaid operations can be expressed as:

$\left[\begin{array}{c}{y}^{\′}\\ u^{\′}\\ v^{\′}\end{array}\right]=\left[\begin{array}{ccc}\mathrm{brightness}& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\×\left[\begin{array}{c}y\\ u\\ v\end{array}\right];$ Wherein, $\left[\begin{array}{ccc}\mathrm{brightness}& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$ For matrix is adjusted in brightness.

Therefore, above-mentioned single processing array:

$\mathrm{TotalMatrix}$

$=\left[\begin{array}{ccc}\mathrm{brightness}& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\×\left[\begin{array}{ccc}1& 0& 0\\ 0& k& 0\\ 0& 0& k\end{array}\right]\×\left[\begin{array}{ccc}1& 0& 0\\ 0& \mathrm{ec}& \mathrm{ef}\\ 0& -\mathrm{ef}& \mathrm{ec}\end{array}\right]\×\left[\begin{array}{ccc}d11& d12& d13\\ d21& d22& d23\\ d31& d32& d33\end{array}\right]\×\left[\begin{array}{ccc}c11& c12& c13\\ c21& c22& c23\\ c31& c32& c33\end{array}\right]=$

$\left[\begin{array}{ccc}t11& t12& t13\\ t21& t22& t23\\ t31& t32& t33\end{array}\right];$

Wherein, coefficient t11, t12, t13 become:

t11＝brightness×d11×c11+brightness×d12×c21+brightness×d13×c31；

t12＝brightness×d11×c12+brightness×d12×c22+brightness×d13×c32；

t13＝brightness×d11×c13+brightness×d12×c23+brightness×d13×c33；

All the other coefficients are identical with formula 1.

Wherein, back-end processing coefficient: it is variable that coefficient brightness is adjusted in hue angle θ, saturation coefficient k, brightness, is provided with in advance by the user usually, and other parameter is all constant.Like this, as long as we calculate single processing array TotalMatrix in advance according to the back-end processing coefficient, just can be only each pixel of image be carried out a matrix multiply operation and a table lookup operation can be finished rear end color treatments operation, greatly reduce amount of calculation, saved the quantity and the complexity of chip logic.

The RGB component of a pixel of image data processing for example, original method need be carried out multiplication 24 times, 14 sub-additions and tabling look-up for 3 times.And the present invention adopts as the rear end color processing method of the single processing array of formula 1 calculating only needs multiplication 9 times, 6 sub-additions and tabling look-up for 1 time.Be that each pixel is saved 15 multiplication, 8 sub-additions and 2 table lookup operations.

In sum, as shown in Figure 3, the color of image processing method of present embodiment comprises following steps:

Step a: will merge into a single processing array behind the used a plurality of matrix multiples that the image pixel point data carried out linear operation in the image processing, the matrix that is used for merging sees above, and repeats no more;

Make i=1;

Step b: the RGB data of i pixel of reading images, these data can be the view data of gathering to be handled obtain;

Step c: the RGB data of i pixel be multiply by above-mentioned single processing array, the pixel number certificate that the output YC separates;

Steps d: the luminance component of the pixel number certificate that above-mentioned YC is separated carries out gamma correction;

Step e:, jump to step b if i＜N makes i=i+1; Otherwise this method finishes.

Wherein, 1≤i≤N, N are image slices vegetarian refreshments sum.

In the present embodiment, single processing array comprises color correction matrix, color space transition matrix, tone adjusts matrix and saturation is adjusted matrix, can also comprise brightness and adjust matrix.Wherein, the color space transition matrix is necessary, and other matrix can select one of them or combination in any, promptly above-mentioned colour correction, tone adjustment, saturation adjustment and brightness adjustment can only carry out one of them or combination in any.

Fig. 4 is the structural representation of present embodiment image processing apparatus.

As shown in Figure 4, image processing apparatus of the present invention comprises:

The integrated treatment unit, disposed the image pixel point data is carried out merging the single processing array that obtains behind a plurality of matrix multiples of linear operation, be used for after receiving the RGB data of each pixel of image, it be multiply by described single processing array, obtain pixel number certificate and output that YC separates;

The luminance component gammate is used for the luminance component of the pixel number certificate of integrated treatment unit output is carried out gamma correction.

The calculating of above-mentioned single processing array is described in detail hereinbefore.

As seen from the above description, for PC camera in the market, generally adopt VGA (VideoGraphics Array, Video Graphics Array) standard, the image size is 640 * 480, be total to about 300,000 pixels, and digital camera is generally five mega pixels; Therefore, adopt rear end of the present invention color processing method greatly to save amount of calculation, accelerated processing speed, saved the chip design logic, reduced energy consumption.

Claims (10)

1. image processing method, this method comprises following steps:

Steps A: with merging into a single processing array behind the used a plurality of matrix multiples that the image pixel point data carried out linear operation in the image processing, comprising the color space transition matrix that the RGB data transaction of pixel is separated to YC;

Step B: the RGB data of each pixel of image be multiply by described single processing array, obtain the pixel number certificate that YC separates, and then carry out the gamma correction of luminance component.

2. image processing method as claimed in claim 1 is characterized in that, described a plurality of matrixes that the image pixel point data is carried out linear operation also comprise color correction matrix used when the RGB adjustment of data with pixel is standard RGB data.

3. image processing method as claimed in claim 1, it is characterized in that, described a plurality of matrixes that the image pixel point data is carried out linear operation also comprise colourity processing array used when the chromatic component of the pixel number certificate of the color space that YC is separated is handled, this colourity processing array is that tone is adjusted matrix, or be saturation adjustment matrix, or form by tone adjustment matrix and the merging of saturation adjustment matrix.

4. image processing method as claimed in claim 1 is characterized in that, described a plurality of matrixes that the image pixel point data is carried out linear operation comprise that also matrix was adjusted in used brightness when the luminance component to the pixel number certificate carried out brightness and adjusts.

5. image processing method as claimed in claim 1 is characterized in that, described color space transition matrix is for being transformed into the pixel number certificate in the matrix of YUV or YCbCr or YIQ color space.

6. image processing method as claimed in claim 1 is characterized in that, described color space transition matrix is for being transformed into the pixel number certificate in the approximate color space conversion matrix of LAB color space.

7. an image processing apparatus is characterized in that, comprising:

The integrated treatment unit, disposed in the image processing image pixel point data is carried out merging the single processing array that obtains behind a plurality of matrix multiples of linear operation, after receiving the RGB data of each pixel of image, it be multiply by described single processing array, obtain pixel number certificate and output that YC separates;

The luminance component gammate is used for the luminance component of the pixel number certificate of integrated treatment unit output is carried out gamma correction.

8. image processing apparatus as claimed in claim 7 is characterized in that, the single processing array of described integrated treatment configuration of cells is formed by following linear matrix merging:

The color space transition matrix is the used matrix of color space that the RGB data transaction of pixel is separated to YC; And

One or combination in any in the following matrix:

Used color correction matrix when the RGB adjustment of data of pixel is standard RGB data;

Matrix was adjusted in used brightness when the luminance component of pixel number certificate was carried out brightness and adjusts;

Used colourity processing array when the chromatic component of the pixel number certificate of the color space that YC is separated is handled.

9. image processing apparatus as claimed in claim 8 is characterized in that, described colourity processing array is that tone is adjusted matrix, or is saturation adjustment matrix, or is formed by tone adjustment matrix and the merging of saturation adjustment matrix.

10. image processing apparatus as claimed in claim 8, it is characterized in that, described color space transition matrix is the matrix that the pixel number certificate is transformed into YUV or YCbCr or YIQ color space, or for the pixel number certificate being transformed into the approximate color space conversion matrix of LAB color space.

Images