CN102457722A - Processing method and device for Bayer image - Google Patents
Processing method and device for Bayer image Download PDFInfo
- Publication number
- CN102457722A CN102457722A CN2010105195243A CN201010519524A CN102457722A CN 102457722 A CN102457722 A CN 102457722A CN 2010105195243 A CN2010105195243 A CN 2010105195243A CN 201010519524 A CN201010519524 A CN 201010519524A CN 102457722 A CN102457722 A CN 102457722A
- Authority
- CN
- China
- Prior art keywords
- pixel
- pixel value
- contiguous
- pixels
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Color Television Image Signal Generators (AREA)
Abstract
The invention discloses a processing method and device for a Bayer image. The processing method comprises the following steps of: reading the original data of the Bayer image in an external memory; for pixels of red R/green G/blue B in the image, and retaining pixel values of the pixels as corresponding R/G/B components of the pixel values after the pixels are processed; carrying out interpolation processing on the original image of the Bayer image, obtaining deletion components of the pixel values after the pixels are processed; caching the data repeatedly read in the interpolation processing process to an on-chip memory; transforming a RGB (Red, Green and Blue) image into a YUV (Luma and Chroma) image; and carrying out video coding on the YUV image. The processing device comprises an external storage, an image interpolation processing unit, the on-chip memory, an image converter and a video coder. According to the processing method and device provided by the invention, only data repeatedly read in a template in the interpolation processing process is needed to be cached to the on-chip memory, thereby the capacity of the on-chip memory and the cost of the interpolation processing of a chip are saved.
Description
Technical field
The present invention relates to a kind of BAYER treatment of picture method and device, particularly relate to a kind of BAYER treatment of picture method and the device that can practice thrift the on-chip memory capacity.
Background technology
Along with the extensive use of camera in field of consumer electronics; Cmos sensor is as the core devices of camera; The BAYER image that it produced is green G pixel and the alternately appearance of blue B pixel in delegation; And alternately occur at next line red R pixel and green G pixel, green G pixel and blue B pixel replace existing and red R pixel and green G pixel and replace and existingly also alternately occur simultaneously.Because the data of original BAYER image do not satisfy the standard of video coding pattern of the input, thus must carry out video input preliminary treatment to the BAYER image, to satisfy the input demand of video encoder.
The BAYER treatment of picture at first is the interpolation processing to the initial data of BAYER image; Through interpolation processing; The pixel value that converts each pixel to has the RGB image of red R, green G and three kinds of color components of blue B; Then the RGB image is obtained the YUV image through colour space transformation, at last the YUV image is encoded, obtain encoded video.
Fig. 1 is the structural representation of the BAYER treatment of picture device of prior art, and is as shown in Figure 1, and traditional BAYER treatment of picture is as independent modular design; The CSI controller is under the triggering of clock pulse; The image information data of the BAYER form of cmos sensor captured in real time is sent to the image interpolation process unit, carries out reception, the processing of data, the RGB image that obtains after handling is carried out colour space transformation; Be met the YUV image of video coding pattern of the input standard; Store the YUV image into external memory storage then, when video encoder need be to the YUV coded image data, read the YUV view data from external memory storage again.
There is the defective of following two aspects in the BAYER image processing apparatus of prior art: on the one hand; In order to accomplish the real-time interpolation processing of grating scanning mode; A bigger row buffering on-chip memory must be arranged; Storage has certain line number and image that have certain width, can make that like this cost of interpolation processing chip is very high; On the other hand; Though the RGB image transitions makes view data significantly reduce after becoming the YUV image; But the data of YUV image still are greater than the initial data of the BAYER image before the interpolation processing; So the memory space of the external memory storage that the BAYER image processing apparatus of prior art needs is bigger, the image transmission bandwidth of requirement is also bigger.
Summary of the invention
The object of the present invention is to provide a kind of BAYER treatment of picture method, the capacity of on-chip memory during with the minimizing image processing, the cost of saving interpolation processing chip.
Another object of the present invention is to provide a kind of BAYER treatment of picture device,, practice thrift the cost of interpolation processing chip to save the capacity of on-chip memory.
A kind of BAYER treatment of picture method for realizing that the object of the invention provides comprises:
The initial data of the BAYER image in the reading external memory;
For the R/G/B pixel in the image, the corresponding R/G/B component of the pixel value after the pixel value that keeps said R/G/B pixel is handled as each pixel;
Initial data to the BAYER image is carried out interpolation processing, obtains the disappearance component of the pixel value after said R/G/B pixel is handled, and with the metadata cache that need repeat to read in the interpolation processing process to on-chip memory;
To carrying out colour space transformation, obtain the YUV image through the RGB image that obtains after the interpolation processing;
The YUV image is carried out video coding, obtain encoded video.
A kind of BAYER treatment of picture device for realizing that the object of the invention provides comprises: external memory storage, image interpolation process unit, on-chip memory, image converter, video encoder;
External memory storage is used to store the initial data of the BAYER image that cmos sensor catches;
The image interpolation process unit; Be used for the initial data that reading external memory is stored; To the R/G/B pixel in the image; The corresponding R/G/B component of the pixel value after the pixel value that keeps the R/G/B pixel is handled as each pixel, and the initial data of BAYER image carried out interpolation processing through the interpolation processing template, obtain the disappearance component of the pixel value after said R/G/B pixel is handled;
On-chip memory is used for the data that memory image interpolation processing process need repeat to read; Image converter is used for obtaining the YUV image to carrying out colour space transformation through the RGB image that obtains after the interpolation processing;
Video encoder is used for the YUV image is encoded, and obtains encoded video.
More excellent ground, BAYER treatment of picture method, the interpolation processing template of employing 5 * 5 is carried out interpolation processing to the initial data of BAYER image.
More excellent ground, BAYER treatment of picture method adopts above-mentioned interpolation processing template that the disappearance component of R/G/B pixel is carried out interpolation calculation and specifically comprises the steps:
For the G pixel; With the pixel value of the pixel value of 4 G pixels the most contiguous of the pixel value of said G pixel, said G pixel, 2 R/B pixels the most contiguous and with the pixel value weighted averages of the most contiguous 2 the contiguous G pixels of the most contiguous said R/B pixel; Calculate the corresponding R/B component of the pixel value after said G pixel is handled; Wherein, The weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of the pixel value of said G pixel, said G pixel and 2 contiguous G pixels is 0, and the weight coefficient sum of the pixel value of said 2 R/B pixels the most contiguous is 1;
For the R/B pixel; Pixel value weighted averages with the pixel value and 4 the contiguous R/B pixels of 4 G pixels the most contiguous of the pixel value of said R/B pixel, said R/B pixel; Calculate the corresponding G component of the pixel value after said R/B pixel is handled; Wherein, The weight coefficient sum of the pixel value of 4 G pixels the most contiguous of said R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of said R/B pixel and said R/B pixel is 0;
For the R/B pixel; With the pixel value of 4 B/R pixels the most contiguous of the pixel value of said R/B pixel, said R/B pixel, the pixel value weighted averages of 4 contiguous R/B pixels; Calculate the corresponding B/R component of the pixel value after said R/B pixel is handled; Wherein, The pixel value weight coefficient sum of 4 B/R pixels the most contiguous of said R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of said R/B pixel and said R/B pixel is 0.
More excellent ground; BAYER treatment of picture method; When asking the R/B component of G pixel; The weight coefficient of the pixel value of G pixel is 3/4, and the weight coefficient that the weight coefficient of the pixel value of the weight coefficient of the pixel value of 4 G pixels the most contiguous of G pixel and 2 contiguous G pixels is the pixel value of-1/8,2 R/B pixel the most contiguous is 1/2.
More excellent ground; BAYER treatment of picture method; When asking the G component of R/B pixel; The weight coefficient of the pixel value of 4 G pixels the most contiguous of R/B pixel is 1/4, and the weight coefficient of the pixel value of R/B pixel is 1/2, and the weight coefficient of the pixel value of 4 contiguous R/B pixels of R/B pixel is-1/8;
When asking the B/R component of R/B pixel; The weight coefficient of the pixel value of 4 B/R pixels the most contiguous of said R/B pixel is 1/4; The weight coefficient of the pixel value of said R/B pixel is 1/2, and the weight coefficient of the pixel value of 4 contiguous R/B pixels of said R/B pixel is-1/8.
More excellent ground, the interpolation processing template of BAYER treatment of picture device is 5 * 5 interpolation processing template.
More excellent ground, the image interpolation process unit of BAYER treatment of picture device is used to adopt above-mentioned 5 * 5 interpolation processing template that the disappearance component of R/G/B pixel is carried out interpolation calculation, and account form is specially:
For the G pixel; With the pixel value of the pixel value of 4 G pixels the most contiguous of the pixel value of said G pixel, said G pixel, 2 R/B pixels the most contiguous and with the pixel value weighted averages of the most contiguous 2 the contiguous G pixels of the most contiguous said R/B pixel; Calculate the corresponding R/B component of the pixel value after said G pixel is handled; Wherein, The weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of the pixel value of said G pixel, said G pixel and 2 contiguous G pixels is 0, and the weight coefficient sum of the pixel value of said 2 R/B pixels the most contiguous is 1;
For the R/B pixel; Pixel value weighted averages with the pixel value and 4 the contiguous R/B pixels of 4 G pixels the most contiguous of the pixel value of said R/B pixel, said R/B pixel; Calculate the corresponding G component of the pixel value after said R/B pixel is handled; Wherein, The weight coefficient sum of the pixel value of 4 G pixels the most contiguous of said R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of said R/B pixel and said R/B pixel is 0;
For the R/B pixel; With the pixel value of 4 B/R pixels the most contiguous of the pixel value of said R/B pixel, said R/B pixel, the pixel value weighted averages of 4 contiguous R/B pixels; Calculate the corresponding B/R component of the pixel value after said R/B pixel is handled; Wherein, The pixel value weight coefficient sum of 4 B/R pixels the most contiguous of said R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of said R/B pixel and said R/B pixel is 0.
More excellent ground, the output of the image converter of BAYER treatment of picture device directly links to each other with the input of video encoder.
More excellent ground, the capacity of the on-chip memory of BAYER treatment of picture device is 400 bytes.
Beneficial effect of the present invention: BAYER treatment of picture method and apparatus of the present invention; Because in the interpolation processing process; Need not store the image that has certain line number and have certain width; Only need so saved the capacity of on-chip memory to a great extent, practice thrift the cost of interpolation processing chip with the metadata cache that repeats to read in the image interpolation process process to on-chip memory.In addition; BAYER treatment of picture method and apparatus of the present invention; External memory stores be the initial data of BAYER image before the interpolation processing; And the data of the initial data of the BAYER image image after with respect to interpolation processing are wanted much less before the interpolation processing, so also saved the capacity of external memory storage, have practiced thrift the cost of image processing.
Description of drawings
Fig. 1 is the structural representation of the BAYER treatment of picture device of prior art;
Fig. 2 is the structural representation of BAYER treatment of picture device one embodiment of the present invention;
Fig. 3 is the flow chart of BAYER treatment of picture method one embodiment of the present invention;
Fig. 4 is the position view of each pixel in the first interpolation processing template in the embodiments of the invention;
Fig. 5 is the position view of each pixel in the second interpolation processing template in the embodiments of the invention;
Fig. 6 is the position view of each pixel in the 3rd interpolation processing template in the embodiments of the invention;
Fig. 7 is the position view of each pixel in the 4th interpolation processing template in the embodiments of the invention.
Embodiment
In order to make the object of the invention, technical scheme and advantage clearer,, BAYER treatment of picture method of the present invention and BAYER image processing apparatus are specified further below in conjunction with accompanying drawing and specific embodiment.
Fig. 3 is the flow chart of BAYER treatment of picture method one embodiment of the present invention, and as shown in Figure 3, this method comprises the steps:
31 steps: the initial data of the BAYER image in the reading external memory;
32 steps: for the blue B pixel of the green G/ of the red R/ in the image, the corresponding R/G/B component of the pixel value after the pixel value that keeps the blue B pixel of the green G/ of said red R/ is handled as each pixel;
33 steps: the initial data to the BAYER image is carried out interpolation processing, obtains the disappearance component of the pixel value after the blue B pixel of the green G/ of said red R/ is handled, and with the metadata cache that repeats to read in the interpolation processing process to on-chip memory;
34 steps:, obtain the YUV image to carrying out colour space transformation through the RGB image that obtains after the interpolation processing;
35 steps: the YUV image is carried out video coding, obtain encoded video.
When the BAYER image processing begins; Under the triggering of clock pulse; The initial data of the BAYER image of storing in the external memory storage is read into the image interpolation process unit; For the blue B pixel of the green G/ of the red R/ in the image, the corresponding R/G/B component of the pixel value after the pixel value that keeps the blue B pixel of the green G/ of red R/ is handled as each pixel, the disappearance component of the pixel value after the processing obtains through the initial data of BAYER image being carried out interpolation processing; Repeat to read the data the interpolation template from external memory storage during for fear of image interpolation process; With the metadata cache that repeats to read in the interpolation processing process to on-chip memory; Accelerated the process of image processing so to a great extent; Also save simultaneously the capacity of on-chip memory, practiced thrift the cost of interpolation processing chip.
In order to make the view data after the processing satisfy video coding pattern of the input standard, the RGB image that obtains is carried out colour space transformation, be met the YUV image of video coding pattern of the input standard.General RGB image obtains having the image of brightness signal Y and two color difference signal R-Y (being U), B-Y (being V) through matrixer; It is the YUV image; The RGB image transitions becomes the YUV image can adopt common method of the prior art, like disclosed method of Chinese patent 200610138960.x etc.
When clock pulse triggers; The YUV image is outputed to video encoder; Adopt basic encryption algorithm like the inter prediction that adopts motion compensation, two-dimensional dct, VLC Run-Length Coding etc. to certain video encoding standard, the YUV image is encoded obtains encoded video.
Because the relative RGB image transitions of initial data of the BAYER image of storing in the external memory storage is that the data behind the YUV image are less, so also saved the capacity of external memory storage greatly, has further reduced the cost of image processing.
The capacity of on-chip memory in order to improve the quality of image interpolation process, is saved in more excellent ground, and the interpolation processing template of BAYER treatment of picture method employing 5 * 5 of the present invention is carried out interpolation processing to the initial data of BAYER image.
More excellent ground; In order further to improve the quality of image interpolation process; BAYER treatment of picture method of the present invention adopts above-mentioned 5 * 5 interpolation template that the initial data of BAYER image is carried out interpolation processing; Obtain the disappearance component of the pixel value of R/G/B pixel; In the interpolation processing process, for the influence to the corresponding R/B component of the pixel value of the G pixel handled of the G pixel of eliminating the neighbor cell territory effectively, the R pixel in neighbor cell territory is to the influence of the corresponding G/B component of the pixel value of the R pixel handled; And the B pixel in neighbor cell territory is to the influence of the corresponding G/R component of the pixel value of the B pixel handled, and interpolation calculation specifically comprises the steps:
In step 1; In order to eliminate of the influence of contiguous G pixel more effectively to the corresponding R/B component of the pixel value of the G pixel handled; When calculating the corresponding R/B component of pixel value after the G pixel is handled; To take into account with 2 the most contiguous inferior pixel values that are close to the G pixels of the most contiguous R/B pixel and carry out weighting, thereby further improve the quality of image processing.
Step 3, for the R/B pixel; With the pixel value of 4 B/R pixels the most contiguous of the pixel value of R/B pixel, R/B pixel, the pixel value weighted averages of 4 contiguous R/B pixels; Calculate the corresponding B/R component of the pixel value after the R/B pixel is handled; Wherein, The weight coefficient sum of the pixel value of 4 B/R pixels the most contiguous of R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of R/B pixel and R/B pixel is 0.
More excellent ground; In order to obtain the performance better pictures; When the G component of the R/B pixel of asking the BAYER image; The weight coefficient of the pixel value of 4 G pixels the most contiguous of R/B pixel is 1/4, and the weight coefficient of the pixel value of R/B pixel is 1/2, and the weight coefficient of the pixel value of 4 contiguous R/B pixels of R/B pixel is-1/8.When asking the B/R component of R/B pixel; The weight coefficient of the pixel value of 4 B/R pixels the most contiguous of R/B pixel is 1/4; The weight coefficient of the pixel value of R/B pixel is 1/2, and the weight coefficient of the pixel value of 4 contiguous R/B pixels of R/B pixel is-1/8.
Fig. 4 is the position view of each pixel in the first interpolation processing template in the embodiments of the invention, and is as shown in Figure 4, and the pixel value that green G pixel and blue B pixel replace the G pixel that the capable j row of existing i in the BAYER image in this template is G
I, j, keep the G component G of the pixel value after this pixel value is handled as this G pixel
Gi, j, i.e. G
Gi, j=G
I, j
The R component G of the pixel value of this G pixel
Ri, j/ B component G
Bi, jPixel value G according to the G pixel
I, j, the G pixel the pixel value G of 4 G pixels the most contiguous
I-1, j-1, G
I-1, j+1, G
I+1, j-1And G
I+1, j+1, the pixel value R of 2 R/B pixels the most contiguous
I-1, jAnd R
I+1, j/ B
I, j-1And B
I, j+1And 2 the inferior pixel value Gs that are close to G pixels the most contiguous with the most contiguous R/B pixel
I-2, jAnd G
I+2, j/ G
I, j-2And G
I, j+2Weighted average obtains, wherein the pixel value G of G pixel
I, j, the pixel value G of 4 G pixels the most contiguous of G pixel
I-1, j-1, G
I-1, j+1, G
I+1, j-1And G
I+1, j+1And the pixel value G of 2 contiguous G pixels
I-2, jAnd G
I+2, j/ G
I, j-2And G
I, j+2The weight coefficient sum be the pixel value R of 0,2 R/B pixel the most contiguous
I-1, jAnd R
I+1, j/ B
I, j-1And B
I, j+1The weight coefficient sum be 1.
More preferably, in order to make the more excellent performance after the image interpolation process, the better effects if of image processing is asked the R/B component G of G pixel
Ri, j/ G
Bi, jThe time, the pixel value G of G pixel
I, jWeight coefficient be 3/4, the pixel value G of 4 G pixels the most contiguous of G pixel
I-1, j-1, G
I-1, j+1, G
I+1, j-1And G
I+1, j+1The pixel value G of weight coefficient and 2 time contiguous G pixels
I-2, jAnd G
I+2, j/ G
I, j-2And G
I, j+2Weight coefficient be the pixel value R of-1/8,2 R/B pixel the most contiguous
I-1, jAnd R
I+1, j/ B
I, j-1And B
I, j+1Weight coefficient be 1/2.The R component G of the pixel value after this G pixel interpolation processing
Ri jWith B component G
Bi, jConcrete calculating formula be:
Fig. 5 is the position view of each pixel in the second interpolation processing template in the embodiments of the invention, and is as shown in Figure 5, and the pixel value that green G pixel and red R pixel replace the G pixel that the capable j row of existing i in the BAYER image in this template is G
I, j, keep the G component G of this pixel value as the pixel value of this G pixel after handling
Gi, j, i.e. G
Gi, j=G
I, jThe R component G of the pixel value of this G pixel
Ri, j/ B component G
Bi, jPixel value G according to the G pixel
I, j, the G pixel the pixel value G of 4 G pixels the most contiguous
I-1, j-1, G
I-1, j+1, G
I+1, j-1And G
I+1, j+1, the pixel value R of 2 R/B pixels the most contiguous
I, j-1And R
I, j+1/ B
I-1, jAnd B
I+1, jAnd 2 the inferior pixel value Gs that are close to G pixels the most contiguous with the most contiguous R/B pixel
I, j-2And G
I, j+2/ G
I-2, jAnd G
I+2, jWeighted average obtains, wherein the pixel value G of G pixel
I, j, the pixel value G of 4 G pixels the most contiguous of G pixel
I-1, j-1, G
I-1, j+1, G
I+1, j-1And G
I+1, j+1And the pixel value G of 2 contiguous G pixels
I, j-2And G
I, j+2/ G
I-2, jAnd G
I+2, jThe weight coefficient sum be the pixel value R of 0,2 R/B pixel the most contiguous
I, j-1And R
I, j+1/ B
I-1, jAnd B
I+1, jThe weight coefficient sum be 1.
More preferably, in order to make the more excellent performance after the image interpolation process, the better effects if of image processing is asked the R/B component G of G pixel
Ri, j/ G
Bi, jThe time, the pixel value G of G pixel
I, jWeight coefficient be 3/4, the pixel value G of 4 G pixels the most contiguous of G pixel
I-1, j-1, G
I-1, j+1, G
I+1, j-1And G
I+1, j+1The pixel value G of weight coefficient and 2 time contiguous G pixels
I, j-2And G
I, j+2/ G
I-2, jAnd G
I+2, jWeight coefficient be the pixel value R of-1/8,2 R/B pixel the most contiguous
I, j-1And R
I, j+1/ B
I-1, jAnd B
I+1, jWeight coefficient be 1/2.The R component G of the pixel value after this G pixel interpolation processing
Ri, jWith B component G
Bi, jConcrete calculating formula be:
Fig. 6 is the position view of each pixel in the 3rd interpolation processing template in the embodiments of the invention, and is as shown in Figure 6, in this template in the BAYER image pixel value of the R pixel of the capable j row of i be R
I, j, keep the R components R of the pixel value after this pixel value is handled as this R pixel
Ri, j, i.e. R
Ri, j=R
I, jThe G components R of the pixel value of this R pixel
Gi, jPixel value R according to the R pixel
I, j, the R pixel the pixel value G of 4 G pixels the most contiguous
I, j-1, G
I, j+1, G
I-1, jAnd G
I+1, jAnd the pixel value R of 4 contiguous R pixels
I, j-2, R
I, j+2, R
I-2, jAnd R
I+2, jWeighted average calculation obtains, wherein, and the pixel value G of 4 G pixels the most contiguous of R pixel
I, j-1, G
I, j+1, G
I-1, jAnd G
I+1, jThe weight coefficient sum be 1, the pixel value R of R pixel
I, jAnd the pixel value R of 4 contiguous R pixels of R pixel
I, j-2, R
I, j+2, R
I-2, jAnd R
I+2, jThe weight coefficient sum be 0.
The B components R of the pixel value of this R pixel
Bi, jPixel value R according to the R pixel
I, j, the R pixel the pixel value B of 4 B pixels the most contiguous
I-1, j-1, B
I-1, j+1, B
I+1, j-1And B
I+1, j+1, 4 contiguous R pixels pixel value R
I, j-2, R
I, j+2, R
I-2, jAnd R
I+2, jWeighted average calculates the corresponding B components R of the pixel value after the R pixel is handled
Bi, j, wherein, the pixel value B of 4 B pixels the most contiguous of R pixel
I-1, j-1, B
I-1, j+1, B
I+1, j-1And B
I+1, j+1The weight coefficient sum be 1, the pixel value R of R pixel
I, jAnd the pixel value R of 4 contiguous R pixels of R pixel
I, j-2, R
I, j+2, R
I-2, jAnd R
I+2, jThe weight coefficient sum be 0.
More preferably, for the more excellent performance that makes image after the image interpolation process, the better effects if of image processing, when the G component of the R pixel of asking the BAYER image, the pixel value G of 4 G pixels the most contiguous of R pixel
I, j-1, G
I, j+1, G
I-1, jAnd G
I+1, jWeight coefficient be 1/4, the pixel value R of R pixel
I, jWeight coefficient be 1/2, the pixel value R of 4 of the R pixel contiguous R pixels
I, j-2, R
I, j+2, R
I-2, jAnd R
I+2, jWeight coefficient be-1/8.Ask the B components R of R pixel
Bi, jThe time, the pixel value B of 4 B pixels the most contiguous of R pixel
I-1, j-1, B
I-i, j+1, B
I+1, j-1And B
I+1, j+1Weight coefficient be 1/4, the pixel value R of R pixel
I, jWeight coefficient be 1/2, the pixel value R of 4 of the R pixel contiguous R pixels
I, j-2, R
I, j+2, R
I-2, jAnd R
I+2, jWeight coefficient be-1/8.
The G components R of the pixel value after this R pixel interpolation processing
Gi, jWith the B components R
Bi, jConcrete calculating formula be:
Fig. 7 is the position view of each pixel in the 4th interpolation processing template in the embodiments of the invention, and is as shown in Figure 7, in this template in the BAYER image pixel value of the B pixel of the capable j row of i be B
I, j, keep the B component B of this pixel value as the pixel value of this B pixel after handling
Bi, j, i.e. B
Bi, j=B
I, jFor this B pixel, with the pixel value B of B pixel
I, j, the B pixel the pixel value G of 4 G pixels the most contiguous
I, j-1, G
I, j+1, G
I-1, jAnd G
I+1, jAnd the pixel value B of 4 contiguous B pixels
I, j-2, B
I, j+2, B
I-2, jAnd B
I+2, jWeighted average calculates the corresponding G component B of the pixel value after the B pixel is handled
Gi, j, wherein, the pixel value G of 4 G pixels the most contiguous of B pixel
I, j-1, G
I, j+1, G
I-1, jAnd G
I+1, jThe weight coefficient sum be 1, the pixel value B of B pixel
I, jAnd the pixel value B of 4 contiguous B pixels of B pixel
I, j-2, B
I, j+2, B
I-2, jAnd B
I+2, jThe weight coefficient sum be 0;
For this B pixel, with the pixel value B of B pixel
I, j, the B pixel the pixel value R of 4 R pixels the most contiguous
I-1, j-1, R
I-1, j+1, R
I+1, j-1And R
I+1, j+1, 4 contiguous B pixels pixel value B
I, j-2, B
I, j+2, B
I-2, jAnd B
I+2, jWeighted average calculates the corresponding R component B of the pixel value after the B pixel is handled
Ri, j, wherein, the pixel value R of 4 R pixels the most contiguous of B pixel
I-1, j-1, R
I-1, j+1, R
I+1, j-1And R
I+1, j+1The weight coefficient sum be 1, the pixel value B of B pixel
I, jAnd the pixel value B of 4 contiguous B pixels of B pixel
I, j-2, B
I, j+2, B
I-2, jAnd B
I+2, jThe weight coefficient sum be 0.
More excellent ground is in order to obtain the performance better pictures, the G component B of the B pixel of BAYER image in asking this template
Gi, jThe time, the pixel value G of 4 G pixels the most contiguous of B pixel
I, j-1, G
I, j+1, G
I-1, jAnd G
I+1, jWeight coefficient be 1/4, the pixel value B of B pixel
I, jWeight coefficient be 1/2, the pixel value B of 4 of the B pixel contiguous B pixels
I, j-2, B
I, j+2, B
I-2, jAnd B
I+2, jWeight coefficient be-1/8.Ask the R component B of B pixel
Ri, jThe time, the pixel value R of 4 R pixels the most contiguous of B pixel
I-1, j-1, R
I-1, j+1, R
I+1, j-1And R
I+1, j+1Weight coefficient be 1/4, the pixel value B of B pixel
I, jWeight coefficient be 1/2, the pixel value B of 4 of the B pixel contiguous B pixels
I, j-2, B
I, j+2, B
I-2, jAnd B
I+2, jWeight coefficient be-1/8.
The G component B of the pixel value after this B pixel interpolation processing
Gi, jWith R component B
Ri, jConcrete calculating formula be:
In the interpolation processing process of BAYER image of the present invention; Has only the storage that repeats to read in the interpolation processing process in on-chip memory; So just save the capacity of on-chip memory, also accelerated the process of image processing simultaneously, practiced thrift the cost of interpolation processing chip.The initial data of the BAYER image of storing in the external memory storage simultaneously is less relatively, so also saved the capacity of external memory storage greatly, has further reduced the cost of image processing.
Fig. 2 is the structural representation of BAYER treatment of picture device one embodiment of the present invention; As shown in Figure 2, BAYER image processing apparatus of the present invention comprises external memory storage, image interpolation process unit, on-chip memory, image converter, the video encoder of the initial data that is used to store the BAYER image that cmos sensor catches.
The CSI controller is under the triggering of clock pulse, and the initial data of the BAYER image that cmos sensor is caught outputs to external memory storage.
The image interpolation process unit is used for the initial data that reading external memory is stored; To the blue B pixel of the green G/ of the red R/ in the image; The corresponding R/G/B component of the pixel value after the pixel value that keeps the blue B pixel of the green G/ of red R/ is handled as each pixel; And the initial data of BAYER image carried out interpolation processing through the interpolation processing template, obtain the disappearance component of the pixel value after the blue B pixel of the green G/ of red R/ is handled, thereby the BAYER image transitions is become the RGB image.
In the interpolation processing process of BAYER image, on-chip memory is used for the data that memory image interpolation processing process need repeat to read, and avoiding repeating reading of data from external memory storage, thereby has accelerated the process of image processing.
Image converter is used for obtaining the YUV image to carrying out colour space transformation through the RGB image that obtains after the interpolation processing, to satisfy video coding pattern of the input standard, satisfies the input demand of video encoder.
Video encoder is used for the YUV image is encoded, and obtains encoded video.
For performance that improves image after the interpolation processing and the capacity of effectively saving on-chip memory, the image interpolation process unit adopts 5 * 5 interpolation processing template that the disappearance component of the R/G/B pixel in the BAYER image is carried out interpolation calculation, and account form is specially:
For the G pixel; With the pixel value of the pixel value of 4 G pixels the most contiguous of the pixel value of G pixel, G pixel, 2 R/B pixels the most contiguous and with the pixel value weighted averages of the most contiguous 2 the contiguous G pixels of the most contiguous R/B pixel; Calculate the corresponding R/B component of the pixel value after the G pixel is handled; Wherein, The weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of the pixel value of G pixel, G pixel and 2 contiguous G pixels is that the weight coefficient sum of the pixel value of 0,2 R/B pixel the most contiguous is 1.
For the R/B pixel; With the pixel value of 4 G pixels the most contiguous of the pixel value of R/B pixel, R/B pixel and the pixel value weighted average of 4 contiguous R/B pixels; Calculate the corresponding G component of the pixel value after the R/B pixel is handled; Wherein, the weight coefficient sum of the pixel value of 4 G pixels the most contiguous of R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of R/B pixel and R/B pixel is 0.
For the R/B pixel; With the pixel value of 4 B/R pixels the most contiguous of the pixel value of R/B pixel, R/B pixel, the pixel value weighted averages of 4 contiguous R/B pixels; Calculate the corresponding B/R component of the pixel value after the R/B pixel is handled; Wherein, the pixel value weight coefficient sum of 4 B/R pixels the most contiguous of R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of R/B pixel and R/B pixel is 0.
The output of image converter directly links to each other with the input of video encoder, has avoided further use external memory storage like this, has saved the capacity of external memory storage.The YUV image of image converter output directly outputs to video encoder under the triggering of clock pulse, through video encoder encodes, obtain encoded video.
Because the interpolation processing template is 5 * 5 interpolation processing template; For satisfying general video encoder like the H.264 input requirement of video encoder; H.264 video encoder once need read 16 * 16 pixel and forms a macro block and encode from external memory storage; The macro block of encoding needs the pixel of 20 * 20 BAYER image, and the on-chip memory of selected like this 400 bytes can be accomplished the high-quality image interpolation process of arbitrary resolution.
Should be noted that at last that obviously those skilled in the art can carry out various changes and modification to the present invention and not break away from the spirit and scope of the present invention.Like this, if of the present invention these revise and modification belongs within the scope of claim of the present invention and equivalent technologies thereof, then the present invention also is intended to comprise these changes and modification.
Claims (10)
1. BAYER treatment of picture method is characterized in that this method comprises:
The initial data of the BAYER image in the reading external memory;
For the R/G/B pixel in the image, the corresponding R/G/B component of the pixel value after the pixel value that keeps said R/G/B pixel is handled as each pixel;
Initial data to the BAYER image is carried out interpolation processing, obtains the disappearance component of the pixel value after said R/G/B pixel is handled, and with the metadata cache that repeats to read in the interpolation processing process to on-chip memory;
To carrying out colour space transformation, obtain the YUV image through the RGB image that obtains after the interpolation processing;
The YUV image is carried out video coding, obtain encoded video.
2. BAYER treatment of picture method according to claim 1 is characterized in that, the interpolation processing template of employing 5 * 5 is carried out interpolation processing to the initial data of BAYER image.
3. BAYER treatment of picture method according to claim 2 is characterized in that, the initial data of BAYER image is carried out interpolation processing, and the disappearance that obtains the pixel value after said R/G/B pixel is handled divides measuring to comprise the steps:
For the G pixel; With the pixel value of the pixel value of 4 G pixels the most contiguous of the pixel value of said G pixel, said G pixel, 2 R/B pixels the most contiguous and with the pixel value weighted averages of the most contiguous 2 the contiguous G pixels of the most contiguous said R/B pixel; Calculate the corresponding R/B component of the pixel value after said G pixel is handled; Wherein, The weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of the pixel value of said G pixel, said G pixel and 2 contiguous G pixels is 0, and the weight coefficient sum of the pixel value of said 2 R/B pixels the most contiguous is 1;
For the R/B pixel; Pixel value weighted averages with the pixel value and 4 the contiguous R/B pixels of 4 G pixels the most contiguous of the pixel value of said R/B pixel, said R/B pixel; Calculate the corresponding G component of the pixel value after said R/B pixel is handled; Wherein, The weight coefficient sum of the pixel value of 4 G pixels the most contiguous of said R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of said R/B pixel and said R/B pixel is 0;
For the R/B pixel; With the pixel value of 4 B/R pixels the most contiguous of the pixel value of said R/B pixel, said R/B pixel, the pixel value weighted averages of 4 contiguous R/B pixels; Calculate the corresponding B/R component of the pixel value after said R/B pixel is handled; Wherein, The weight coefficient sum of the pixel value of 4 B/R pixels the most contiguous of said R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of said R/B pixel and said R/B pixel is 0.
4. BAYER treatment of picture method according to claim 3; It is characterized in that; When asking the R/B component of G pixel; The weight coefficient of the pixel value of said G pixel is 3/4, and the weight coefficient of the pixel value of the weight coefficient of the pixel value of 4 G pixels the most contiguous of said G pixel and 2 contiguous G pixels is-1/8, and the weight coefficient of the pixel value of said 2 R/B pixels the most contiguous is 1/2.
5. BAYER treatment of picture method according to claim 3 is characterized in that:
When asking the G component of R/B pixel; The weight coefficient of the pixel value of 4 G pixels the most contiguous of said R/B pixel is 1/4; The weight coefficient of the pixel value of said R/B pixel is 1/2, and the weight coefficient of the pixel value of 4 contiguous R/B pixels of said R/B pixel is-1/8;
When asking the B/R component of R/B pixel; The weight coefficient of the pixel value of 4 B/R pixels the most contiguous of said R/B pixel is 1/4; The weight coefficient of the pixel value of said R/B pixel is 1/2, and the weight coefficient of the pixel value of 4 contiguous R/B pixels of said R/B pixel is-1/8.
6. a BAYER treatment of picture device is characterized in that, comprising: external memory storage, image interpolation process unit, on-chip memory, image converter, video encoder;
Said external memory storage is used to store the initial data of the BAYER image that cmos sensor catches;
Said image interpolation process unit; Be used for reading the initial data that said external memory storage is stored; To the R/G/B pixel in the image; The corresponding R/G/B component of the pixel value after the pixel value that keeps said R/G/B pixel is handled as each pixel, and the initial data of BAYER image carried out interpolation processing through the interpolation processing template, obtain the disappearance component of the pixel value after said R/G/B pixel is handled;
Said on-chip memory is used for the data that memory image interpolation processing process need repeat to read;
Said image converter is used for obtaining the YUV image to carrying out colour space transformation through the RGB image that obtains after the interpolation processing;
Said video encoder is used for the YUV image is encoded, and obtains encoded video.
7. BAYER treatment of picture device according to claim 6 is characterized in that, said interpolation processing template is 5 * 5 interpolation processing template.
8. BAYER treatment of picture device according to claim 7 is characterized in that, said image interpolation process unit is used to adopt said interpolation processing template that the disappearance component of R/G/B pixel is carried out interpolation calculation, and account form is specially:
For the G pixel; With the pixel value of the pixel value of 4 G pixels the most contiguous of the pixel value of said G pixel, said G pixel, 2 R/B pixels the most contiguous and with the pixel value weighted averages of the most contiguous 2 the contiguous G pixels of the most contiguous said R/B pixel; Calculate the corresponding R/B component of the pixel value after said G pixel is handled; Wherein, The weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of the pixel value of said G pixel, said G pixel and 2 contiguous G pixels is 0, and the weight coefficient sum of the pixel value of said 2 R/B pixels the most contiguous is 1;
For the R/B pixel; Pixel value weighted averages with the pixel value and 4 the contiguous R/B pixels of 4 G pixels the most contiguous of the pixel value of said R/B pixel, said R/B pixel; Calculate the corresponding G component of the pixel value after said R/B pixel is handled; Wherein, The weight coefficient sum of the pixel value of 4 G pixels the most contiguous of said R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of said R/B pixel and said R/B pixel is 0;
For the R/B pixel; With the pixel value of 4 B/R pixels the most contiguous of the pixel value of said R/B pixel, said R/B pixel, the pixel value weighted averages of 4 contiguous R/B pixels; Calculate the corresponding B/R component of the pixel value after said R/B pixel is handled; Wherein, The pixel value weight coefficient sum of 4 B/R pixels the most contiguous of said R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of said R/B pixel and said R/B pixel is 0.
9. according to claim 6,7 or 8 described BAYER treatment of picture devices, it is characterized in that the output of said image converter directly links to each other with the input of said video encoder.
10. BAYER treatment of picture device according to claim 9 is characterized in that the capacity of said on-chip memory is 400 bytes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010519524.3A CN102457722B (en) | 2010-10-26 | 2010-10-26 | Processing method and device for Bayer image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010519524.3A CN102457722B (en) | 2010-10-26 | 2010-10-26 | Processing method and device for Bayer image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102457722A true CN102457722A (en) | 2012-05-16 |
| CN102457722B CN102457722B (en) | 2014-01-22 |
Family
ID=46040317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010519524.3A Active CN102457722B (en) | 2010-10-26 | 2010-10-26 | Processing method and device for Bayer image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102457722B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103716606A (en) * | 2013-12-30 | 2014-04-09 | 上海富瀚微电子有限公司 | Bayer domain image downsampling method and device and camera equipment |
| CN105611256A (en) * | 2015-12-28 | 2016-05-25 | 南京航空航天大学 | Bayer format image interpolation method based on gradients and display device based on FPGA |
| CN105847770A (en) * | 2015-01-15 | 2016-08-10 | 展讯通信（上海）有限公司 | Image processing chip and caching method for image data in chip |
| CN106412542A (en) * | 2016-02-03 | 2017-02-15 | 周彩章 | Image processing method and system |
| CN107016343A (en) * | 2017-03-06 | 2017-08-04 | 西安交通大学 | A kind of traffic lights method for quickly identifying based on Bel's format-pattern |
| CN108650515A (en) * | 2012-07-09 | 2018-10-12 | 株式会社尼康 | Decoding apparatus and image compressing device |
| CN110049337A (en) * | 2019-05-24 | 2019-07-23 | 安翰科技(武汉)股份有限公司 | The compression processing method and system of capsule endoscope bayer image |
| WO2021007742A1 (en) * | 2019-07-15 | 2021-01-21 | 上海极清慧视科技有限公司 | Compression method for obtaining video file, decompression method, system, and storage medium |
| WO2022141515A1 (en) * | 2020-12-31 | 2022-07-07 | 深圳市大疆创新科技有限公司 | Video encoding method and device and video decoding method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1595454A (en) * | 2004-06-25 | 2005-03-16 | 闫宇松 | A high compression ratio portrait coding-decoding method |
| CN1632829A (en) * | 2004-12-27 | 2005-06-29 | 北京中星微电子有限公司 | A method for implementing image zoom |
| CN1761312A (en) * | 2005-11-03 | 2006-04-19 | 西安交通大学 | Method and circuit for zooming digital video image based on dual quadratic interpolation |
| CN1799492A (en) * | 2005-12-02 | 2006-07-12 | 清华大学 | Quasi-lossless image compression and decompression method of wireless endoscope system |
| US7139022B1 (en) * | 2002-06-19 | 2006-11-21 | Neomagic Corp. | Edge enhancer for RGB-Beyer to YUV 4:2:0 converter with sharpened-Y feedback to U, V transformer |
-
2010
- 2010-10-26 CN CN201010519524.3A patent/CN102457722B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7139022B1 (en) * | 2002-06-19 | 2006-11-21 | Neomagic Corp. | Edge enhancer for RGB-Beyer to YUV 4:2:0 converter with sharpened-Y feedback to U, V transformer |
| CN1595454A (en) * | 2004-06-25 | 2005-03-16 | 闫宇松 | A high compression ratio portrait coding-decoding method |
| CN1632829A (en) * | 2004-12-27 | 2005-06-29 | 北京中星微电子有限公司 | A method for implementing image zoom |
| CN1761312A (en) * | 2005-11-03 | 2006-04-19 | 西安交通大学 | Method and circuit for zooming digital video image based on dual quadratic interpolation |
| CN1799492A (en) * | 2005-12-02 | 2006-07-12 | 清华大学 | Quasi-lossless image compression and decompression method of wireless endoscope system |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108650515A (en) * | 2012-07-09 | 2018-10-12 | 株式会社尼康 | Decoding apparatus and image compressing device |
| CN108650516A (en) * | 2012-07-09 | 2018-10-12 | 株式会社尼康 | Compression set, image compressing device and filming apparatus |
| CN103716606B (en) * | 2013-12-30 | 2015-10-14 | 上海富瀚微电子股份有限公司 | A kind of method of Bayer area image down-sampling, device and picture pick-up device |
| CN103716606A (en) * | 2013-12-30 | 2014-04-09 | 上海富瀚微电子有限公司 | Bayer domain image downsampling method and device and camera equipment |
| CN105847770A (en) * | 2015-01-15 | 2016-08-10 | 展讯通信（上海）有限公司 | Image processing chip and caching method for image data in chip |
| CN105847770B (en) * | 2015-01-15 | 2017-08-08 | 展讯通信（上海）有限公司 | The caching method of view data in picture processing chip and the chip |
| CN105611256B (en) * | 2015-12-28 | 2017-08-01 | 南京航空航天大学 | A kind of Bayer format interpolation method based on gradient and the display device based on FPGA |
| CN105611256A (en) * | 2015-12-28 | 2016-05-25 | 南京航空航天大学 | Bayer format image interpolation method based on gradients and display device based on FPGA |
| CN106412542A (en) * | 2016-02-03 | 2017-02-15 | 周彩章 | Image processing method and system |
| CN107016343A (en) * | 2017-03-06 | 2017-08-04 | 西安交通大学 | A kind of traffic lights method for quickly identifying based on Bel's format-pattern |
| CN110049337A (en) * | 2019-05-24 | 2019-07-23 | 安翰科技(武汉)股份有限公司 | The compression processing method and system of capsule endoscope bayer image |
| CN110049337B (en) * | 2019-05-24 | 2021-05-25 | 安翰科技(武汉)股份有限公司 | Compression processing method and system for capsule endoscope bayer image |
| WO2021007742A1 (en) * | 2019-07-15 | 2021-01-21 | 上海极清慧视科技有限公司 | Compression method for obtaining video file, decompression method, system, and storage medium |
| WO2022141515A1 (en) * | 2020-12-31 | 2022-07-07 | 深圳市大疆创新科技有限公司 | Video encoding method and device and video decoding method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102457722B (en) | 2014-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102457722B (en) | Processing method and device for Bayer image | |
| JP4769039B2 (en) | Digital signal encoding and decoding apparatus and method | |
| JP4687216B2 (en) | Image signal processing apparatus, camera system, and image signal processing method | |
| KR0183688B1 (en) | Image encoding method and device | |
| CN100592759C (en) | Image capturing apparatus, imaging circuit, and image capturing method | |
| JP4687215B2 (en) | Image signal processing apparatus, camera system, and image signal processing method | |
| JP4084826B2 (en) | Image compression device and method | |
| CN102356635A (en) | Video acquisition and processing systems | |
| CN107483850B (en) | low-delay high-definition digital image transmission method | |
| RU2008150330A (en) | COMPRESSING VIDEO DATA WITHOUT VISIBLE LOSS | |
| US9019399B2 (en) | Two-dimensional signal encoding device | |
| US8823832B2 (en) | Imaging apparatus | |
| KR100504649B1 (en) | Device for compressing image data | |
| JP2011223519A (en) | Data converter | |
| WO2009087783A1 (en) | Data generator for coding, method of generating data for coding, decoder and decoding method | |
| US8224103B2 (en) | Image encoding method and device, image decoding method and device, and imaging device | |
| US20050212933A1 (en) | Luminance and color difference signal generation apparatus, image data compression apparatus, and image processing system | |
| JP2010098352A (en) | Image information encoder | |
| JP2011015347A (en) | Apparatus and method for processing image, program and recording medium | |
| KR100664550B1 (en) | Method for transferring encoded data and image pickup device performing the method | |
| US20050259879A1 (en) | Video decoder with deblocker within decoding loop | |
| Lin et al. | Adjusted 4: 2: 2 chroma subsampling strategy for compressing mosaic videos with arbitrary RGB color filter arrays in HEVC | |
| CN101742279B (en) | Compression ratio adjustment method and electronic device of JPEG (Joint Photographic Experts Group) image | |
| US20040252894A1 (en) | Image compression apparatus and image processing system | |
| JP2009124278A (en) | Imaging device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |