Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T3/00—Geometric image transformations in the plane of the image
  • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
  • G06T3/4015—Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T3/00—Geometric image transformations in the plane of the image
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
  • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
  • H04N25/46—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by combining or binning pixels
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10024—Color image
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2209/00—Details of colour television systems
  • H04N2209/04—Picture signal generators
  • H04N2209/041—Picture signal generators using solid-state devices
  • H04N2209/042—Picture signal generators using solid-state devices having a single pick-up sensor
  • H04N2209/045—Picture signal generators using solid-state devices having a single pick-up sensor using mosaic colour filter
  • H04N2209/046—Colour interpolation to calculate the missing colour values
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N9/00—Details of colour television systems
  • H04N9/64—Circuits for processing colour signals

Definitions

• the present invention relates to the field of image processing, and in particular, to a method and apparatus for processing a image based on a bayer format.
• bayer which is a red and green filter interval.
• the blue-green filter is alternately arranged in an alternating cycle, so the bayer format is the most common CCD, CMOS image sensor data output format.
• the image pixels in the bayer format generally have four arrangement modes, as shown in Figure 1, taking the upper left picture as an example: Odd scan line output RGRG ... even scan line output GBGB — According to the large-area coloring characteristics of the human eye's response to the color bandwidth, it is not necessary to output three colors at the same time for each pixel. Therefore, when data is sampled, the 1st, 2nd, 3rd, 4th, ... pixels of the odd scan lines respectively sample and output R, G, R, G, ... data; the 1st, 2, 3 of the even scan lines , 4, ... pixels sample and output G, B, G, B, ... data. In actual processing, the R, G, and B signals of each pixel are composed of one color signal output by the pixel itself and other color signals output by adjacent pixels. This sampling method can reduce the sampling frequency by more than 60% while substantially reducing the image quality.
• the bayer format can save image transmission bandwidth and storage space than the full color format, in some cases, the bayer format image is still processed to further reduce the image size.
• binning processing is one of the images directly directed to the bayer format. A method of processing pixel binning.
• Binning is an image reading method that adds adjacent pixels and then averages them. As a new pixel output. Binning can be done in the horizontal direction, in the vertical direction, or simultaneously in both directions. This can reduce the image size and reduce the amount of image data. In addition, since the value of one new pixel is represented by the average value of a plurality of pixels, the image noise is also suppressed to some extent.
• Figure 2 shows the principle of binning in the case of l/2biiming, where R, G, and B represent the red, green, and blue pixels in the bayer format image, respectively.
• the upper and left numbers represent the abscissa and ordinate, respectively, to facilitate reference to the pixels.
• the left part of Figure 2 is the original bayer image of the image sensor output, and the right side is the bayer image with horizontal and vertical l/2binning.
• the red pixel with the coordinates of (0,0) after Binning is equal to the four pixels of coordinates (0,0), (0,2), (2,0) and (2,2) in the original image.
• Figure 3 shows a schematic diagram of only the horizontal 1/2 binning operation, where each pixel on the right is the mean of two pixels horizontally adjacent to the original image on the left, ie the value of the first red pixel after binning [ R ( 0,0 ) +R ( 2,0 ) ]/2.
• the image after binning has a size in the horizontal direction of 1/2 of the original image, and remains unchanged in the vertical direction.
• Figure 4 shows a schematic diagram of only the vertical direction l/2binning operation, where each pixel on the right is the mean of two pixels vertically adjacent to the original image on the left side, that is, the value of the first red pixel after binning Is [R ( 0,0 ) +R ( 0,2 ) ]/2.
• the size of the image after binning in the vertical direction is the original image. 1/2, the horizontal direction remains unchanged.
• the other ratios of binning and l/2binning are the same, except that there are more pixels involved in the average calculation. Specifically, binning in a direction of 1/n will have n pixels participating in the average calculation, and the size of the image after binning becomes the original l/n.
• the traditional binning process uses the averaging method of the cartridge, which causes false details after color interpolation, thereby seriously degrading the image quality.
• the technical solution of the present invention provides a processing method based on a bayer format image, including: performing binning processing in a row direction and/or binning processing in a column direction on an image to be processed, and arranging pixels in the processed image.
• the mode is the same as the arrangement mode of the pixels in the image to be processed;
• the binning process includes: determining a position of the output pixel; selecting, from the image to be processed, a plurality of pixels having the same color type as the output pixel A weighted average is obtained to obtain a value of the output pixel, wherein a position of the selected plurality of pixels and a position of the output pixel satisfy a weighted average relationship.
• the image to be processed in the binning process in the row direction, is an original image; in the binning process in the column direction, the image to be processed is a binning processed image in a row direction.
• the image to be processed in the binning process in the column direction, is an original image; in the binning process in the row direction, the image to be processed is a binning processed image in a column direction.
• the positions of the output pixels in the image to be processed are equally spaced.
• the binning process is 1/n binning processing, and the output pixel has a position interval of n in the image to be processed.
• the binning process is 1/n binning processing, and the total number of selected pixels is 2n-1.
• the binning process is 1/n binning processing, and the total number of selected pixels is n.
• the technical solution of the present invention further provides a processing method based on a bayer format image, comprising: a row processing unit and/or a column processing unit, wherein the row processing unit performs binning in a row direction for an image to be processed.
• the column processing unit performs a binning process in a column direction for the image to be processed, and the arrangement mode of the pixels in the processed image is the same as the arrangement mode of the pixels in the image to be processed; each processing unit includes:
• a determining unit configured to determine a position of the output pixel
• a weighting unit configured to select, from the image to be processed, a plurality of pixels having the same color type as the color type of the output pixel, and perform weighted averaging to obtain a value of the output pixel, wherein the selected pixels are located at a position The position of the output pixel satisfies a weighted average relationship.
• the technical solution of the present invention considers the position of the output pixel in the image to be processed in the binning process, and selects a number of pixels from the image to be processed based on the position of the output pixel to perform weighted averaging to obtain The value of the output pixel. Therefore, the technical solution of the present invention can not only achieve the effect of reducing the image size, reducing the amount of image data, and reducing the image noise of the conventional binning, but also enabling the binning processed image to not introduce any false details after color interpolation, thereby ensuring the image. The quality of the treatment.
• Figure 1 is a schematic diagram of four arrangement patterns of bayer format images
• FIG. 2 is a schematic diagram of a conventional binning method for simultaneously performing 1/2 binning processing in the row direction and the column direction;
• Figure 3 is a schematic diagram of the conventional binning method for performing only 1/2 binning processing in the row direction
• Figure 4 is a schematic diagram of the conventional binning method for only 1/2 binning processing in the column direction
• FIG. 5 is a conventional binning method that only performs row direction 1/2 binning processing of a row of pixels in an image.
• FIG. 6 is a schematic flowchart of a method for processing a image based on a bayer format according to an embodiment of the present invention.
• FIG. 7 is a schematic diagram of a method for processing a row-based binning process in a bayer format image processing method according to an embodiment of the present invention; Bayer image pixel and binning processed image pixel schematic diagram;
• FIG. 8 is a bayer format image processing method according to an embodiment of the present invention, when column direction binning processing is performed, a column of bayer image pixels to be processed and binning processing Image pixel diagram;
• Fig. 9 is a view showing the configuration of a processing apparatus based on a bayer format image according to an embodiment of the present invention.
• FIG. 5 only shows a row of pixels in the horizontal direction.
• the first row of pixels of Figure 5 is the original pixel before binning, and its position in the original bayer image is numbered above.
• the pixel after Binning is given in the second line, and its position in the bayer image after binning is indicated by a number below.
• the pixel generation relationship before and after Binning is indicated by an arrow.
• the weight of each pixel participating in the averaging is the same, for example, the first pixel R, (0) generated by binning in Figure 5, whose value is equal to [R(0)+R(2)]/ 2.
• the pixels obtained after averaging should be in the middle of the two, and equidistant from the two, that is, at the position of the original bayer image coordinates (1).
• the position of other pixels after binning in the original image can be obtained.
• the spacing between the prime and the pixels is not equal. For example, the green pixel G after binning, (l), and the left red pixel R, (0) are much smaller than the distance between the red pixel R and the right pixel (2). .
• the same phenomenon occurs when binning is performed in the vertical direction.
• each pixel of the bayer format image has only one color type, to obtain a color image, it is necessary to color interpolate the image in the bayer format, and use the pixels adjacent to each pixel to calculate the other two types missing from the pixel.
• Color type In the interpolation process, the distance between the pixels and the pixels must be equal, so that the weights can be correctly applied to the adjacent pixels for interpolation calculation.
• the output of the traditional binning operation causes the distance between the bayer image pixels to be no longer equal, so that the color interpolation cannot correctly apply weights to the adjacent pixels, thereby forming false details and seriously degrading the image quality.
• the technical solution of the present invention provides a processing method and apparatus based on a bayer format image, and the technical solution of the invention can not only reduce the image size, reduce the amount of image data, reduce image noise, but also ensure the image after re-color interpolation. the quality of.
• the processing method based on the bayer format image includes: performing binning processing on the image to be processed, and the arrangement pattern of pixels in the processed image is the same as the arrangement pattern of pixels in the image to be processed; the binning processing includes:
• Determining a position of the output pixel selecting, from the image to be processed, a plurality of pixels of the same color type as the output pixel, performing weighted averaging to obtain a value of the output pixel, wherein the selected number of pixels are located The position with the output pixel satisfies a weighted average relationship.
• the binning process can be performed only in the row direction of the image to be processed, wherein the image to be processed is the original image, that is, the image output by the image sensor.
• the binning process can also be performed only in the column direction of the image to be processed, wherein The image is the original image.
• the binning process may also be performed in the row direction of the image to be processed, and then in the column direction to be processed, wherein in the binning process in the row direction, the image to be processed is the original image, and the binning in the column direction In the process, the image to be processed is a binning processed image in the row direction.
• the binning process may also be performed in the column direction of the image to be processed, and then in the row direction to be processed, wherein in the binning process in the column direction, the image to be processed is the original image, and the binning in the row direction In the process, the image to be processed is a binning processed image in the column direction.
• the processing method based on the bayer format image of the present embodiment first performs step S11, and determines whether to perform binning processing in the row direction for the image to be processed. If the determination result is yes, step S12 is performed to determine the output pixel. Position, in this embodiment, the position of the output pixel specifically refers to the position of the output pixel in the image to be processed; if the determination result is no, step S14 is performed, and the image to be processed is determined whether to perform binning processing in the column direction. And the arrangement pattern of the pixels in the image processed in the above step is the same as the arrangement pattern of the pixels in the image to be processed. In other embodiments, it may be determined whether or not the binning process in the column direction is performed, and it is determined whether or not the binning process in the row direction is performed.
• the binning process in the row direction and the binning process in the column direction are independent of each other.
• the binning process in the row direction may be performed first, or the binning process in the column direction may be performed first, or the binning process in the row or column direction may be performed only.
• the binning process in the row direction is performed first, and the binning process in the column direction is performed first, and the binning process is 1/2 binning processing as an example.
• step S12 is performed to determine the position of the output pixel.
• the binning process in the row direction is performed first, and the binning process in the column direction is performed as an example. Therefore, the position of the output pixel described in step S12 is determined according to the position of the pixel in the original image, and then the position of the output pixel is determined in step S15, which is the pixel in the image obtained by performing binning processing on the original image in the row direction.
• the position determining, that is, the binning processing in the column direction of step S15 is to perform further processing based on the binning processing of the image line direction in steps S11 to S14.
• the arrangement pattern of the output pixels is the same as the arrangement pattern of the pixels in the image to be processed, and the positions of the output pixels in the image to be processed are equally spaced.
• the position interval is equal to two. Referring to FIG. 7 as an example of 1/2 binning in the horizontal direction, a row of bayer image pixels to be processed and image pixels after binning processing thereof are given, and the arrangement pattern of the output pixels is RGRG ...
• the arrangement pattern of the image pixels is the same, in order to ensure that the arrangement pattern of the pixels in the output pixel and the image to be processed is the same, the color type of the first pixel of each line after the binning processing in the row direction and the first line of the line on the image to be processed
• the pixel color types are the same; the position interval between the output pixels is 2, for example, the output pixel R, (0) is in the position of the image to be processed, G(1), and the output pixel G, (1)
• the position in the image to be processed is the position of G ( 3 ), and the position interval between the two is 2.
• the positional interval between the output pixels is n.
• step S 13 is performed, and a plurality of pixels whose color type is the same as the color type of the output pixel are selected from the image to be processed, and weighted average is performed to obtain a value of the output pixel, where the selected pixels are located.
• the position with the output pixel satisfies a weighted average relationship, and the weighted weight in the weighted average decreases as the distance between the selected pixel and the output pixel increases.
• the output image is selected from the image to be processed.
• the pixel G of the coordinate (7) and the pixel G of the output pixel coordinate (3) have the same position and the color type is the same, then select and select from the image to be processed.
• the selected adjacent pixels of the same color are directly adjacent two pixels of the same color, because of the 1/2 binning process, G ( 7 ) plus G ( 5 ) adjacent to G ( 7 ) and G ( 9 ) is sufficient to express the value of G, (3), so no more adjacent pixels of the same color type are selected, and there is no fixed limit on the number of adjacent pixels to be selected, so that the pixel after binning processing can be reasonably calculated.
• the preferred number of the adjacent pixels is 2 ⁇ -1, and of course, more than 2n-1 adjacent pixels may be selected.
• the distance between the pixels G (5) and G (9) and the output pixel G, (3) in the selected image to be processed is farther than the distance between G (7) and G, (3), so the two
• the weight is set to 1
• the weight of G (7) is larger than G (5) and G (9)
• the weight is set to 2
• the weight of the pixels participating in the weighted average has no fixed value, and only the value satisfying the weight is processed according to the distance binning.
• the weight is smaller and smaller, and in addition, the same weight can be set for two pixels symmetric about the position of the output pixel.
• G, ( 3 ) [G ( 5 ) + 2*G (7) + G (9) ] / 4, where G (5) and G (9)
• the weights are the same, and the mean of the two is located between G (5) and G (9).
• This position is not only coincident with the position of G (7) in the image to be processed, but also the final output pixel G, (3). position. Since the position of G (7) and the output pixel coincide, it can be regarded as the closest distance.
• the weight is larger than G (5) and G (9). In other embodiments, the weight settings will vary accordingly due to the different neighboring pixels selected.
• the color type is the same from the image to be processed, and
• the output pixel is adjacent to a plurality of pixels.
• the pixel G of the coordinate (5) and the pixel R of the output pixel coordinate (2) are different in color type, and the output pixel is selected from the image to be processed.
• the selected adjacent color pixels are two color types directly adjacent to G (5)
• the same pixel because it is 1/2 binning processing, R (4) plus R (6) 2 pixel values are enough to express the value of R, (2), so no more adjacent pixels of the same color type are selected.
• the preferred number of the adjacent pixels is n, and of course, more than n adjacent pixels may be selected.
• the weight setting varies according to the selected neighboring pixels. For details, refer to the setting method of the weight in the first case described above, and the description will not be repeated again. According to the above steps S12 and S13, the values of the output pixels of the respective rows are sequentially calculated. It should be noted that, in the binning process in the horizontal direction (row direction), the number of pixels participating in the weighted average may be insufficient in the last few columns of the image.
• step S14 is performed to determine whether to perform binning processing in the column direction. If the determination result is yes, step S15 is performed to determine the position of the output pixel; if the determination result is no, the image processing procedure is ended.
• Step S15 can be combined with FIG. 8 and refer to the implementation of step S12 above. The difference is that step S15 determines that the image to be processed in the position distribution of the image to be processed is the binning processed image in the row direction, that is, The binning processing in the column direction of step S15 is performed by performing binning processing in the row direction on the original image in steps S12 to S13.
• Step S12 determines the position distribution of the output pixels in the row direction of the image to be processed
• step S15 determines the position distribution of the output pixels in the column direction of the image to be processed.
• step S16 is performed to select a plurality of pixels whose color type is the same as the color type of the output pixel from the image to be processed, and perform weighted averaging to obtain the value of the output pixel.
• the position of the selected plurality of pixels and the position of the output pixel satisfy a weighted average relationship, and the weight of the weighted average weight decreases as the distance between the selected pixel and the output pixel increases.
• the bayer format image processing apparatus of this embodiment includes: a selection unit A1, a row processing unit A2, and a column processing unit A3. a selection unit A1 for determining whether to perform binnig processing in the row direction and binning processing in the column direction; the row processing unit A2 performing binning processing in the row direction for the image to be processed; the column processing unit A3 is for processing The image performs binning processing in the column direction.
• the row processing unit A2 includes: a determining unit A21, configured to determine a position of the output pixel in a row direction in the image to be processed, the output pixel being a pixel in the processed image, and an arrangement pattern of pixels in the processed image and The arrangement mode of the pixels in the image to be processed is the same; the weighting unit A22 is connected to the determining unit A21, and is configured to select, from the row direction of the image to be processed, a plurality of pixels of the same color type as the output pixel. A weighted average is obtained to obtain a value of an output pixel in a row direction, wherein a position of the selected number of pixels and a position of the output pixel satisfy a weighted average relationship.
• the column processing unit A3 includes: a determining unit A31, configured to determine a position of the output pixel in a column direction in the image to be processed, the output pixel being a pixel in the processed image, and an arrangement of pixels in the processed image
• the mode is the same as the arrangement mode of the pixels in the image to be processed
• the weighting unit A32 is connected to the determining unit A31, and is configured to select, from the column direction of the image to be processed, a color type that is the same as the color type of the output pixel. Pixels are weighted averaged to obtain values of output pixels in the column direction, wherein the positions of the selected pixels and the position of the output pixels satisfy a weighted average relationship.
• the image data in the bayer format is acquired by the obtaining unit (not shown), and then the selection unit A1 performs the determination and selection. If the binning processing in the row direction needs to be performed, the row processing unit A2 is selected to perform the row direction. Binning processing; if it is necessary to perform binning processing in the column direction, the column processing unit A3 is selected to perform binning processing in the column direction. Specific binning processing can refer to The above processing method based on the bayer format image is not described here.
• the processing device based on the bayer format image may also include only the row processing unit, that is, performing binning processing only in the row direction on the original image; or only including the column processing unit, that is, performing column direction only on the original image. Binning processing.
• the Bayer format image processing method and apparatus provided by the technical solution of the present invention reduce the image size, reduce the data amount of the image, and reduce the image noise by using a plurality of pixel weighted average values, and the processing is completed.
• the bayer format of the image to be processed is not changed, and the technical solution can ensure the quality of the image processing. No false details are introduced after the color interpolation.
• the technical solution of the present invention calculates the cartridge and is regular, which is beneficial for high performance occasions. Hardware implementation.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Color Television Image Signal Generators (AREA)

Priority Applications (5)

Applications Claiming Priority (1)

Publications (1)

Family

ID=50067363

Family Applications (1)

Country Status (5)

Cited By (3)

Families Citing this family (14)

Citations (3)

Family Cites Families (5)

• 2012
  • 2012-08-07 US US14/123,227 patent/US9262805B2/en active Active
  • 2012-08-07 CN CN201280004313.3A patent/CN103733220B/zh active Active
  • 2012-08-07 WO PCT/CN2012/079763 patent/WO2014022965A1/zh active Application Filing
  • 2012-08-07 IN IN2138MUN2013 patent/IN2013MN02138A/en unknown
  • 2012-08-07 EP EP12877603.6A patent/EP2728545B1/en active Active

Patent Citations (3)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events