CN104253985A - Demosaicking method and image processor used for demosaicking - Google Patents
Demosaicking method and image processor used for demosaicking Download PDFInfo
- Publication number
- CN104253985A CN104253985A CN201310254727.8A CN201310254727A CN104253985A CN 104253985 A CN104253985 A CN 104253985A CN 201310254727 A CN201310254727 A CN 201310254727A CN 104253985 A CN104253985 A CN 104253985A
- Authority
- CN
- China
- Prior art keywords
- pixel
- interpolation
- direction interpolation
- type selecting
- circle type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Processing (AREA)
Abstract
The invention relates to a demosaicking method. The demosaicking method comprises steps that: a selection type are set, and the selection type includes target pixels and at least one circumferential pixel near the target pixels; sampling multiple sample pixels of at least one sample image is carried out according to the selection type, and corresponding relations between the at least one circumferential pixel of the sample image and the target pixels in the direction of interpolation are recorded for each sampling; and result statistics of multiple samplings is carried out to train an interpolation direction learning model.
Description
Technical field
The present invention relates to image processing technique, relate more specifically to image solution mosaic technology.
Background technology
Separate mosaic (demosaicking), being also called colour reconstruction (color reconstruction), is a kind of image processing technique initial data (raw data) acquired after color filter being reconstructed into full-color image (full color image).In initial data, each pixel only has solid color, is generally red R, one of green G or blue these three primary colors of B, and after solution mosaic, each pixel can have redgreenblue simultaneously.
The process of this colour reconstruction must pass through interpolative operation.With regard to each pixel, in it, direction interpolation (direction of interpolation) can be vertical direction, horizontal direction or other various direction.For example, when interior direction interpolation is vertical, can select above certain pixel and the pixel of below to carry out interpolation calculating; When interior direction interpolation is level, can select on the left of certain pixel and the pixel on right side to carry out interpolation calculating.But in some special image, the interior direction interpolation of mistake can produce the image of distortion.Such as, when carrying out interpolation to the marginate image of tool and calculating, if the interior direction interpolation selected and edge move towards, inconsistent (such as edge moves towards vertical, interior direction interpolation is level), then there is the lines (zipper artifacts) as the similar slide fastener of part after colour reconstruction possibly, cause the generation of scrambled image.
Therefore, a kind of solution mosaic method that effectively can reduce above-mentioned image distortion is needed.
Summary of the invention
The invention provides a kind of solution mosaic method.Described solution mosaic method comprises the following steps: setting circle type selecting state, wherein said circle type selecting state coverage goal pixel and the neighboring pixel that at least one is close to described object pixel; Sample respectively with multiple sampled pixels of described circle type selecting state at least one sample image, to note down in each sub-sampling, the corresponding relation between the described neighboring pixel of described sample image and described object pixel on interior direction interpolation; And the result that statistics repeatedly samples is to train interior direction interpolation learning model.
The present invention separately provides a kind of image processor for separating mosaic.Described image processor comprises: interior direction interpolation training unit, for setting circle type selecting state, and wherein said circle type selecting state coverage goal pixel and the neighboring pixel that at least one is close to described object pixel; Sample respectively with multiple sampled pixels of described circle type selecting state at least one sample image, to note down in each sub-sampling, the corresponding relation between the described neighboring pixel of described sample image and described object pixel on interior direction interpolation; And add up the training result repeatedly sampled; And interior direction interpolation learning model, be coupled to described interior direction interpolation training unit, for the training result repeatedly sampled described in storing.
Accompanying drawing explanation
Fig. 1 is the solution mosaic method flow diagram according to one embodiment of the invention.
Fig. 2 A is image initial data schematic diagram acquired after performing color filter 110 with Bayer filter in one embodiment.
Fig. 2 B is the embodiment simplified according to the present invention one adds " circle type selecting state " schematic diagram in the initial data of Fig. 2 A.
Fig. 2 C is the partial pixel schematic diagram of a sample image.
Fig. 3 is the schematic diagram adding square " circle type selecting state " according to the present invention one preferred embodiment in the initial data of Fig. 2 A.
Fig. 4 is for separating the image processor of mosaic method in one embodiment of the invention.
[symbol description]
100 ~ separate mosaic method;
S110 ~ S140; S1402 ~ S1406 ~ step;
P0 ~ P14 ~ pixel;
R ~ redness;
G ~ green;
B ~ blueness;
400 ~ image processor;
410 ~ interior direction interpolation training unit;
420 ~ interior direction interpolation learning model;
430 ~ interpolation direction calculating unit.
Embodiment
Hereafter for introducing most preferred embodiment of the present invention.Each embodiment is for illustration of principle of the present invention, but non-for limiting the present invention.Scope of the present invention is when being as the criterion with claims.
Separate mosaic method
The invention provides a kind of solution mosaic method newly.Fig. 1 is the solution mosaic method flow diagram according to one embodiment of the invention.In solution mosaic method 100 of the present invention, key step S120 is used for selecting or calculate the interior direction interpolation of each pixel in specific image.As shown in Figure 1, after it is executed in color filter (color filtering) 110 and before interpolation calculating 130.Fig. 2 A is image initial data schematic diagram acquired after performing color filter 110 with Bayer filter in one embodiment.Owing to adopting general Bayer color filters, the pixel quantity of this embodiment Green (G) is the twice of red (R) and blue (B).As described in the prior art, in this initial data, each pixel only has solid color, and in order to rebuild the rgb value of each pixel, the interpolation that must perform step S130 calculates.But the drawback of the image distortion produced in order to avoid adopting single interior direction interpolation (such as: horizontal or vertical) simply, step S120 of the present invention can select optimal interior direction interpolation for each specific pixel in described specific image respectively.
In detail, step S120 of the present invention is performed by following manner: " in the described specific pixel that the data provided to enclose type selecting state and interior direction interpolation learning model calculate described specific image direction interpolation ".Fig. 2 B is the schematic diagram adding " circle type selecting state " in the embodiment of the present invention one simplification at the partial pixel of Fig. 2 A initial data.As shown in the figure, in the embodiment that this simplifies, so-called " circle type selecting state " presents triangular form substantially, and its scope covered comprises " object pixel P4 " for direction interpolation in calculating, and " neighboring pixel P0 ~ P2 " adjacent with described object pixel.But to calculate " red value " (R value) of object pixel P4, in a preferred embodiment, the non-pixel P1 for redness can be ignored (that is, neighboring pixel is P0 and P2) by so-called neighboring pixel.
Cardinal principle of the present invention is: certain object pixel and its neighboring pixel have the corresponding relation of certain degree on interior direction interpolation, and described corresponding relation is by obtaining a large amount of image training.Wherein, via training a large amount of corresponding relations produced can be stored in aforesaid " interior direction interpolation learning model ", for the use calculating direction interpolation in specific pixel backward.In other words, with regard to the embodiment of Fig. 2 B, when learning direction interpolation in neighboring pixel P0 and P2, then the due interior direction interpolation of object pixel P4 can be learnt by described interior direction interpolation learning model.Therefore, the present invention still comprises this step of direction interpolation learning model 140 in the foundation shown in Fig. 1, and this step will in hereafter describing in detail.
The step S140 that the present invention sets up interior direction interpolation learning model also comprises: in step S1402, and setting is circle type selecting state as the aforementioned; In step S1404, sample respectively with multiple sampled pixels of described circle type selecting state at least one sample image, with record in each sub-sampling, the corresponding relation between the described neighboring pixel of described sample image and described object pixel on interior direction interpolation; And in step S1406, the result that statistics repeatedly samples is to train direction interpolation learning model in.It should be noted that, although " circle type selecting state " object is herein in " sampling ", different with aforementioned object-" in calculating direction interpolation " in the step s 120, but both " circle type selecting state " must be of similar shape (being such as similarly the triangle shown in Fig. 2 B), cover corresponding object pixel and with neighboring pixel position, begin, in computing, there is meaning.In addition, have to be noted that all pixels have had the interior direction interpolation determined all in " sample image " of step S1404, therefore " sample image " is different from aforementioned waiting to calculate interior direction interpolation " specific image " herein.Those skilled in the art can adopt a large amount of and representative pattern as described " sample image ", have direction interpolation learning model in statistical significance in order to training.
Fig. 2 C is the partial pixel schematic diagram of a sample image.In this sample image, the interior direction interpolation of relative target pixel P4, neighboring pixel P0 is " vertically " (referred to as " V "), and the interior direction interpolation of neighboring pixel P2 is " level " (referred to as " H ").In this sub-sampling, find that the interior direction interpolation of object pixel P4 is " H ", then this corresponding relation can be recorded among " interior direction interpolation learning model ", and form can be as shown in the table when the interior direction interpolation of neighboring pixel P0 and P2 is (V, H):
Then, in the same way above-mentioned sampling process is repeated to pixels all in described sample image, and in step S1406, add up the result repeatedly sampled, to obtain the conditional probability in various situation.The possibility of result of statistics is as shown in the table:
When interior direction interpolation learning model is set up complete, namely by direction interpolation in each specific pixel in the described specific image of abovementioned steps S120 calculating.For example, when neighboring pixel P0 and the P2 of specific image under circle form slection state has interior direction interpolation (H, V) time, then can learn that the interior direction interpolation of object pixel P4 be that the probability of vertically (V) and level (H) is respectively 1/4 and 3/4 by the data (as shown above) of direction interpolation learning model in inquiry.Then, in a better simply embodiment, the interior direction interpolation of object pixel P4 can be set as conditional probability the higher person, is this means, set as " level " (H); Or in another more accurate embodiment, first calculate object pixel P4 at conditional probability that is vertical and horizontal interpolation direction, then with weights 1/4 and 3/4, its weight summation is calculated respectively to both, using as the last interpolate value of described object pixel P4.Then, this step S120 of the present invention can repeat until the interior direction interpolation of all pixels of described specific image obtains all.It should be noted that, although only explain with direction interpolation in two kinds (vertical and level) in the present embodiment, but, in other embodiments, those skilled in the art can stipulate various interior direction interpolation according to spirit of the present invention, the upper right tilted direction etc. such as combined by upper right 2.
The method that the present invention separates mosaic under a simple embodiment has been described in detail in detail above.Although have to be noted that previous embodiment adopts leg-of-mutton " circle type selecting state ", but this embodiment is only simplified illustration, in other embodiments, the shape of " circle type selecting state " need not as limit.For example, " circle type selecting state " of the present invention can have less scope, such as its neighboring pixel is defined as the upper left single pixel of object pixel (the pixel P0 in such as Fig. 2 B), just this way is usual less with practical value with the interior direction interpolation learning model obtained.In a preferred embodiment, " circle type selecting state " can present square substantially.Due to the pixel that these embodiment coverage goal pixel periphery all directions are contiguous, therefore, the interior direction interpolation learning model trained has suitable practical value.
Fig. 3 is for adding the schematic diagram of square " circle type selecting state " in Fig. 2 A original data portion pixel according to the present invention one preferred embodiment.In this embodiment, object pixel is P4, its neighboring pixel then (calculates with the interpolation of R value for P0, P2, P6 and P8, slightly can remove pixel P1, P3, P5 and P7 of G value), and the described neighboring pixel that waits can be further divided into " preceding pixel " P0, the P2 be arranged in before described object pixel P4, and " rear pixel " P6 and P8 after being arranged in described object pixel P4.It should be noted that under this circle type selecting state, when calculating the interior direction interpolation of object pixel P4, must " rear pixel " P6 and P8 not yet calculating interior direction interpolation be listed in consideration.Therefore, the present invention can in the described specific pixel P4 of calculating before direction interpolation, the first interior direction interpolation of tentative described rear pixel P6 and P8.For example, the method of the interior direction interpolation of tentative described rear pixel (such as P6) comprising: by the numerical value difference of more described rear pixel P6 on the direction that each may carry out interpolation between neighbor, and fix tentatively having the minimum direction of numerical value difference as direction interpolation in described rear pixel.In detail, if pixel P6 is 50 with the R value difference of pixel P8 in the horizontal direction, and when being 60 with the R value difference of pixel P12 in the vertical direction, then the interior direction interpolation of described rear pixel P6 can be fixed tentatively for " level ".Outside, when take turns to described after pixel P6 and P8 be regarded as " specific pixel ", and when in it, direction interpolation also obtains according to aforementioned manner calculating, then, in one embodiment, separately can recalculate direction interpolation in described specific pixel P4 with direction interpolation in the pixel P6 newly obtained, to replace the value previously calculated; Or, in another embodiment, be averaged recalculating direction interpolation in the pixel P4 that obtains with the value previously calculated.Must recognize, this step recalculated need not adopt and identical above " enclosing type selecting state ", and, because its execution mode is numerous, therefore repeat no more other embodiment herein.
In other embodiments; aforesaid " circle type selecting state " can expand further or change to other shape; such as rectangle, trapezoidal etc.; this is all in the scope of the present invention's wish protection, and those skilled in the art can be modified as various sampling according to spirit of the present invention, set up the method for direction interpolation in interior direction interpolation learning model and calculating pixel.
For separating the image processor of mosaic
Adopt multiple embodiment that the method for the various solution mosaic of the present invention is described in detail in detail above.In addition, the present invention separately provides a kind of image processor to perform the solution mosaic method of previous embodiment.Fig. 4 is for separating the image processor of mosaic method in one embodiment of the invention.Image processor 400 of the present invention has interior direction interpolation training unit 410, interior direction interpolation learning model 420, and interpolation direction calculating unit 430.In one embodiment, image processor 400 of the present invention also comprise be executed in interpolation direction calculating before, for the color filter element of color filter, and after being executed in interpolation direction calculating, for calculating the interpolation computing unit (not shown) of color interpolation value.But, the target that color filter element and interpolation computing unit non-invention are mainly discussed, therefore repeat no more.
Interior direction interpolation training unit 410 of the present invention, for performing abovementioned steps S1402, S1404 and S1406, comprise: setting is enclosed type selecting state, sampled respectively with multiple sampled pixels of described circle type selecting state at least one sample image, with record in each sub-sampling, the corresponding relation between the described neighboring pixel of described sample image and described object pixel on interior direction interpolation and add up the result repeatedly sampled.Interior direction interpolation learning model 420 of the present invention is coupled to described interior direction interpolation training unit 410, the result of above-mentioned sampling can be trained.Interpolation direction calculating unit 430 of the present invention is coupled to described interior direction interpolation learning model 420, for performing abovementioned steps S120, that is, direction interpolation in the specific pixel that the data provided with described circle type selecting state and described interior direction interpolation learning model calculate specific image.Wherein, " circle type selecting state " can as mentioned before, be the triangle shown in Fig. 2 B, the square shown in Fig. 3, or other various shape, such as trapezoidal, rectangle etc.
It should be noted that, when the described neighboring pixel in the circle type selecting state that an embodiment adopts comprises the rear pixel be arranged in after described object pixel (the circle type selecting state such as shown in Fig. 4), then described interpolation direction calculating unit 430 is also for before direction interpolation in the described specific pixel calculating described specific image, the first interior direction interpolation of tentative described rear pixel, tentative method comprises: the numerical value difference on more described rear pixel may carry out interpolation direction its each and between neighbor, and fix tentatively having the minimum direction of numerical value difference as direction interpolation in described rear pixel.In addition, in a preferred embodiment, described interpolation direction calculating unit 430 after also the interior direction interpolation of pixel is obtained by formal calculating in the rear, recalculates the interior direction interpolation of described specific pixel.Perform identical function because the present invention has for the image processor 400 separating mosaic method with aforementioned solution mosaic method 100 and reach identical effect, and the various embodiments of mosaic method 100 of the present invention have been described in detail in detail above, so place no longer repeats other related embodiment of image processor 400 of the present invention.
Though the present invention discloses as above with preferred embodiment; so it is not intended to limit scope of the present invention, any those skilled in the art, without departing from the spirit and scope of the present invention; when doing a little change and retouching, therefore protection scope of the present invention is as the criterion when limiting depending on claims.
Claims (12)
1. separate a mosaic method, comprise the following steps:
Setting circle type selecting state, wherein said circle type selecting state coverage goal pixel and the neighboring pixel that at least one is close to described object pixel;
Sample respectively with multiple sampled pixels of described circle type selecting state at least one sample image, to note down in each sub-sampling, the corresponding relation between the described neighboring pixel of described sample image and described object pixel on interior direction interpolation; And
The result that statistics repeatedly samples is to train interior direction interpolation learning model.
2. solution mosaic method according to claim 1, also comprises the following steps:
Direction interpolation in the specific pixel that the data provided with described circle type selecting state and described interior direction interpolation learning model calculate specific image.
3. solution mosaic method according to claim 2, the described neighboring pixel in wherein said circle type selecting state comprises:
Preceding pixel, before it is arranged in described object pixel; And/or
Rear pixel, after it is arranged in described object pixel.
4. solution mosaic method according to claim 3, wherein, when the described neighboring pixel in described circle type selecting state comprises described rear pixel, then in the described specific pixel calculating described specific image, before direction interpolation, first fix tentatively the interior direction interpolation of described rear pixel.
5. solution mosaic method according to claim 4, wherein the step of the interior direction interpolation of tentative described rear pixel comprises:
Numerical value difference on more described rear pixel may carry out interpolation direction its each between neighbor, and fix tentatively having the minimum direction of numerical value difference as direction interpolation in described rear pixel.
6. solution mosaic method according to claim 5, also comprises:
In the described rear pixel of calculating after direction interpolation, recalculate the interior direction interpolation of described specific pixel.
7., for separating an image processor for mosaic, comprising:
Interior direction interpolation training unit, for setting circle type selecting state, wherein said circle type selecting state coverage goal pixel and the neighboring pixel that at least one is close to described object pixel; Sample respectively with multiple sampled pixels of described circle type selecting state at least one sample image, to note down in each sub-sampling, the corresponding relation between the described neighboring pixel of described sample image and described object pixel on interior direction interpolation; And add up the result repeatedly sampled; And
Interior direction interpolation learning model, is coupled to described interior direction interpolation training unit, for the training result repeatedly sampled described in storing.
8. image processor according to claim 7, also comprises:
Interpolation direction calculating unit, is coupled to described interior direction interpolation learning model, direction interpolation in the specific pixel that the data for providing with described circle type selecting state and described interior direction interpolation learning model calculate specific image.
9. image processor according to claim 8, the described neighboring pixel in wherein said circle type selecting state comprises:
Preceding pixel, before it is arranged in described object pixel; And/or
Rear pixel, after it is arranged in described object pixel.
10. image processor according to claim 9, wherein, when the described neighboring pixel in described circle type selecting state comprises described rear pixel, then described interpolation direction calculating unit is also for before direction interpolation in the described specific pixel calculating described specific image, first the interior direction interpolation of tentative described rear pixel.
11. image processors according to claim 10, numerical value difference on the more described rear pixel of wherein said interpolation direction calculating unit may carry out interpolation direction its each between neighbor, and fix tentatively having the minimum direction of numerical value difference as direction interpolation in described rear pixel.
12. image processors according to claim 11, wherein said interpolation direction calculating unit after direction interpolation, recalculates the interior direction interpolation of described specific pixel in the described rear pixel of calculating.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310254727.8A CN104253985A (en) | 2013-06-25 | 2013-06-25 | Demosaicking method and image processor used for demosaicking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310254727.8A CN104253985A (en) | 2013-06-25 | 2013-06-25 | Demosaicking method and image processor used for demosaicking |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104253985A true CN104253985A (en) | 2014-12-31 |
Family
ID=52188467
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310254727.8A Pending CN104253985A (en) | 2013-06-25 | 2013-06-25 | Demosaicking method and image processor used for demosaicking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104253985A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080247671A1 (en) * | 2007-04-05 | 2008-10-09 | Fumihito Yasuma | Image Processing Device |
| TW200945257A (en) * | 2008-04-24 | 2009-11-01 | Univ Nat Taiwan Science Tech | Method for rebuilding color information of pixels |
| TW201016000A (en) * | 2008-10-08 | 2010-04-16 | Silicon Integrated Sys Corp | Apparatus and method for low angle interpolation |
| CN102769756A (en) * | 2011-05-02 | 2012-11-07 | 索尼公司 | Image processing device, image processing method, and program |
-
2013
- 2013-06-25 CN CN201310254727.8A patent/CN104253985A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080247671A1 (en) * | 2007-04-05 | 2008-10-09 | Fumihito Yasuma | Image Processing Device |
| TW200945257A (en) * | 2008-04-24 | 2009-11-01 | Univ Nat Taiwan Science Tech | Method for rebuilding color information of pixels |
| TW201016000A (en) * | 2008-10-08 | 2010-04-16 | Silicon Integrated Sys Corp | Apparatus and method for low angle interpolation |
| CN102769756A (en) * | 2011-05-02 | 2012-11-07 | 索尼公司 | Image processing device, image processing method, and program |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Losson et al. | Comparison of color demosaicing methods | |
| CN106504281B (en) | Image quality enhancing and filtering method applied to cmos image sensor | |
| CN102254301B (en) | Demosaicing method for CFA (color filter array) images based on edge-direction interpolation | |
| JP2008015946A5 (en) | ||
| CN102630019B (en) | Demosaicking method and device | |
| EP3547680A1 (en) | Image processing method and filter array | |
| CN103974044A (en) | Mosaic image processing method | |
| CN103595981B (en) | Based on the color filter array image demosaicing method of non-local low rank | |
| CN104732561B (en) | The interpolation method and device of a kind of image | |
| KR102224851B1 (en) | Image Processing Device and Image Processing System performing sub-pixel interpolation | |
| US20170053379A1 (en) | Demosaicing methods and apparatuses using the same | |
| Tsai et al. | Heterogeneity-projection hard-decision color interpolation using spectral-spatial correlation | |
| CN102893609B (en) | Image processing apparatus and control method for image processing apparatus | |
| KR20080106585A (en) | Method and arrangement for generating a color video signal | |
| CN101282486A (en) | Image processing device | |
| CN103747222B (en) | A kind of method that color artifact of image suppresses and device | |
| CN101998127A (en) | Signal processing device, imaging device, and signal processing method | |
| CN113454687A (en) | Image processing method, apparatus and system, computer readable storage medium | |
| Tsai et al. | A new edge-adaptive demosaicing algorithm for color filter arrays | |
| CN103530856B (en) | Method and system for removing salt and pepper noise from Bayer image | |
| CN107622477A (en) | A kind of RGBW images joint demosaicing and deblurring method | |
| US20150055861A1 (en) | Methods and Systems for Image Demosaicing | |
| KR100565429B1 (en) | Apparatus and method for reconstructing missing color values in a color filter array | |
| CN116503259B (en) | Mosaic interpolation method and system | |
| TWI751124B (en) | Method for processing signals arising from a colour image capture matrix, and corresponding sensor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20141231 |