CN102377911B - Image processing apparatus, image processing method, and camera module - Google Patents
Image processing apparatus, image processing method, and camera module Download PDFInfo
- Publication number
- CN102377911B CN102377911B CN201110210377.6A CN201110210377A CN102377911B CN 102377911 B CN102377911 B CN 102377911B CN 201110210377 A CN201110210377 A CN 201110210377A CN 102377911 B CN102377911 B CN 102377911B
- Authority
- CN
- China
- Prior art keywords
- mentioned
- edge
- marginal information
- correction
- signal level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/20—Circuitry for controlling amplitude response
- H04N5/205—Circuitry for controlling amplitude response for correcting amplitude versus frequency characteristic
- H04N5/208—Circuitry for controlling amplitude response for correcting amplitude versus frequency characteristic for compensating for attenuation of high frequency components, e.g. crispening, aperture distortion correction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
Abstract
In an image processing apparatus according to one embodiment, an edge correcting unit replaces a signal level of a target pixel with a maximum value of signal levels of a plurality of peripheral pixels when a sum of the signal level of the target pixel and the edge information is more than the maximum value, and the edge correcting unit replaces the signal level of the target pixel with a minimum value of the signal levels of the plurality of peripheral pixels when a sum of the signal level of the target pixel and the edge information is less than the minimum value.
Description
The application enjoys the priority of the Japanese patent application No. 2010-186517 filing an application on August 23rd, 2010, quotes in this application the full content of this Japan's patent application.
Technical field
Present embodiment relates generally to image processing apparatus, image processing method and video camera module.
Background technology
In recent years, for cost degradation and slimming, be mounted in video camera module in portable phone etc. and sometimes formed by less lens number or use MTF(modulation transfer function: modulation transfer function) situation of the lens that characteristic is lower.As the fuzzy method that reduces image by Digital Signal Processing in perception, known have for example use the edge of band pass filter (BPF) to emphasize to process.
In edge is emphasized to process, the edge being extracted by BPF (edge) composition is implemented to the processing of regulation, carry out addition (overshoot is additional) or subtraction (undershoot is additional) to marginal portion.In the case of by using MTF characteristic lower lens to obtain comprising the image of obtuse marginal portion, in the edge based on this image (edge) emphasizes to process, overshoot and undershoot expansion and the fuzzy and noise that brings become problem.
As the method addressing this problem, for example, have and use based on EDOF(extended depth of field: extended depth-of-field) situation of the fuzzy revising processed.In order to use EDOF to process, the digital processing that needs to use the special lens of processing for EDOF and can be suitable for this lens performance.
The use that EDOF processes causes that the versatility brought of special structure that EDOF processes declines, circuit scale expands because being used for, the capacity increase of memory etc., and the problem of chip size and power consumption aspect becomes problem.
Summary of the invention
The object of this invention is to provide a kind of obtuse image processing apparatus, image processing method and the video camera module that can suitably revise by simple structure the marginal portion comprising in image.
The image processing apparatus of execution mode, has: edge extraction portion, extracts marginal information the picture signal before implementing interpolation processing, edge correction portion, the above-mentioned marginal information of extracting based on above-mentioned edge extraction portion is implemented edge correction, and emphasizing contour portion, using the signal of above-mentioned edge correction that has passed through above-mentioned edge correction portion as object, implement emphasizing contour processing, above-mentioned edge correction portion has: maximum value calculation portion, and calculating is arranged in the maximum of the signal level of multiple neighboring pixels of concerned pixel periphery, minimum value calculating part, calculates the minimum value in the signal level of above-mentioned multiple neighboring pixels, and restriction handling part, implement the restriction processing of above-mentioned edge correction, above-mentioned restriction handling part, just be in the above-mentioned marginal information of being extracted by above-mentioned edge extraction portion, and the signal level of above-mentioned concerned pixel with above-mentioned marginal information sum than in the large situation of above-mentioned maximum, by selecting above-mentioned maximum as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned maximum, negative in the above-mentioned marginal information of being extracted by above-mentioned edge extraction portion, and the signal level of above-mentioned concerned pixel with above-mentioned marginal information sum than in the little situation of above-mentioned minimum value, by selecting above-mentioned minimum value as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned minimum value.
In addition, the image processing method of another execution mode is that this image processing method comprises: process from implementing interpolation the step of extracting marginal information previous picture signal, implement the step of edge correction based on extracted above-mentioned marginal information, and using the signal that has passed through the correction of above-mentioned edge as object, implement the step of emphasizing contour processing, the step of implementing the correction of above-mentioned edge comprises: the peaked step of calculating the signal level of the multiple neighboring pixels that are arranged in concerned pixel periphery, calculate the step of the minimum value in the signal level of above-mentioned multiple neighboring pixels, and implement the step of the restriction processing of above-mentioned edge correction, implement in the step of above-mentioned restriction processing, just be in above-mentioned marginal information, and the signal level of above-mentioned concerned pixel with above-mentioned marginal information sum than in the large situation of above-mentioned maximum, by selecting above-mentioned maximum as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned maximum, negative in above-mentioned marginal information, and the signal level of above-mentioned concerned pixel with above-mentioned marginal information sum than in the little situation of above-mentioned minimum value, by selecting above-mentioned minimum value as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned minimum value.
In addition, the video camera module of another execution mode, has: lens unit, is taken into the light from subject, imageing sensor, generates the picture signal with the photophase being taken into by said lens unit, and image processing apparatus, to the above-mentioned picture signal real-time image processing from above-mentioned imageing sensor, above-mentioned image processing apparatus has: edge extraction portion, extracts marginal information the picture signal before implementing interpolation processing, edge correction portion, the above-mentioned marginal information of extracting based on above-mentioned edge extraction portion is implemented edge correction, and emphasizing contour portion, using the signal of above-mentioned edge correction that has passed through above-mentioned edge correction portion as object, implement emphasizing contour processing, above-mentioned edge correction portion has: maximum value calculation portion, and calculating is arranged in the maximum of the signal level of multiple neighboring pixels of concerned pixel periphery, minimum value calculating part, calculates the minimum value in the signal level of above-mentioned multiple neighboring pixels, and restriction handling part, implement the restriction processing of above-mentioned edge correction, above-mentioned restriction handling part, just be in the above-mentioned marginal information of being extracted by above-mentioned edge extraction portion, and the signal level of above-mentioned concerned pixel with above-mentioned marginal information sum than in the large situation of above-mentioned maximum, by selecting above-mentioned maximum as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned maximum, negative in the above-mentioned marginal information of being extracted by above-mentioned edge extraction portion, and the signal level of above-mentioned concerned pixel with above-mentioned marginal information sum than in the little situation of above-mentioned minimum value, by selecting above-mentioned minimum value as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned minimum value.
According to the image processing apparatus of said structure, image processing method and video camera module, can suitably revise the not sharp keen of the marginal portion that is included in image by simple structure.
Brief description of the drawings
Fig. 1 is the piece figure that has used the digital camera of the image processing apparatus of the 1st execution mode.
Fig. 2 is the piece figure that represents the details of edge correction portion.
Fig. 3 is the figure that pays close attention to the arrangement of pixel and neighboring pixel.
Fig. 4 A is the figure that edge correction describes to Fig. 4 D.
Fig. 5 is the piece figure that represents the details of marginal information generating unit.
Fig. 6 represents AG and the figure by the coring noise reduction value (coring value) of AG interlock operational part output and the relation of yield value.
Fig. 7 is the piece figure of the edge correction portion that uses in the image processing apparatus of the 2nd execution mode.
Fig. 8 is the piece figure that has used the digital camera of the image processing apparatus of the 3rd execution mode.
Fig. 9 is the piece figure that represents the details of edge correction portion.
Figure 10 is the figure that pays close attention to the arrangement of pixel and neighboring pixel.
Embodiment
According to execution mode, image processing apparatus has edge extraction portion and edge correction portion.The picture signal of edge extraction portion before being implemented interpolation processing, extract marginal information.Edge correction portion is implemented edge correction based on the marginal information of being extracted by edge extraction portion.Edge correction portion in the case of the signal level of concerned pixel and marginal information sum larger than the maximum of the signal level of multiple neighboring pixels, implement the signal level of concerned pixel to replace with this peaked edge correction.Neighboring pixel is positioned at the periphery of concerned pixel.Edge correction portion in the case of the signal level of concerned pixel and marginal information sum less than the minimum value of the signal level of multiple neighboring pixels, implement the signal level of concerned pixel to replace with the edge correction of minimum value.
Below, with reference to accompanying drawing, explain image processing apparatus, image processing method and video camera module about execution mode.In addition, do not limit the present invention by these execution modes.
Fig. 1 is the piece figure that has used the digital camera of the image processing apparatus of the 1st execution mode.Digital camera 70 has video camera module 71, memory 72 and display part 73.Video camera module 71 shot object image of making a video recording.Memory 72 is preserved the image of being photographed by video camera module 71.Display part 73 shows the image of being photographed by video camera module 71.Display part 73 is for example liquid crystal display.
Video camera module 71 is by the shooting of shot object image, to memory 72 and display part 73 output image signals.Memory 72 is according to operation of user etc., to display part 73 output image signals.Display part 73 shows image according to the picture signal of inputting from video camera module 71 or memory 72.
Video camera module 71 possesses digital signal processor (digital signal processor; DSP) 1, lens unit 3, imageing sensor 4 and analog digital converter (analog to digital converter; ADC) 5.Lens unit 3 is taken into the light from subject, by imageing sensor 4 imaging shot object images.
The light being taken into by lens unit 3 is transformed to signal Electricity lotus by imageing sensor 4, thus shooting shot object image.Imageing sensor 4 is taken into the signal level of red (R), green (G), blue (B) with the order of arranging corresponding to Bayer, generate analog picture signal.ADC5 will be transformed to digital form from analog form from the picture signal of imageing sensor 4.
DSP1 is to implementing various image processing from the data image signal of ADC5.Be located at the interim data image signal storing from ADC5 of line storage 10 in DSP1.Damage correction portion 11 and noise elimination portion 12 common row memories 10.
Damage correction portion 11 implements to be used for the damage correction of the damaged part (damage) of revising the data image signal that the pixel of undesired performance function causes in imageing sensor 4 to data image signal from line storage 10.Noise elimination portion 12 implements to be used for falling low noise noise Processing for removing.Shade (shading) operational part 19 calculates the shade correction factor that is used for implementing shade correction.
The coefficient of digital amplifier (AMP) circuit 13 based on being calculated by AWB/AE operational part 18 described later and the shade correction factor being calculated by shade operational part 19 calculate digital AMP coefficient.In addition, digital AMP circuit 13 is multiplied by digital AMP coefficient to the data image signal after treatment having passed through in damage correction portion 11 and noise elimination portion 12.
Line storage 14 stores the data image signal being multiplied by after digital AMP coefficient temporarily.Pixel interpolating portion 15 carries out interpolation processing (demosaic processing) by the order of arranging with Bayer is transmitted to the data image signal coming from line storage 14, generates the sensitivity signal of RGB.Colour matrix portion 16 implements to be used for obtaining the colour matrix calculation process (colorrendering quality processing) of colorrendering quality to the sensitivity signal of RGB.
Marginal information generating unit 17 is processed picture signal (being suitably called below " RAW image ") generation marginal information 25 before according to the interpolation of implementing in pixel interpolating portion 15.Pixel interpolating portion 15 and marginal information generating unit 17 common row memories 14.AWB/AE operational part 18 is used for Automatic white balance (auto white balance according to the sensitivity calculated signals of RGB; AWB) adjusting, automatic exposure (auto exposure; AE) the each coefficient regulating.
Gamma-corrected portion 20 implements to be used for the gamma-corrected of gray scale of correction image to the sensitivity signal of RGB.YUV transformation component 21 is by generating brightness (Y) signal and aberration (UV) signal according to the sensitivity signal of RGB, such as, by picture signal (YUV422 etc.) conversion from RGB form to YUV form.Line storage 22 stores the Y-signal from YUV transformation component 21 and UV signal temporarily.
The marginal information 25 of edge correction portion 23 based on being generated by marginal information generating unit 17, implements to be used for reducing the edge correction of not sharp keen (fuzzy) that be included in the marginal portion in image.Edge correction portion 23, using the Y-signal obtaining by the conversion in YUV transformation component 21 as object, is implemented edge correction.
Emphasizing contour portion 24, using the Y-signal of edge correction that has passed through edge correction portion 23 as object, implements emphasizing contour processing.The correction factor that emphasizing contour portion 24 is used the position calculation of imaging conditions based on imageing sensor 4 and each pixel to go out, implements emphasizing contour processing.DSP1 output has been passed through the emphasizing contour Y-signal after treatment in emphasizing contour portion 24 and has been kept at the UV signal in line storage 22.
Fig. 2 is the piece figure that represents the details of edge correction portion.Line storage 22 keeps the data image signal of two row amounts.In edge correction portion 23, input remains on two row in line storage 22 and this amounts to the data of 3 row by 1 row of inputting to line storage 22.
Fig. 3 is the figure that pays close attention to the arrangement of pixel and neighboring pixel.Concerned pixel 37 and neighboring pixel 38 are included in matrix, this matrix be in the pixel that in horizontal direction and vertical direction, Bayer is arranged, in horizontal direction be for example 3 with in color pixel and vertical direction being the 3 row L1 that comprise same color pixel, the matrix of L2, L3.Concerned pixel 37 is the pixels that are positioned at the center of 3 × 3 matrix.Neighboring pixel 38 is 8 pixels that are positioned at the periphery of concerned pixel 37.In addition, it is situations of 8 that neighboring pixel 38 is not limited to respect to 1 concerned pixel 37, as long as multiple, how much be can.
Maximum (being called " maximum " the to place an order) A in the signal level of 8 neighboring pixels 38 calculates in maximum value calculation portion 31.Minimum value calculating part 32 calculates minimum value (being called " minimum value " the to place an order) C in the signal level of 8 neighboring pixels 38.The signal level (main line signal) 36 of edge correction portion 23 to concerned pixel 37 adds the marginal information 25 being generated by marginal information generating unit 17.Restriction handling part 33 is implemented the restriction processing of edge correction according to condition.
Be connected to comparator (COMP) 34 in maximum value calculation portion 31 by the maximum A being calculated by maximum value calculation portion 31, compare with signal level 36 and the marginal information 25 sum D of concerned pixel 37.In the situation that A>D sets up, COMP34 for example exports " 1 ".In the invalid situation of A>D, COMP34 for example exports " 0 ".
Be connected to COMP35 on minimum value calculating part 32 by the minimum value C being calculated by minimum value calculating part 32, compare with signal level 36 and the marginal information 25 sum D of concerned pixel 37.In the situation that C<D sets up, COMP35 for example exports " 1 ".In the invalid situation of C<D, COMP35 for example exports " 0 ".
Restriction handling part 33 for example, according to positive and negative (be made as " 1 " when negative, be made as " 0 " when timing) of marginal information 25 and COMP34,35 comparative result, the signal level that selection is used concerned pixel 37 by edge correction.So-called marginal information 25 positive and negative, refers to difference positive and negative of the signal level of adjacent pixel.
That the in the situation that A>D setting up just and in COMP34, restriction handling part 33 is selected and D in marginal information 25.Just and in COMP34, in the invalid situation of A>D, to limit handling part 33 and select maximum A in marginal information 25.
Like this, edge correction portion 23 marginal information 25 be just and and D be maximum A above in the situation that, implement the signal level of concerned pixel 37 36 to replace with the edge correction of maximum A.Edge correction portion 23, when marginal information 25 is timings, is made as maximum A by the upper limit of the signal level that concerned pixel 37 is distributed.
To bear and in COMP35, C<D sets up in the situation that, restriction handling part 33 is selected the signal level 36 and marginal information 25 sum D of concerned pixel 37 in marginal information 25.Bear and in the invalid situation of C<D, limit handling part 33 and select minimum value C in COMP35 in marginal information 25.
Like this, edge correction portion 23 marginal information 25 be bear and and D be below minimum value C in the situation that, implement the signal level of concerned pixel 37 36 to replace with the edge correction of minimum value C.When edge correction portion 23 is negative when marginal information 25, the lower limit of the signal level that concerned pixel 37 is distributed is made as to minimum value B.
Fig. 4 A is the figure that edge correction describes to Fig. 4 D.Fig. 4 A represents sharp keen enforcement edge, marginal portion to emphasize the example of situation about processing.More sharply, overshoot or undershoot are fewer in the variation of signal level, fuzzy fewer, can carry out desirable edge and emphasize.
With respect to this, as shown in Figure 4 B, for the milder marginal portion of the variation of signal level, implement same so that the variation of signal level becomes the significantly edge for the main starting point emphasizes to process.In the case, as dotted in Fig. 4 C, due to edge emphasize the impact of processing relate to commencing signal level variation part and change the part finishing, thereby the expansion of overshoot and undershoot produces the fuzzy significant situation that becomes.In addition,, in emphasizing as basic edge at the image to comprise obtuse marginal portion to process, the inadequate lifting of signal level, the noise that reduction produces become problem.
The signal level that concerned pixel 37 is distributed is restricted to the level between maximum A and minimum value C by the edge correction portion 23 of present embodiment.Thus, edge correction portion 23 as shown in Figure 4 D, can suppress not sharp keen such edge correction of marginal portion.
Fig. 5 is the piece figure that represents the details of marginal information generating unit.Marginal information generating unit 17 has BPF41 and marginal information handling part 42.BPF41 is as the edge extraction portion performance function of extracting marginal information 25 from RAW image.Marginal information handling part 42 is implemented to make its interlock in the processing of analog gain (AG) to the marginal information 25 of being extracted by BPF41.
Line storage 14 keeps the data image signal of 4 row amounts.In BPF41, input remains on 4 row in line storage 14 and this amounts to the data of 5 row by 1 row of inputting to line storage 14.
AE operation blocks 46 is included in the AWB/AE operational part 18 shown in Fig. 1.AE operation blocks 46 is calculated AG47, and exports to marginal information generating unit 17 and imageing sensor 4.
Marginal information handling part 42 has AG interlock operational part 43, coring noise reduction process portion 44 and Nonlinear Processing portion 45.Marginal information handling part 42 maintains predefined coring noise reduction value 48 and yield value 49.The computing that AG interlock operational part 43 implements to make coring noise reduction value 48 that marginal information handling part 42 keeps and yield value 49 and AG47 to change linkedly.
Fig. 6 represents AG and the figure by the coring noise reduction value of AG interlock operational part output and the relation of yield value.AG interlock operational part 43 carries out the computing that makes coring noise reduction value 48 increase corresponding to the increase of AG47.In addition, AG interlock operational part 43 carries out the computing that makes yield value 49 reduce corresponding to the increase of AG47.In addition, AG47 is set to higher value in the time that subject is darker.
Coring noise reduction process portion 44 implements to use the coring noise reduction (coring) of the coring noise reduction value 48 after the computing in AG interlock operational part 43 to process to the marginal information 25 of being extracted by BPF41.Coring noise reduction process portion 44 in the case of the absolute value of signal level than coring noise reduction be worth 48 little, signal level is made as to zero.
Nonlinear Processing portion 45 to through marginal information after the coring noise reduction process in coring noise reduction process portion 44 25 implement to use through the Nonlinear Processing of the yield value 49 after the computing in AG interlock operational part 43.
Like this, marginal information handling part 42 in the case of the brighter situation of subject and darker, the coring noise reduction value 48 using in coring noise reduction process and the yield value 49 using in Nonlinear Processing are changed.Thus, edge correction portion 23 can be suitable for the edge correction of the lightness of subject.
In addition, coring noise reduction value 48 and yield value 49 are not limited to the situation that linearity changes along with the variation of AG, can in AG with which kind of form interlock.Marginal information handling part 42 is not limited to implement the two situation of coring noise reduction process and Nonlinear Processing, just can as long as implement at least one.And then marginal information handling part 42 also can be implemented the processing beyond coring noise reduction process and Nonlinear Processing to the marginal information 25 of being extracted by BPF41.
DSP1 is by using edge correction portion 23, compared with being provided for the situation of the special structure that EDOF processes etc., can alleviate the fuzzy of marginal portion with simple structure.By more than, DSP1 can suitably revise the not sharp keen of the marginal portion that is included in image by simple structure.
AG47, monochrome information etc. that edge correction portion 23 also can calculate according to the marginal information of being extracted by BPF41, by AE operation blocks 46, come " opening ", " pass " of toggle edges correction.Thus, DSP1, by according to the correction of photography conditions toggle edges " opening ", " pass ", can suppress mistake correction.
Fig. 7 is the piece figure of the edge correction portion that uses in the image processing apparatus of the 2nd execution mode.The edge correction portion 50 of present embodiment is used skew to maximum A and minimum value C.Give identical label and the repetitive description thereof will be omitted for the part identical with the 1st execution mode.
Edge correction portion 50 maintains to the predefined deviant 51 of maximum A with to the predefined deviant 52 of minimum value C.Edge correction portion 50 adds deviant 51 to the maximum A being calculated by maximum value calculation portion 31.Edge correction portion 50 is when marginal information 25(is with reference to Fig. 1) be timing, implement to add maximum A after the deviant 51 edge correction as the upper limit.
In addition, edge correction portion 50 deducts deviant 52 from the minimum value C being calculated by minimum value calculating part 32.When edge correction portion 50 is negative when marginal information 25, implement to have deducted minimum value C after the deviant 52 edge correction as lower limit.
DSP1, by using edge correction portion 50, can carry out with simple structure the obtuse correction of marginal portion.In addition, DSP1 uses skew by the maximum A to for limiting edge correction and minimum value C, can also obtain the effect of emphasizing by the additional edge bringing of momentum.
Deviant 51,52 also can change linkedly with AG.Thus, edge correction portion 50 can reduce the impact of the noise of edges by trimming center.Edge correction portion 50 is not limited to both of maximum A and minimum value C to use the situation of skew, as long as use skew just passable at least one party in maximum A and minimum value C.Thus, DSP1 can obtain the effect of being emphasized by the additional edge bringing of at least one party in overshoot and undershoot.
Fig. 8 is the piece figure that has used the digital camera of the image processing apparatus of the 3rd execution mode.Possesses the edge correction portion 60 of implementing edge correction taking RAW image as object as the DSP2 of the image processing apparatus of present embodiment.Give identical label for the part identical with the 1st execution mode, and the repetitive description thereof will be omitted.
Pixel interpolating portion 15 and edge correction portion 60 common row memories 14.Edge correction portion 60 taking through the data image signal after the multiplication of the digital AMP coefficient in digital AMP circuit 13 as object, implement edge correction.Emphasizing contour portion 24, taking the Y-signal that obtains by the conversion of YUV transformation component 21 as object, implements emphasizing contour processing.
Fig. 9 is the piece figure that represents the details of edge correction portion.Line storage 14 keeps the data image signal of 4 row amounts.In edge correction portion 60, be transfused to 4 row that remain in line storage 14 and be about to 1 row inputted to line storage 14 that this amounts to the data of 5 row.
Figure 10 is the figure that pays close attention to the arrangement of pixel and neighboring pixel.Concerned pixel 67 and neighboring pixel 68 are included in matrix, and this matrix is in the pixel that in horizontal direction and vertical direction, Bayer is arranged, to be for example in 5 and vertical direction to be the matrix of 5 row L1～L5 in horizontal direction.Concerned pixel 67 is the pixels that are positioned at the center of 5 × 5 matrix.Neighboring pixel 68 be arranged in the periphery of concerned pixel 67 pixel, with 8 pixels of concerned pixel 67 homochromy use.In addition, neighboring pixel 68 be not limited to be and the situation of the pixel of concerned pixel 67 homochromy use, can be also the pixel that the color different from concerned pixel 67 used, as long as multiple, be several can.
Edge correction portion 60 has maximum value calculation portion 61, minimum value calculating part 62, restriction handling part 63, BPF64 and marginal information handling part 65.The maximum A in the signal level of 8 neighboring pixels 68 calculates in maximum value calculation portion 61.Minimum value calculating part 62 calculates the minimum value C in the signal level of 8 neighboring pixels 68.
BPF64 is as the edge extraction portion performance function of extracting marginal information 25 from RAW image.In BPF64, be transfused to two row in 4 row that remain in line storage 14 and be about to 1 row inputted to line storage 14 that this amounts to the data of 3 row.A behavior from line storage 14 to BPF64 two row of input comprises the row L3 of concerned pixel 67.
Marginal information handling part 65 is implemented interlock in the processing of AG47 to the marginal information 25 of being extracted by BPF64.BPF64 and marginal information handling part 65 form the marginal information generating unit that generates marginal information 25.
Restriction handling part 63 is according to condition, with the restriction handling part 33(of the 1st execution mode with reference to Fig. 2) similarly implement the restriction processing of edge correction.Restriction handling part 63 is by the maximum A being calculated by maximum value calculation portion 61, compare with signal level 66 and the marginal information 25 sum D of concerned pixel 67.Restriction handling part 63 is by the minimum value C being calculated by minimum value calculating part 62, compare with signal level 66 and the marginal information 25 sum D of concerned pixel 67.
Restriction handling part 63 according to positive and negative, the maximum A of marginal information 25 and and the comparative result of D, minimum value C and and the comparative result of D, select signal level concerned pixel 67 being used by edge correction.
Marginal information 25 be just and A>D set up in the situation that, restriction handling part 63 select and D.Be just and in the invalid situation of A>D in marginal information 25, restriction handling part 63 is selected maximum A.
Like this, edge correction portion 60 marginal information 25 be just and and D be maximum A above in the situation that, implement the signal level of concerned pixel 67 66 to replace with the edge correction of maximum A.Edge correction portion 60 is timings in marginal information 25, and the upper limit of the signal level that concerned pixel 67 is distributed is made as to maximum A.
Marginal information 25 be bear and C<D set up in the situation that, restriction handling part 63 select and D.Marginal information 25 be bear and the invalid situation of C<D under, restriction handling part 63 select minimum value C.
Like this, edge correction portion 60 marginal information 25 be bear and and D be below minimum value C in the situation that, implement the signal level of concerned pixel 67 66 to replace with the edge correction of minimum value C.When edge correction portion 60 is negative when marginal information 25, the lower limit of the signal level that concerned pixel 67 is distributed is made as to minimum value B.
Edge correction portion 60 is exported the signal level of being selected by restriction handling part 63.Pixel interpolating portion 15 using comprise through the RAW image of the revised signal level in edge in edge correction portion 60 as object, implement interpolation processing.
The signal level that concerned pixel 67 is distributed is limited in the level between maximum A and minimum value C by the edge correction portion 60 of present embodiment.Thus, edge correction portion 60 can suppress not sharp keen such edge correction of marginal portion.
DSP2, by using edge correction portion 60, can carry out by simple structure the obtuse correction of marginal portion.The object that DSP2 has made the edge correction making in edge correction portion 60 is RAW image, structure by edge correction portion 60 and pixel interpolating portion 15 common row memories 14.Thus, DSP2, compared with processing the line storage of use with respect to pixel interpolating and be additionally provided for the situation of line storage of edge correction, can cut down circuit scale.
About the 1st, the 2nd and the image processing apparatus of the 3rd execution mode also can be beyond digital camera electronic equipment, for example, with use in the portable phone of video camera etc.
Some embodiments of the present invention have been described, but these execution modes point out as an example, and do not mean that restriction scope of invention.These new execution modes can be implemented with other various forms, can carry out various omissions, substitute, change in the scope of purport that does not depart from invention.These execution modes and distortion thereof are included in scope of invention and purport, and are included in the invention and its scope being equal to of recording in claims.
Claims (13)
1. an image processing apparatus, is characterized in that,
Have:
Edge extraction portion, extracts marginal information the picture signal before implementing interpolation processing;
Edge correction portion, the above-mentioned marginal information of extracting based on above-mentioned edge extraction portion is implemented edge correction; And
Emphasizing contour portion, using the signal of above-mentioned edge correction that has passed through above-mentioned edge correction portion as object, implements emphasizing contour processing,
Above-mentioned edge correction portion has:
Maximum value calculation portion, calculating is arranged in the maximum of the signal level of multiple neighboring pixels of concerned pixel periphery;
Minimum value calculating part, calculates the minimum value in the signal level of above-mentioned multiple neighboring pixels; And
Restriction handling part, implements the restriction processing of above-mentioned edge correction,
Above-mentioned restriction handling part,
In the case of the above-mentioned marginal information of being extracted by above-mentioned edge extraction portion be just and the signal level of above-mentioned concerned pixel larger than above-mentioned maximum with above-mentioned marginal information sum, by selecting above-mentioned maximum as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned maximum
In the case of the above-mentioned marginal information of being extracted by above-mentioned edge extraction portion be bear and the signal level of above-mentioned concerned pixel less than above-mentioned minimum value with above-mentioned marginal information sum, by selecting above-mentioned minimum value as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned minimum value.
2. image processing apparatus as claimed in claim 1, is characterized in that,
Also have:
Line storage, keeps above-mentioned picture signal; And
Pixel interpolating portion, implements above-mentioned interpolation processing;
The shared above line memory of above-mentioned pixel interpolating portion and above-mentioned edge extraction portion.
3. image processing apparatus as claimed in claim 1, is characterized in that, above-mentioned edge correction portion, taking from having passed through luminance signal that above-mentioned interpolation picture signal after treatment obtains as object, is implemented the correction of above-mentioned edge.
4. image processing apparatus as claimed in claim 1, is characterized in that, above-mentioned edge correction portion, to process previous picture signal as object through above-mentioned interpolation, is implemented the correction of above-mentioned edge.
5. image processing apparatus as claimed in claim 1, is characterized in that, also has marginal information handling part, and this marginal information handling part is implemented to make its interlock in the processing of analog gain to the above-mentioned marginal information of being extracted by above-mentioned edge extraction portion.
6. image processing apparatus as claimed in claim 1, is characterized in that, above-mentioned edge correction portion is used skew at least one party in above-mentioned maximum and above-mentioned minimum value.
7. an image processing method, is characterized in that,
This image processing method comprises:
Process from implementing interpolation the step of extracting marginal information previous picture signal;
Implement the step of edge correction based on extracted above-mentioned marginal information; And
Using the signal that has passed through the correction of above-mentioned edge as object, implement the step of emphasizing contour processing,
The step of implementing the correction of above-mentioned edge comprises:
Calculating is arranged in the peaked step of the signal level of multiple neighboring pixels of concerned pixel periphery;
Calculate the step of the minimum value in the signal level of above-mentioned multiple neighboring pixels; And
Implement the step of the restriction processing of above-mentioned edge correction,
Implement in the step of above-mentioned restriction processing,
In the case of above-mentioned marginal information be just and the signal level of above-mentioned concerned pixel larger than above-mentioned maximum with above-mentioned marginal information sum, by selecting above-mentioned maximum as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned maximum
In the case of above-mentioned marginal information be bear and the signal level of above-mentioned concerned pixel less than above-mentioned minimum value with above-mentioned marginal information sum, by selecting above-mentioned minimum value as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned minimum value.
8. image processing method as claimed in claim 7, is characterized in that,
Keep above-mentioned picture signal by line storage;
Use the above-mentioned picture signal of reading from shared above line memory, implement extraction and the above-mentioned interpolation processing of above-mentioned marginal information.
9. image processing method as claimed in claim 7, is characterized in that, taking from having passed through luminance signal that above-mentioned interpolation picture signal after treatment obtains as object, implements the correction of above-mentioned edge.
10. image processing method as claimed in claim 7, is characterized in that, to process previous picture signal as object through above-mentioned interpolation, implements the correction of above-mentioned edge.
11. image processing methods as claimed in claim 7, is characterized in that, also implement to make its interlock in the processing of analog gain to extracted above-mentioned marginal information.
12. image processing methods as claimed in claim 7, is characterized in that, in above-mentioned edges by trimming center, at least one party in above-mentioned maximum and above-mentioned minimum value are used to skew.
13. 1 kinds of video camera modules, is characterized in that,
Have:
Lens unit, is taken into the light from subject;
Imageing sensor, generates the picture signal with the photophase being taken into by said lens unit; And
Image processing apparatus, to the above-mentioned picture signal real-time image processing from above-mentioned imageing sensor;
Above-mentioned image processing apparatus has:
Edge extraction portion, extracts marginal information the picture signal before implementing interpolation processing;
Edge correction portion, the above-mentioned marginal information of extracting based on above-mentioned edge extraction portion is implemented edge correction; And
Emphasizing contour portion, using the signal of above-mentioned edge correction that has passed through above-mentioned edge correction portion as object, implements emphasizing contour processing,
Above-mentioned edge correction portion has:
Maximum value calculation portion, calculating is arranged in the maximum of the signal level of multiple neighboring pixels of concerned pixel periphery;
Minimum value calculating part, calculates the minimum value in the signal level of above-mentioned multiple neighboring pixels; And
Restriction handling part, implements the restriction processing of above-mentioned edge correction,
Above-mentioned restriction handling part,
In the case of the above-mentioned marginal information of being extracted by above-mentioned edge extraction portion be just and the signal level of above-mentioned concerned pixel larger than above-mentioned maximum with above-mentioned marginal information sum, by selecting above-mentioned maximum as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned maximum
In the case of the above-mentioned marginal information of being extracted by above-mentioned edge extraction portion be bear and the signal level of above-mentioned concerned pixel less than above-mentioned minimum value with above-mentioned marginal information sum, by selecting above-mentioned minimum value as the signal level that above-mentioned concerned pixel is used, the correction of above-mentioned edge is restricted to above-mentioned minimum value.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP186517/2010 | 2010-08-23 | ||
| JP2010186517A JP5398667B2 (en) | 2010-08-23 | 2010-08-23 | Image processing device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102377911A CN102377911A (en) | 2012-03-14 |
| CN102377911B true CN102377911B (en) | 2014-09-03 |
Family
ID=45594127
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110210377.6A Expired - Fee Related CN102377911B (en) | 2010-08-23 | 2011-07-26 | Image processing apparatus, image processing method, and camera module |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US8559747B2 (en) |
| JP (1) | JP5398667B2 (en) |
| CN (1) | CN102377911B (en) |
| TW (1) | TWI456974B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10623759B2 (en) | 2012-06-13 | 2020-04-14 | Sony Corporation | Decoupling enhancements in sample adaptive offset (SAO) for high efficiency video encoder (HEVC) |
| JP6074566B2 (en) * | 2012-07-02 | 2017-02-08 | ソニー株式会社 | Enhanced sample adaptive offset (SAO) separation for high-efficiency video encoder (HEVC) |
| KR102056686B1 (en) | 2013-02-18 | 2019-12-18 | 삼성디스플레이 주식회사 | Image processing part, display apparatus having the same and method of processing image using the same |
| JP6403490B2 (en) * | 2014-08-20 | 2018-10-10 | キヤノン株式会社 | Image processing apparatus, image forming apparatus, image processing method, and program. |
| JP6696361B2 (en) * | 2016-08-31 | 2020-05-20 | コニカミノルタ株式会社 | Image processing apparatus, image forming apparatus and program |
| JP6794858B2 (en) * | 2017-02-06 | 2020-12-02 | コニカミノルタ株式会社 | Image processing device and image processing method |
| JP6813004B2 (en) * | 2018-06-28 | 2021-01-13 | Ｊｆｅスチール株式会社 | Steel non-pressure lower width detector and its detection method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10164371A (en) * | 1996-11-26 | 1998-06-19 | Minolta Co Ltd | Image data processor and image data processing method |
| JP2003123063A (en) * | 2001-10-18 | 2003-04-25 | Olympus Optical Co Ltd | Image processor |
| JP2004088285A (en) * | 2002-08-26 | 2004-03-18 | Sanyo Electric Co Ltd | Image processing apparatus |
| JP2010068361A (en) * | 2008-09-12 | 2010-03-25 | Noritsu Koki Co Ltd | Photographic image processing method, photographic image processing program, and photographic image processor |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000115526A (en) | 1998-10-06 | 2000-04-21 | Hitachi Ltd | Picture processor and edge processing method |
| KR100671212B1 (en) * | 1999-12-31 | 2007-01-18 | 엘지.필립스 엘시디 주식회사 | Method for forming poly silicon |
| US7505604B2 (en) * | 2002-05-20 | 2009-03-17 | Simmonds Precision Prodcuts, Inc. | Method for detection and recognition of fog presence within an aircraft compartment using video images |
| JP4104475B2 (en) | 2003-03-18 | 2008-06-18 | シャープ株式会社 | Contour correction device |
| US7773115B2 (en) * | 2004-12-15 | 2010-08-10 | Texas Instruments Incorporated | Method and system for deblurring digital camera images using reference image and motion estimation |
| JP2008178075A (en) * | 2006-12-18 | 2008-07-31 | Sony Corp | Display control device, display control method, and program |
| WO2009019808A1 (en) * | 2007-08-03 | 2009-02-12 | Panasonic Corporation | Image data generating apparatus, method, and program |
| JP4868249B2 (en) | 2007-10-10 | 2012-02-01 | 株式会社Ｊｖｃケンウッド | Video signal processing device |
| JP5045421B2 (en) * | 2007-12-21 | 2012-10-10 | ソニー株式会社 | Imaging apparatus, color noise reduction method, and color noise reduction program |
| JP2009200884A (en) * | 2008-02-22 | 2009-09-03 | Seiko Epson Corp | Image data processor and image data processing method |
| JP2009206552A (en) * | 2008-02-26 | 2009-09-10 | Victor Co Of Japan Ltd | Image processing apparatus |
| JP4875032B2 (en) | 2008-08-11 | 2012-02-15 | 株式会社東芝 | Solid-state imaging device |
| TWI399703B (en) * | 2009-12-02 | 2013-06-21 | Micro Star Int Co Ltd | Forward and backward resizing method |
-
2010
- 2010-08-23 JP JP2010186517A patent/JP5398667B2/en not_active Expired - Fee Related
-
2011
- 2011-06-09 TW TW100120167A patent/TWI456974B/en not_active IP Right Cessation
- 2011-06-13 US US13/158,588 patent/US8559747B2/en not_active Expired - Fee Related
- 2011-07-26 CN CN201110210377.6A patent/CN102377911B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10164371A (en) * | 1996-11-26 | 1998-06-19 | Minolta Co Ltd | Image data processor and image data processing method |
| JP2003123063A (en) * | 2001-10-18 | 2003-04-25 | Olympus Optical Co Ltd | Image processor |
| JP2004088285A (en) * | 2002-08-26 | 2004-03-18 | Sanyo Electric Co Ltd | Image processing apparatus |
| JP2010068361A (en) * | 2008-09-12 | 2010-03-25 | Noritsu Koki Co Ltd | Photographic image processing method, photographic image processing program, and photographic image processor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102377911A (en) | 2012-03-14 |
| US20120045144A1 (en) | 2012-02-23 |
| TW201225630A (en) | 2012-06-16 |
| JP2012044617A (en) | 2012-03-01 |
| US8559747B2 (en) | 2013-10-15 |
| TWI456974B (en) | 2014-10-11 |
| JP5398667B2 (en) | 2014-01-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102377911B (en) | Image processing apparatus, image processing method, and camera module | |
| KR101061866B1 (en) | Image processing apparatus for performing gradation correction on the target image | |
| KR101099401B1 (en) | Image processing apparatus and computer-readable medium | |
| CN104954771B (en) | Carry out the image processing equipment and image processing method of color range correction | |
| JP4678218B2 (en) | Imaging apparatus and image processing method | |
| EP1931130A2 (en) | Image processing apparatus, image processing method, and program | |
| US20070242875A1 (en) | Image processing device and method | |
| JP5343726B2 (en) | Image processing apparatus and image processing program | |
| US20090091643A1 (en) | Image-Acquisition Apparatus | |
| US8144211B2 (en) | Chromatic aberration correction apparatus, image pickup apparatus, chromatic aberration amount calculation method, and chromatic aberration amount calculation program | |
| US8885078B2 (en) | Image processing apparatus, image processing method, and recording medium storing image processing program | |
| US9712797B2 (en) | Image processing apparatus, image pickup apparatus, image processing method, and non-transitory computer-readable medium | |
| JP5569359B2 (en) | IMAGING CONTROL DEVICE, IMAGING DEVICE, AND IMAGING DEVICE CONTROL METHOD | |
| KR101353863B1 (en) | Image processing apparatus, image processing method and camera module | |
| CN103581537A (en) | Image processing device, image processing method and solid-state imaging device | |
| JP6375806B2 (en) | Image processing apparatus, method, and imaging apparatus | |
| JP2008085634A (en) | Imaging apparatus and image processing method | |
| US8957999B2 (en) | Method of selective aperture sharpening and halo suppression using chroma zones in CMOS imagers | |
| JP6316006B2 (en) | SUBJECT SEARCH DEVICE, ITS CONTROL METHOD, CONTROL PROGRAM, AND IMAGING DEVICE | |
| JP2009239974A (en) | Automatic exposing device and method | |
| JP2008011291A (en) | Imaging apparatus | |
| JP2010187050A (en) | Image processor, imaging apparatus, and image processing method | |
| JP2013242830A (en) | Image processing apparatus, image processing program, and image processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20140903 Termination date: 20160726 |