CN107038695A - A kind of image interfusion method and mobile device - Google Patents
A kind of image interfusion method and mobile device Download PDFInfo
- Publication number
- CN107038695A CN107038695A CN201710260280.3A CN201710260280A CN107038695A CN 107038695 A CN107038695 A CN 107038695A CN 201710260280 A CN201710260280 A CN 201710260280A CN 107038695 A CN107038695 A CN 107038695A
- Authority
- CN
- China
- Prior art keywords
- image
- pixel
- subregion
- super
- chrominance information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Abstract
The invention discloses a kind of image interfusion method, this method is used to handle the first image and the second image shot for same object simultaneously, wherein the first image has monochrome information, the second image has chrominance information, including：To each pixel in the first image：Search whether exist and its content identical pixel in the second image；If in the presence of the chrominance information of the pixel in the second image found to be assigned to the corresponding pixel points in the first image；Super-pixel subregion is carried out to the first image according to monochrome information, in each super-pixel subregion, chrominance information is screened according to predetermined condition, the 3rd image is obtained；According to the predetermined method of sampling to still the pixel without chrominance information carries out color filling in the 3rd image, make all pixels point that all there is chrominance information；And blend the monochrome information of the chrominance information of corresponding pixel points and the 3rd image, the image after being merged.The present invention also discloses corresponding mobile device in the lump.
Description
Technical field
The present invention relates to technical field of image processing, especially a kind of image interfusion method and mobile device.
Background technology
(that is, it is by taking pictures of constituting of two cameras using to double systems of taking the photograph in increasing mobile device now
System), such as there are double mobile phone, cameras for taking the photograph system.It is existing it is double take the photograph system, be mostly based on depth information to image at
Reason, to realize 3D effect, such as image background virtualization, object segmentation, 3D scannings, auxiliary focusing, action recognition.Typically this
Under application scenarios, the spacing for keeping as big as possible between two cameras is may require that, to obtain higher depth of field precision.
Under another scene, optical zoom is realized using double systems of taking the photograph, it is different from the past in back-end processing image pixel,
To reach the processing mode of amplification effect (that is, Digital Zoom), double dual camera combination dies for taking the photograph system using " remote burnt+near burnt "
Formula, then by later stage synthesis processing, so as to realize optical zoom.
Under above-mentioned various application scenarios, how the image that system output color is abundant, picture quality is higher is taken the photograph using double,
Be the invention solves the problems that a problem.
The content of the invention
Therefore, the invention provides a kind of image co-registration scheme, with try hard to solve or at least alleviate exist above to
A few problem.
According to an aspect of the invention, there is provided a kind of image interfusion method, this method is used to handle simultaneously for same
The first image and the second image that one object is shot, wherein the first image has monochrome information, the second image has chrominance information,
The method comprising the steps of：To each pixel in the first image：Searched whether in the second image in the presence of identical with its content
Pixel；If in the presence of the chrominance information of the pixel in the second image found to be assigned to the correspondence in the first image
Pixel；Super-pixel subregion is carried out to the first image according to monochrome information, in each super-pixel subregion, according to predetermined condition pair
Chrominance information is screened, and obtains the 3rd image；According to the predetermined method of sampling to not having chrominance information in the 3rd image still
Pixel carry out color filling, make all pixels point all have chrominance information；And by the chrominance information of corresponding pixel points and
The monochrome information of 3rd image is blended, the image after being merged.
Alternatively, in the image interfusion method according to the present invention, super-pixel is carried out to the first image according to monochrome information
The step of subregion, includes：Clustering processing is carried out using monochrome information and the image of pixel space coordinate pair first, multiple super pictures are obtained
Plain subregion.
Alternatively, in the image interfusion method according to the present invention, the step of carrying out clustering processing to the first image includes：
The distance between each pixel value in the first image is calculated, each pixel is obtained to the initial distance collection of other pixels；According to
The number of partitions calculated permutations interval of setting, and according to the initial center point of each subregion of arrangement pitch generation；All subregions are traveled through, point
Pixel in the initial center vertex neighborhood of each subregion is not calculated, to the distance value of the initial center point, to be somebody's turn to do if the distance value is less than
Any distance value that pixel initial distance is concentrated, then be included into the subregion by the pixel；After traversal terminates, according in each subregion
The monochrome information and pixel space coordinate of pixel calculate the new central point of the subregion, and calculate the new central point with it is initial in
The difference of heart point；And the step of above-mentioned traversal subregion of iteration, calculating new central point, until new central point and a upper center
The difference of point is less than first threshold, and cluster terminates.
Alternatively, in the image interfusion method according to the present invention, after the step of cluster terminates, in addition to：Judge poly-
Whether the super-pixel subregion that class is obtained connects；And if judge that a certain super-pixel subregion is not connected, the subregion is included into its phase
Adjacent subregion.
Alternatively, in the image interfusion method according to the present invention, judge that the step of whether super-pixel subregion connects includes：
Count the size of each super-pixel subregion；And if the super-pixel subregion is less than Second Threshold, judges that the subregion is not connected.
Alternatively, in the image interfusion method according to the present invention, the initial center of each subregion is generated according to arrangement pitch
The step of point, also includes：Initial center point and the Grad of pixel in its eight neighborhood are calculated, the minimum pixel of Grad is chosen
Put the initial center point as the subregion.
Alternatively, in the image interfusion method according to the present invention, in addition to step：According to bright between two pixels
Degree distance and physical distance weighted sum obtain the distance value of described two pixels.
Alternatively, in the image interfusion method according to the present invention, in addition to step：According to the total pixel number of the first image
Arrangement pitch is calculated with the number of partitions of setting：
Wherein, N is the total pixel number of the first image, and k is the number of partitions set.
Alternatively, in the image interfusion method according to the present invention, Second Threshold is calculated by the arrangement pitch and obtained.
Alternatively, in the image interfusion method according to the present invention, chrominance information is carried out according to predetermined condition to screen
The step of to three images, includes：To the pixel with chrominance information, sorted by the order of chrominance information from small to large；And
Remove the chrominance information of the pixel outside predetermined interval.
Alternatively, in the image interfusion method according to the present invention, predetermined interval and pixel in corresponding super-pixel subregion
It is total related.
Alternatively, according to the present invention image interfusion method in, according to the predetermined method of sampling in the 3rd image still
The step of pixel without chrominance information carries out color filling includes：On the basis of the monochrome information of the 3rd image, to it
Chrominance information carries out down-sampling at least one times and obtains at least one down-sampled images, until the size of down-sampled images is 2*2 pictures
Element, during down-sampling, calculates the chrominance information of each down-sampled images and records respective pixel space coordinate；And from 2*
The down-sampled images of 2 Pixel Dimensions start, and down-sampled images are up-sampled by the opposite sequence of down-sampling, until returning to the
The size of three images, in upsampling process, according to the chrominance information of each down-sampled images and respective pixel space coordinate meter
Calculate the chrominance information of each up-sampling image.
Alternatively, in the image interfusion method according to the present invention, the chrominance information or meter of each down-sampled images are calculated
The step of chrominance information for calculating each up-sampling image, includes：According to pixel in the neighborhood of a pixel four or diagonal four neighborhood
Chrominance information weights the chrominance information for trying to achieve the pixel, wherein, the brightness distance of weight coefficient and corresponding pixel points is negatively correlated.
Alternatively, in the image interfusion method according to the present invention, chrominance information includes the first colour difference information and the second color
Poor information.
Alternatively, in the image interfusion method according to the present invention, the first image is gray level image, and the second image is
Coloured image.
According to another aspect of the invention there is provided a kind of mobile device, including：Dual camera imaging system, suitable for same
When gather the first image and the second image；One or more processors；Memory；One or more programs, one of them or it is many
Individual program storage in memory and is configured as being performed by one or more processors, and one or more of programs include using
In the instruction for performing the either method in method as described above.
Alternatively, in the mobile device according to the present invention, dual camera imaging system includes a black and white camera imaging
System and a colour imagery shot imaging system.
There is provided a kind of computer-readable storage medium for storing one or more programs according to another aspect of the invention
Matter, one or more of programs include instruction, and the instruction is when computing device so that computing device as above institute
Either method in the method stated.
According to the image co-registration scheme of the present invention, using the monochrome information of the first image as the brightness of final fused images,
The characteristics of black and white camera imaging system noise is few, details is abundant, dynamic contrast is high can be given full play to；And then use super picture
Plain subregion amendment has been painted information, to reduce the spilling of the color in fusion process, while enhancing the robustness of algorithm again；Then
Phase simulates diffusion process of the color in full figure again, obtains final fused images, both ensure that fused image rich color,
There is very high picture quality again.
Brief description of the drawings
In order to realize above-mentioned and related purpose, some illustrative sides are described herein in conjunction with following description and accompanying drawing
Face, these aspects indicate the various modes of principles disclosed herein that can put into practice, and all aspects and its equivalent aspect
It is intended to fall under in the range of theme claimed.The following detailed description by being read in conjunction with the figure, the disclosure it is above-mentioned
And other purposes, feature and advantage will be apparent.Throughout the disclosure, identical reference generally refers to identical
Part or element.
Fig. 1 shows the organigram of mobile device 100 according to an embodiment of the invention；
Fig. 2 shows the flow chart of image interfusion method 200 according to an embodiment of the invention；
Fig. 3 A show the partial schematic diagram of down-sampled images according to an embodiment of the invention；And
Fig. 3 B show the partial schematic diagram of up-sampling image according to an embodiment of the invention.
Embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although showing the disclosure in accompanying drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
Limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
Complete conveys to those skilled in the art.
Fig. 1 shows the organigram of mobile device 100 according to an embodiment of the invention.Reference picture 1, movement is set
Standby 100 include：Memory interface 102, one or more data processors, image processor and/or CPU 104,
And peripheral interface 106.Memory interface 102, one or more processors 104 and/or peripheral interface 106 both can be discrete
Element, can also be integrated in one or more integrated circuits.In mobile device 100, various elements can by one or
A plurality of communication bus or signal wire are coupled.Sensor, equipment and subsystem may be coupled to peripheral interface 106, to help
Realize a variety of functions.For example, motion sensor 110, optical sensor 112 and range sensor 114 may be coupled to peripheral interface
106, to facilitate the functions such as orientation, illumination and ranging.Other sensors 116 can equally be connected with peripheral interface 106, such as fixed
Position system (such as GPS), temperature sensor, biometric sensor or other sensor devices, it is possible thereby to help reality
Apply the function of correlation.
Camera sub-system 120 and optical sensor 122 can be used for the camera of convenient such as recording photograph and video clipping
The realization of function, wherein camera sub-system and optical sensor for example can be charge coupling device (CCD) or complementary metal oxygen
Compound semiconductor (CMOS) optical sensor.In the present invention, camera sub-system 120 be arranged to it is double take the photograph system, for simultaneously
Obtain two images of same target object.Alternatively, this pair takes the photograph integrated mode of the system using " colour+black and white ", i.e., one
Colour imagery shot imaging system and a black and white camera imaging system.In the image procossing in later stage, chrominance information mainly by
Colour imagery shot imaging system is provided, and detailed information is main by the offer of black and white camera imaging system.
Communication function, wherein wireless communication can be helped to realize by one or more radio communication subsystems 124
System can include radio-frequency transmitter and emitter and/or light (such as infrared) Receiver And Transmitter.Radio communication subsystem
124 particular design and embodiment can depend on one or more communication networks that mobile device 100 is supported.For example,
Mobile device 100 can include being designed to supporting GSM network, GPRS network, EDGE network, Wi-Fi or WiMax network and
The communication subsystem 124 of BlueboothTM networks.Audio subsystem 126 can be with loudspeaker 128 and the phase coupling of microphone 130
Close, to help to implement the function of enabling voice, such as speech recognition, speech reproduction, digital record and telephony feature.
I/O subsystems 140 can include touch screen controller 142 and/or other one or more input controllers 144.
Touch screen controller 142 may be coupled to touch-screen 146.For example, the touch-screen 146 and touch screen controller 142 can be with
The contact and movement or pause carried out therewith is detected using any one of a variety of touch-sensing technologies, wherein sensing skill
Art includes but is not limited to capacitive character, resistive, infrared and surface acoustic wave technique.Other one or more input controllers 144
May be coupled to other input/control devicess 148, such as one or more buttons, rocker switch, thumb wheel, infrared port,
The pointer device of USB port, and/or instruction pen etc.One or more button (not shown)s can include raising one's voice for control
The up/down button of device 128 and/or the volume of microphone 130.
Memory interface 102 can be coupled with memory 150.The memory 150 can be deposited including high random access
Reservoir and/or nonvolatile memory, such as one or more disk storage equipments, one or more optical storage apparatus, and/
Or flash memories (such as NAND, NOR).Memory 150 can store an operating system 152, such as Android, IOS or
Windows Phone etc operating system.The operating system 152 can include being used to handle basic system services and execution
Instruction dependent on the task of hardware.Memory 150 can also store and apply 154.These are applied in operation, can be from memory
150 are loaded on processor 104, and are run on the operating system run via processor 104, and utilize operating system
And the interface that bottom hardware is provided realizes the desired function of various users, such as instant messaging, web page browsing, pictures management.
It is operating system offer or that operating system is carried using that can be independently of.In some implementations, apply
154 can be one or more programs.
The invention provides based on " colour+black and white " double image co-registration schemes for taking the photograph system, by mobile device 100
Corresponding one or more programs are stored in memory 150 to realize above-mentioned functions.It should be noted that the signified shifting of the present invention
Dynamic equipment 100 can be the mobile phone with above-mentioned construction, flat board, camera etc..
" colour+black and white " double synthetic schemes for taking the photograph system are current a great problems, can generally be selected " with coloured image
It is used as Primary Reference image " " Primary Reference image is used as using black white image " " reference picture is only used as using coloured image " etc..
However, double systems of taking the photograph can inevitably bring occlusion effect, and most composition algorithm speed is slower, or, synthesis
The problem of noise is more still occurs afterwards, syncretizing effect is bad largely to have impact on Consumer's Experience.
Fig. 2 shows the flow chart of image interfusion method 200 according to an embodiment of the invention.
According to one embodiment of present invention, two images collected by mobile device 100, wherein one is black and white
Image, one is coloured image, and this two images are designated as：First image and the second image, wherein, the first image has brightness
Information, the second image has chrominance information.It should be noted that being not to say that the second image (that is, coloured image) does not have here
There is monochrome information, simply in the present embodiment, its chrominance information is more paid close attention to for the second image.
In addition, this method 200 is also not limited to from mobile device 100 obtain the first image and the second image.With bright
It for example can be by two mobile devices placed parallel to spend the first image of information and the second image with chrominance information
Gained is shot to same object.The present invention is without limitation.
As shown in Fig. 2 this method starts from step S210, using the first image as master map, to each picture in the first image
Vegetarian refreshments, searched whether in the second image exist with its content identical pixel (that is, the pixel being mutually matched), if depositing
The chrominance information of pixel in the second image found is then being assigned to the corresponding pixel points in the first image.Due to regarding
The relation of difference, coordinate position of the same content in the first image and the second image may be different, for example, in photo
Some table angle, is located at the 100th row the 200th row in the first image, and the 100th row the 250th of correspondence is arranged in the second image, that
, in an embodiment of the present invention, it is believed that (100,250) in (100,200) and the second image in the first image are mutual
Matching, at this moment, the chrominance information of (100,250) pixel in the second image can be assigned in the first image (100,
200) chrominance information.
Embodiments in accordance with the present invention, it is contemplated that YC separation, are transformed into YUV passages by the RGB image collected and carry out
Processing, now, monochrome information are Y channel values, and chrominance information is the value of the passage of U, V two, and U passages record the first colour difference information, and V leads to
The colour difference information of trace record second.Further, it is contemplated that retain the colouring information of image as much as possible, " YUV can be used
4：4：4 " form is handled.Alternatively it is also possible to be handled in color spaces such as HSI, HSV, as long as ensureing light tone point
From.
After step S210 processing, part pixel has chrominance information in the first image, i.e. be state of having painted,
Pixel that also partial pixel point matches because can not find and remain unchanged and keep grey states.
Then in step S220, super-pixel subregion is carried out to the first image according to monochrome information, according to the reality of the present invention
Mode is applied, super-pixel subregion is exactly to be clicked through using the monochrome information of the first image and the pixel of the image of pixel space coordinate pair first
Row clustering processing, obtains multiple super-pixel subregions.
More specifically, the step of carrying out clustering processing to the first image includes：
A. calculate the distance between each pixel value in the first image, obtain each pixel to other pixels it is initial away from
From collection.Optionally, the distances of two pixels is defined both the distance between comprising brightness Y, also contains between coordinate (x, y)
Distance, so distance value is obtained according to the brightness distance between two pixels and physical distance weighted sum.
B. according to the number of partitions of setting and the total pixel number calculated permutations interval of the first image, i.e.
Arrangement pitch
Wherein, N is the total pixel number of the first image, and k is the number of partitions set.
The initial center point of each subregion is generated according still further to arrangement pitch.If directly selecting the first of each subregion by arrangement pitch S
Beginning central point, possible central point can fall on edge or noise, to solve this problem, according to the present embodiment, calculate initial
The Grad of central point and pixel in its eight neighborhood, chooses the minimum pixel of Grad as the initial center of the subregion
Point.
C. all subregions are traveled through, (e.g., each initial center point is calculated in the initial center vertex neighborhood of each subregion respectively
In 2S*2S regions) pixel to the initial center point distance value, if the distance value be less than the pixel initial distance concentrate
Any distance value, then be included into the subregion by the pixel.
D. after traversal terminates, the subregion is calculated according to the monochrome information and pixel space coordinate of pixel in each subregion
New central point, alternatively, with new central point of the average value of (Y, x, y) of all pixels in subregion as the subregion, and is counted
Calculate the difference of the new central point and initial center point；And
E. the above-mentioned traversal subregion of iteration, (i.e. step c, step d), up to new central point the step of calculate new central point
It is less than first threshold with the difference of a upper central point, cluster terminates.
After cluster terminates, can there are the base point and this subregion in some subregions not in a region, therefore,
Also need to confirm again the connectedness of each subregion after cluster, it is ensured that one region unit of subregion mark after each cluster.Specifically
Way is：Judge whether each super-pixel subregion that cluster is obtained connects, when judging that a certain super-pixel subregion is not connected, just will
The subregion is included into its adjacent sectors, to ensure the connectedness of each subregion.Alternatively, the size of each super-pixel subregion is counted, if
The super-pixel subregion is less than Second Threshold, then judges that the subregion is not connected.In the present embodiment, Second Threshold by arrangement pitch Lai
It is determined that, it is S/4 such as to choose Second Threshold.
So far, the super-pixel subregion to the first image is completed, then in each super-pixel subregion, according to predetermined condition
Chrominance information is screened, the 3rd image is obtained.Specifically, in each super-pixel subregion, to what is painted through step S210
Pixel, is ranked up by the order of its chrominance information from small to large, then only retains the colourity of the pixel in predetermined interval
Information.Alternatively, predetermined interval is related to pixel sum in correspondence super-pixel subregion.
The purpose for the arrangement is that in order to strengthen the robustness that pixel " is mutually matched " algorithm, preventing the leakage between super-pixel
Color problem.
For example, have 100 pixels in certain super-pixel subregion, wherein there are 60 pixels to be painted through step S210,40 in addition
Pixel also keeps grey states.The UV passages of this 60 pixels are so directed to, by taking U as an example, one can be carried out from small to large
Sequence, it is however generally that, can be by the maximum 5% in the U after sequence, the color of the corresponding points of minimum 5% to removing, such as here
Just erase common 60*5%*2=6 point, therefore only 60-6=54 point is coloured, and 40+6=46 pixel in addition
Point is then no chrominance information, proceeds colouring algorithm below., can be by the channel combined places of UV in actual mechanical process
Reason, only retains the chrominance information that U, V value after those sequences are respectively positioned on the pixel in 5%~95% segment, that is to say, that only
Want to have in U, V passage one it is excessive or too small, then the chrominance information of two passages is all abandoned.
Then in step S230, according to the predetermined method of sampling to not having the pixel of chrominance information in the 3rd image still
Point carries out color filling, all pixels point is all had chrominance information.
According to a kind of implementation of the present invention, on the basis of the monochrome information of the 3rd image, its chrominance information is carried out
Down-sampling obtains at least one down-sampled images at least one times, until the size of down-sampled images is 2*2 pixels.In other words,
The process of down-sampling be exactly image by the high pressure compression process of half-breadth half be recycled to minimum dimension 2*2 process (in compression process,
If pixel count can not be divided exactly by 2, round up).For example, image original size is 1920*1080, then down-sampling process is
1920*1080→960*540→480*270→240*135→120*68→60*34→30*17→15*9→8*5→4*3→
2*2, the corresponding image of above-mentioned each size is referred to as one-level down-sampled images.
During down-sampling, calculate the chrominance information of each down-sampled images and record respective pixel space coordinate.
Alternatively, the colourity of the pixel is tried to achieve according to the chrominance information weighting of pixel in the neighborhood of a pixel four or diagonal four neighborhood
Information.According to the embodiment of the present invention, judged according to the space coordinate of pixel, when row coordinate+row coordinate is even number, root
Chrominance information is tried to achieve according to the pixel and the weighting of four neighborhood pixels；When row coordinate is even number and row coordinate is also even number, root
Chrominance information is tried to achieve according to the pixel and the weighting of diagonal four neighborhoods pixel.
As shown in Figure 3A, coordinate points F=w1F+w2(a1B+a2E+a3G+a4J),
Coordinate points K=w1K+w2(a1F+a2H+a3N+a4P), wherein, w1And w20.5, weight coefficient a can be taken1、a2、a3、a4
Brightness distance with corresponding pixel points is negatively correlated, that is to say, that the point close with Strehl ratio value Y, its chromatic value U, V also can
Relatively, take 1/ such as weight coefficient (| Y1-Y2|α+ ε) or exp (- (Y1-Y2)2/σ2) etc., wherein, Y1-Y2Represent two pictures
The luminance difference of vegetarian refreshments.
After down-sampling terminates, since the down-sampled images of 2*2 Pixel Dimensions, by the opposite sequence of down-sampling to down-sampling figure
As being up-sampled, until returning to the size of the 3rd image.
It is each according to the chrominance information of each down-sampled images and respective pixel spatial coordinates calculation in upsampling process
Up-sample the chrominance information of image.As the corresponding upsampling process of down-sampling process above be 2*2 → 4*3 → 8*5 → 15*9 →
30*17 → 60*34 → 120*68 → 240*135 → 480*270 → 960*540 → 1920*1080, the corresponding figure of above-mentioned each size
Picture referred to as one-level down-sampled images.
During up-sampling, retain the pixel recorded during down-sampling and its respective pixel space coordinate, remaining position
Temporarily mend 0.By (LPF) method as described above, the chrominance information of other pixels is calculated.According to the present invention's
Embodiment, judges according to the space coordinate of pixel, when row coordinate is odd number and row coordinate is also odd number, according to the pixel
Chrominance information is tried to achieve in the diagonal four neighborhoods pixel weighting of point；When row coordinate+row coordinate is odd number, according to the neighborhood of pixel four
Chrominance information is tried to achieve in pixel weighting.
As shown in Figure 3 B, coordinate points F=b1A+b2C+b3I+b4K, coordinate points G=b1C+b2F+b3H+b4K, wherein, weighting system
Number b1、b2、b3、b4It is negatively correlated with the brightness of corresponding pixel points distance, as described above, weight coefficient can take 1/ (| Y1-Y2|α+ε)
Or exp (- (Y1-Y2)2/σ2) etc., wherein, Y1-Y2Represent the luminance difference of two pixels.
From the above mentioned, during whole " down-sampling --- up-sampling " color filling, be using monochrome information Y as
It is oriented to, border can be kept by so doing, it is to avoid color is crossed the border during coloring in.
Finally, in step S240, the brightness of the chrominance information of the corresponding pixel points obtained by filling and the 3rd image is believed
Manner of breathing is merged, the image after being merged.
According to the image co-registration scheme of the present invention, using the monochrome information of the first image as the brightness of final fused images,
The characteristics of black and white camera imaging system noise is few, details is abundant, dynamic contrast is high can be given full play to；And then use super picture
Plain subregion amendment has been painted information, to reduce the spilling of the color in fusion process, while enhancing the robustness of algorithm again；Then
" down-sampling --- up-sampling " process of phase is equivalent to simulating diffusion process of the color in full figure, and using brightness Y for guiding, phase
When introducing border limitation during being spread in color, do not spread transboundary, it is to avoid mistake is painted, the fused images finally given
Not only rich color had been ensure that but also with very high picture quality.
Various technologies described herein can combine hardware or software, or combinations thereof is realized together.So as to the present invention
Method and apparatus, or the process and apparatus of the present invention some aspects or part can take embedded tangible media, such as it is soft
The form of program code (instructing) in disk, CD-ROM, hard disk drive or other any machine readable storage mediums,
Wherein when program is loaded into the machine of such as computer etc, and when being performed by the machine, the machine becomes to put into practice this hair
Bright equipment.
In the case where program code is performed on programmable computers, computing device generally comprises processor, processor
Readable storage medium (including volatibility and nonvolatile memory and/or memory element), at least one input unit, and extremely
A few output device.Wherein, memory is arranged to store program codes；Processor is arranged to according to the memory
Instruction in the described program code of middle storage, performs the image interfusion method of the present invention.
By way of example and not limitation, computer-readable medium includes computer-readable storage medium and communication media.Calculate
Machine computer-readable recording medium includes computer-readable storage medium and communication media.Computer-readable storage medium storage such as computer-readable instruction,
The information such as data structure, program module or other data.Communication media is general modulated with carrier wave or other transmission mechanisms etc.
Data-signal processed passes to embody computer-readable instruction, data structure, program module or other data including any information
Pass medium.Any combination above is also included within the scope of computer-readable medium.
It should be appreciated that in order to simplify the disclosure and help to understand one or more of each inventive aspect, it is right above
The present invention exemplary embodiment description in, each feature of the invention be grouped together into sometimes single embodiment, figure or
In person's descriptions thereof.However, the method for the disclosure should be construed to reflect following intention：I.e. claimed hair
The bright feature more features required than being expressly recited in each claim.More precisely, as the following claims
As book reflects, inventive aspect is all features less than single embodiment disclosed above.Therefore, it then follows specific real
Thus the claims for applying mode are expressly incorporated in the embodiment, wherein each claim is used as this hair in itself
Bright separate embodiments.
Those skilled in the art should be understood the module or unit or group of the equipment in example disclosed herein
Part can be arranged in equipment as depicted in this embodiment, or alternatively can be positioned at and the equipment in the example
In different one or more equipment.Module in aforementioned exemplary can be combined as a module or be segmented into addition multiple
Submodule.
Those skilled in the art, which are appreciated that, to be carried out adaptively to the module in the equipment in embodiment
Change and they are arranged in one or more equipment different from the embodiment.Can be the module or list in embodiment
Member or component be combined into a module or unit or component, and can be divided into addition multiple submodule or subelement or
Sub-component.In addition at least some in such feature and/or process or unit exclude each other, it can use any
Combination is disclosed to all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so to appoint
Where all processes or unit of method or equipment are combined.Unless expressly stated otherwise, this specification (including adjoint power
Profit is required, summary and accompanying drawing) disclosed in each feature can or similar purpose identical, equivalent by offer alternative features come generation
Replace.
The present invention is also disclosed in the lump：
A8, the method as any one of A3-7, in addition to step：According to the total pixel number of the first image and setting
The number of partitions calculates arrangement pitch.
A9, the method as described in A8, wherein,
Arrangement pitchWherein, N is the total pixel number of the first image, and k is the number of partitions set.
A10, the method as any one of A5-9, wherein Second Threshold are calculated by the arrangement pitch and obtained.
A11, the method as any one of A1-10, wherein, chrominance information is screened according to predetermined condition
The step of three images, includes：To the pixel with chrominance information, sorted by the order of chrominance information from small to large；And go
The chrominance information of pixel in addition in predetermined interval.
A12, the method as described in A11, wherein, predetermined interval is related to pixel sum in correspondence super-pixel subregion.
A13, the method as any one of A1-12, wherein, according to the predetermined method of sampling in the 3rd image still not
The step of pixel with chrominance information carries out color filling includes：On the basis of the monochrome information of the 3rd image, to its color
Degree information carries out down-sampling at least one times and obtains at least one down-sampled images, until the size of down-sampled images is 2*2 pixels,
During down-sampling, calculate the chrominance information of each down-sampled images and record respective pixel space coordinate；And from 2*2 pictures
The down-sampled images of plain size start, and down-sampled images are up-sampled by the opposite sequence of down-sampling, until returning to the 3rd
The size of image, in upsampling process, according to the chrominance information of each down-sampled images and respective pixel spatial coordinates calculation
The chrominance information of each up-sampling image.
A14, the method as described in A13, wherein, calculate the chrominance information of each down-sampled images or calculate each up-sampling
The step of chrominance information of image, includes：According to the chrominance information weighting of pixel in the neighborhood of a pixel four or diagonal four neighborhood
The chrominance information of the pixel is tried to achieve, wherein, the brightness distance of weight coefficient and corresponding pixel points is negatively correlated.
A15, the method as any one of A1-14, wherein, chrominance information includes the first colour difference information and the second aberration
Information.
A16, the method as any one of A1-15, wherein, the first image is gray level image, and the second image is
Coloured image.
Although in addition, it will be appreciated by those of skill in the art that some embodiments described herein include other embodiments
In included some features rather than further feature, but the combination of the feature of be the same as Example does not mean in of the invention
Within the scope of and form different embodiments.For example, in the following claims, times of embodiment claimed
One of meaning mode can be used in any combination.
In addition, be described as herein can be by the processor of computer system or by performing for some in the embodiment
Method or the combination of method element that other devices of the function are implemented.Therefore, with for implementing methods described or method
The processor of the necessary instruction of element forms the device for implementing this method or method element.In addition, device embodiment
Element described in this is the example of following device：The device is used to implement as in order to performed by implementing the element of the purpose of the invention
Function.
As used in this, unless specifically stated so, come using ordinal number " first ", " second ", " the 3rd " etc.
Description plain objects are merely representative of the different instances for being related to similar object, and are not intended to imply that the object being so described must
Must have the time it is upper, spatially, in terms of sequence or given order in any other manner.
Although describing the present invention according to the embodiment of limited quantity, above description, the art are benefited from
It is interior it is clear for the skilled person that in the scope of the present invention thus described, it can be envisaged that other embodiments.Additionally, it should be noted that
The language that is used in this specification primarily to readable and teaching purpose and select, rather than in order to explain or limit
Determine subject of the present invention and select.Therefore, in the case of without departing from the scope and spirit of the appended claims, for this
Many modifications and changes will be apparent from for the those of ordinary skill of technical field.For the scope of the present invention, to this
The done disclosure of invention is illustrative and not restrictive, and it is intended that the scope of the present invention be defined by the claims appended hereto.
Claims (10)
1. a kind of image interfusion method, methods described is used to handle while the first image and the second figure that are shot for same object
Picture, wherein described first image have monochrome information, and the second image has chrominance information, and methods described includes step：
To each pixel in described first image：
Search whether exist and its content identical pixel in second image；
If in the presence of the chrominance information of the pixel in the second image found to be assigned to the respective pixel in the first image
Point；
Super-pixel subregion is carried out to described first image according to monochrome information, in each super-pixel subregion, according to predetermined condition
Chrominance information is screened, the 3rd image is obtained；
According to the predetermined method of sampling to still the pixel without chrominance information carries out color filling in the 3rd image, make to own
Pixel all has chrominance information；And
The monochrome information of the chrominance information of corresponding pixel points and the 3rd image is blended, the image after being merged.
2. it is the method for claim 1, wherein described according to step of the monochrome information to the first image progress super-pixel subregion
Suddenly include：
Clustering processing is carried out using monochrome information and the image of pixel space coordinate pair first, multiple super-pixel subregions are obtained.
3. method as claimed in claim 2, wherein, the step of carrying out clustering processing to the first image includes：
The distance between each pixel value in the first image is calculated, each pixel is obtained to the initial distance collection of other pixels；
According to the number of partitions calculated permutations interval of setting, and according to the initial center point of each subregion of arrangement pitch generation；
All subregions are traveled through, distance of the pixel to the initial center point in the initial center vertex neighborhood of each subregion is calculated respectively
Value, if the distance value is less than any distance value that the pixel initial distance is concentrated, the subregion is included into by the pixel；
After traversal terminates, the new center of the subregion is calculated according to the monochrome information and pixel space coordinate of pixel in each subregion
Point, and calculate the difference of the new central point and initial center point；And
The step of above-mentioned traversal subregion of iteration, calculating new central point, until the difference of new central point and a upper central point is small
In first threshold, cluster terminates.
4. method as claimed in claim 3, wherein, after the step of cluster terminates, in addition to：
Judge whether the super-pixel subregion that cluster is obtained connects；And
If judging, a certain super-pixel subregion is not connected, and the subregion is included into its adjacent sectors.
5. method as claimed in claim 4, wherein, it is described to judge that the step of whether super-pixel subregion connects includes：
Count the size of each super-pixel subregion；And
If the super-pixel subregion is less than Second Threshold, judge that the subregion is not connected.
6. the method as any one of claim 3-5, wherein, the initial center that each subregion is generated according to arrangement pitch
The step of point, also includes：
Initial center point and the Grad of pixel in its eight neighborhood are calculated, the minimum pixel of Grad is chosen and is used as the subregion
Initial center point.
7. the method as any one of claim 3-6, in addition to step：
The distance value of described two pixels is obtained according to the brightness distance between two pixels and physical distance weighted sum.
8. a kind of mobile device, including：
Dual camera imaging system, suitable for gathering the first image and the second image simultaneously；
One or more processors；
Memory；
One or more programs, wherein one or more of program storages are in the memory and are configured as by described one
Individual or multiple computing devices, one or more of programs include being used to perform according in claim 1-7 methods describeds
The instruction of either method.
9. mobile device as claimed in claim 8, wherein, the dual camera imaging system includes a black and white camera imaging
System and a colour imagery shot imaging system.
10. a kind of computer-readable recording medium for storing one or more programs, one or more of programs include instruction,
The instruction is when computing device so that the computing device is according in the method as described in claim 1-7
Either method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710260280.3A CN107038695A (en) | 2017-04-20 | 2017-04-20 | A kind of image interfusion method and mobile device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710260280.3A CN107038695A (en) | 2017-04-20 | 2017-04-20 | A kind of image interfusion method and mobile device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107038695A true CN107038695A (en) | 2017-08-11 |
Family
ID=59535821
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710260280.3A Pending CN107038695A (en) | 2017-04-20 | 2017-04-20 | A kind of image interfusion method and mobile device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107038695A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948652A (en) * | 2017-11-21 | 2018-04-20 | 青岛海信电器股份有限公司 | A kind of method and apparatus for carrying out image conversion |
| CN108389165A (en) * | 2018-02-02 | 2018-08-10 | 成都西纬科技有限公司 | A kind of image de-noising method |
| CN108470327A (en) * | 2018-03-27 | 2018-08-31 | 成都西纬科技有限公司 | Image enchancing method, device, electronic equipment and storage medium |
| CN111435972A (en) * | 2019-01-15 | 2020-07-21 | 杭州海康威视数字技术股份有限公司 | Image processing method and device |
| CN112200757A (en) * | 2020-09-29 | 2021-01-08 | 北京灵汐科技有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN113034369A (en) * | 2021-04-07 | 2021-06-25 | 北京佰才邦技术股份有限公司 | Image generation method and device based on multiple cameras and computer equipment |
| CN113129400A (en) * | 2021-03-17 | 2021-07-16 | 维沃移动通信有限公司 | Image processing method, image processing device, electronic equipment and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105827965A (en) * | 2016-03-25 | 2016-08-03 | 维沃移动通信有限公司 | Image processing method based on mobile terminal and mobile terminal |
| CN106462986A (en) * | 2014-06-12 | 2017-02-22 | 汤姆逊许可公司 | Methods and systems for color processing of digital images |
| EP3328065A1 (en) * | 2016-11-29 | 2018-05-30 | Guangdong Oppo Mobile Telecommunications Corp., Ltd | Dual-camera focusing method and apparatus, and terminal device |
-
2017
- 2017-04-20 CN CN201710260280.3A patent/CN107038695A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106462986A (en) * | 2014-06-12 | 2017-02-22 | 汤姆逊许可公司 | Methods and systems for color processing of digital images |
| CN105827965A (en) * | 2016-03-25 | 2016-08-03 | 维沃移动通信有限公司 | Image processing method based on mobile terminal and mobile terminal |
| EP3328065A1 (en) * | 2016-11-29 | 2018-05-30 | Guangdong Oppo Mobile Telecommunications Corp., Ltd | Dual-camera focusing method and apparatus, and terminal device |
Non-Patent Citations (4)
| Title |
|---|
| _JOHN_TIAN_: "opencv-图像金字塔-上采样-下采样", 《CSDN博客HTTPS://BLOG.CSDN.NET/TIANZHAIXING2013/ARTICLE/DETAILS/8752100》 * |
| WYROVER: "book-code/OpenCV图像处理编程实例/chapter4/4-4.cpp", 《GITHUB,HTTPS://GITHUB.COM/WYROVER/BOOK-CODE/BLOB/7F4883D9030D553BC6BCFA3DA685E34789839900/OPENCV》 * |
| 刘俊: "一种快速的超像素分割方法", 《电子科技》 * |
| 科尼利厄斯等: "《光电系统分析与设计》", 31 July 2015, 国防工业出版社 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948652A (en) * | 2017-11-21 | 2018-04-20 | 青岛海信电器股份有限公司 | A kind of method and apparatus for carrying out image conversion |
| CN107948652B (en) * | 2017-11-21 | 2021-04-13 | 海信视像科技股份有限公司 | Method and equipment for image conversion |
| CN108389165A (en) * | 2018-02-02 | 2018-08-10 | 成都西纬科技有限公司 | A kind of image de-noising method |
| CN108470327A (en) * | 2018-03-27 | 2018-08-31 | 成都西纬科技有限公司 | Image enchancing method, device, electronic equipment and storage medium |
| CN108470327B (en) * | 2018-03-27 | 2022-05-17 | 成都西纬科技有限公司 | Image enhancement method and device, electronic equipment and storage medium |
| CN111435972A (en) * | 2019-01-15 | 2020-07-21 | 杭州海康威视数字技术股份有限公司 | Image processing method and device |
| CN111435972B (en) * | 2019-01-15 | 2021-03-23 | 杭州海康威视数字技术股份有限公司 | Image processing method and device |
| CN112200757A (en) * | 2020-09-29 | 2021-01-08 | 北京灵汐科技有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| WO2022068584A1 (en) * | 2020-09-29 | 2022-04-07 | 北京灵汐科技有限公司 | Image processing method and apparatus, computer device, and storage medium |
| CN113129400A (en) * | 2021-03-17 | 2021-07-16 | 维沃移动通信有限公司 | Image processing method, image processing device, electronic equipment and readable storage medium |
| CN113129400B (en) * | 2021-03-17 | 2023-02-24 | 维沃移动通信有限公司 | Image processing method, image processing device, electronic equipment and readable storage medium |
| CN113034369A (en) * | 2021-04-07 | 2021-06-25 | 北京佰才邦技术股份有限公司 | Image generation method and device based on multiple cameras and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107038695A (en) | A kind of image interfusion method and mobile device | |
| CN107820020A (en) | Method of adjustment, device, storage medium and the mobile terminal of acquisition parameters | |
| CN107087107A (en) | Image processing apparatus and method based on dual camera | |
| US20130114894A1 (en) | Blending of Exposure-Bracketed Images Using Weight Distribution Functions | |
| CN103973978A (en) | Method and electronic device for achieving refocusing | |
| CN105323497A (en) | Constant bracket for high dynamic range (cHDR) operations | |
| CN107395898A (en) | A kind of image pickup method and mobile terminal | |
| CN108269280A (en) | The processing method and mobile terminal of a kind of depth image | |
| CN110493527A (en) | Main body focusing method, device, electronic equipment and storage medium | |
| CN108924427A (en) | A kind of video camera focus method, device and video camera | |
| CN108335323A (en) | A kind of weakening method and mobile terminal of image background | |
| CN110084204B (en) | Image processing method and device based on target object posture and electronic equipment | |
| CN114096994A (en) | Image alignment method and device, electronic equipment and storage medium | |
| WO2017215194A1 (en) | Image processing method and device and storage medium thereof | |
| CN111353965A (en) | Image restoration method, device, terminal and storage medium | |
| CN114390201A (en) | Focusing method and device thereof | |
| US10481774B2 (en) | Area-dependent image enhancement | |
| CN109658360B (en) | Image processing method and device, electronic equipment and computer storage medium | |
| JP2021189527A (en) | Information processing device, information processing method, and program | |
| JP5381498B2 (en) | Image processing apparatus, image processing program, and image processing method | |
| US20130058568A1 (en) | Image processing apparatus, image processing method, and non-transitory computer readable medium | |
| US9886767B2 (en) | Method, apparatus and computer program product for segmentation of objects in images | |
| CN106402717B (en) | A kind of AR control method for playing back and intelligent desk lamp | |
| JP2016167258A (en) | Method, device and computer program product of reducing chromatic aberration in deconvolution images | |
| CN109961083A (en) | For convolutional neural networks to be applied to the method and image procossing entity of image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170811 |