CN106851092A - A kind of infrared video joining method and device - Google Patents
A kind of infrared video joining method and device Download PDFInfo
- Publication number
- CN106851092A CN106851092A CN201611259450.8A CN201611259450A CN106851092A CN 106851092 A CN106851092 A CN 106851092A CN 201611259450 A CN201611259450 A CN 201611259450A CN 106851092 A CN106851092 A CN 106851092A
- Authority
- CN
- China
- Prior art keywords
- image
- characteristic point
- video
- module
- video camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000005304 joining Methods 0.000 title claims abstract description 12
- 239000011159 matrix material Substances 0.000 claims abstract description 42
- 238000012937 correction Methods 0.000 claims abstract description 18
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 17
- 239000000284 extract Substances 0.000 claims abstract description 8
- 239000013598 vector Substances 0.000 claims description 23
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 10
- 238000002156 mixing Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 4
- 238000003860 storage Methods 0.000 claims description 4
- 230000008901 benefit Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 238000012544 monitoring process Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000005855 radiation Effects 0.000 description 6
- 230000035945 sensitivity Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000005286 illumination Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 238000012897 Levenberg–Marquardt algorithm Methods 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000007323 disproportionation reaction Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/682—Vibration or motion blur correction
- H04N23/684—Vibration or motion blur correction performed by controlling the image sensor readout, e.g. by controlling the integration time
- H04N23/6845—Vibration or motion blur correction performed by controlling the image sensor readout, e.g. by controlling the integration time by combination of a plurality of images sequentially taken
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of infrared video joining method and device, methods described includes：Extract the characteristic point of each image in infrared video, the matching characteristic point of image two-by-two is set up to list, obtain the eigenmatrix and attitude matrix of video camera, level is carried out to image and stretches treatment and exposure compensating treatment, during each image projected into same spheric coordinate system, seamless spliced, the spliced panorama sketch of generation is carried out with multilayer hybrid algorithm in overlapping region.The present invention is operated and real-time exposure compensating by carrying out brightness correction to video, realizes the multichannel infrared video panoramic mosaic with automatic straightening and real-time exposure compensating function.
Description
Technical field
The present invention relates to technical field of video processing, more particularly to a kind of infrared video joining method and device.
Background technology
At home on many fields of security protection, being widely used for thermal camera carries out round-the-clock monitoring, but due to single channel
The field range of video camera is narrower, and the scope of monitoring is typically expanded by revolving-turret, and wheel is swept not only bad for sight for a long time
Examine, and there is monitoring blind area, very big potential safety hazard is brought to security protection industry, pass through to use at present and be capable of the red of large scene
Outer real time monitoring apparatus are narrow and small to solve the problems, such as monitoring range.
For the video that infrared camera is gathered, generally require to carry out splicing to the video of multi-path camera collection,
Present technology employs various algorithms of different in splicing and calculates complicated, and the operand for calculating in real time is huge, and video
Splicing is high to requirement of real-time for picture splicing, and the splicing meter of 25 frame N roads images must be completed in 1 second
Calculate, being otherwise easily lost frame causes picture not smooth or discontinuous, or even internal memory overflows the deadlock for causing machine.
Have in the prior art and the scheme of video-splicing of carrying out is spliced by bilinearity, it uses the bilinear interpolation method to carry out
Splicing, once linear interpolation calculation is carried out in x and y both directions respectively, but this scheme, occur in overlapping region after splicing
Ghost phenomena, it is rough that zigzag occurs in curve.
The content of the invention
The embodiment of the present invention proposes a kind of infrared video joining method, and methods described includes：
The characteristic point of each image in infrared video is extracted, the descriptor of each characteristic point is calculated；
Descriptor according to the characteristic point and the matching of stochastical sampling unification algorism image two-by-two, set up described in image two-by-two
Matching characteristic point to list；
The affine matrix of video camera is calculated according to the matching characteristic point, the affine matrix according to the video camera is with most
Small intrinsic method solves upward vector, and carrying out level to described image according to the upward vector stretches treatment；
The summation of all images light intensity and gain product difference in overlapping region in the video is calculated, and solution is obtained
The gain coefficient of each image, compensation is exposed according to the gain coefficient to each image；
During each image projected into same spheric coordinate system, (Multi-Band is mixed with multilayer in overlapping region
Blending) algorithm carries out seamless spliced, the spliced panorama sketch of generation；
Wherein, exposure compensating is carried out after stretching treatment to described image level.
The embodiment of the present invention additionally provides a kind of infrared video splicing apparatus, and described device includes rectification module, matching meter
Calculate module, level and stretch module, exposure compensation module and concatenation module；
The rectification module, the compensation coefficient for calculating each pixel, according to the compensation coefficient in video
Each image carries out correction pretreatment, and extracts the characteristic point of each image in video；
The matching primitives module, the descriptor of each characteristic point for obtaining, root are extracted for calculating the extraction module
According to descriptor and stochastical sampling unification algorism the matching image two-by-two of the characteristic point, set up described in image two-by-two matching characteristic
Point is to list；
The level stretches module, and the matching characteristic point for being obtained according to the matching primitives module is calculated to be taken the photograph
The affine matrix of camera, the affine matrix according to the video camera solves upward vector with minimum intrinsic method, according to it is described to
Upper vector carries out level and stretches treatment to described image；
The exposure compensation module, for calculating all images light intensity and gain product in overlapping region in the video
The summation of difference, and the gain coefficient for obtaining each image is solved, each image is exposed according to the gain coefficient
Light is compensated；
The concatenation module, for the level to be stretched into every width figure that module and exposure compensation module treatment are obtained
In as projecting to same spheric coordinate system, carried out with multilayer hybrid algorithm in overlapping region seamless spliced, generated spliced
Panorama sketch.
Have the beneficial effect that：
The video-splicing scheme that the present invention is provided, by carrying out corrective operations and real-time exposure compensating to video, for many
The problem of the strong and weak different and skewness of brightness of image caused by sensor sensitivity difference has been carried out certainly between portion's thermal camera
Dynamic correction, changes over time the uneven problem of caused brightness of image and enters simultaneously for intensity of illumination and object radiation infrared intensity
Go exposure compensating, the image seamless of splicing, without ghost image, splicing effect is optimal, is truly realized with automatic straightening and reality
When exposure compensating function multichannel infrared video panoramic mosaic.
Brief description of the drawings
Specific embodiment of the invention is described below with reference to accompanying drawings, wherein：
Fig. 1 shows infrared video joining method flow chart in the embodiment of the present invention one；
Fig. 2 shows infrared video joining method flow chart in the embodiment of the present invention two；
Two images are divided using acceleration robust features (SURF) algorithm during Fig. 3 a and Fig. 3 b show the embodiment of the present invention two
The characteristic point schematic diagram indescribably got；
Fig. 4 a show the Feature Points Matching figure obtained using the search of KD trees in the embodiment of the present invention two；
Fig. 4 b show the Feature Points Matching figure of maximum quantity stochastical sampling in the embodiment of the present invention two unanimously after treatment；
Fig. 5 a show the straight preceding panorama sketch of the reclaimed water horizontal drawing of the embodiment of the present invention two；
Fig. 5 b show the reclaimed water horizontal drawing of the embodiment of the present invention two it is straight after panorama sketch；
Fig. 6 shows the design sketch after exposure compensating treatment in the embodiment of the present invention two；
Fig. 7 a show the panorama sketch before brightness correction in the embodiment of the present invention two；
Fig. 7 b show the panorama sketch after brightness correction in the embodiment of the present invention two；
Fig. 8 shows the structural representation of infrared video splicing apparatus in the embodiment of the present invention three；
Fig. 9 shows the splicing effect figure of infrared video splicing apparatus in the embodiment of the present invention three.
Specific embodiment
In order that technical scheme and advantage become more apparent, below in conjunction with accompanying drawing to of the invention exemplary
Embodiment is described in more detail, it is clear that described embodiment is only a part of embodiment of the invention, rather than
The exhaustion of all embodiments.And in the case where not conflicting, the feature in embodiment and embodiment in this explanation can be mutual
It is combined.
In actual use, because the sensitivity of different thermal cameras probe has differences, and different optical frames
The imaging of head there is also difference, and same scene is shot, and between the image that different cameras are collected, its brightness is existed
Difference, or even inside a video camera acquired image, there is the phenomenon of brightness irregularities in upper and lower, left and right directions, this
A little factors greatly affect the whole structure of panorama, and the present invention carried out correction pretreatment before video-splicing to original image
Operation, enable to the infrared brightness uniformity in video.
When infrared monitor in real time is carried out, with the change of sunlight exposure intensity and angle, object reflection in monitor area
Intensity with infrared radiation is slowly varying, and the brightness of camera acquisition to image also changing therewith, therefore the present invention passes through
Need exposure compensating incessantly to process in splicing, adjust the gain coefficient of each road image, view picture panorama sketch can be made
Brightness can uniform throughout integrally keep basically identical.
The SD image of single channel video camera 25 frames of collection per second, is 704576 per two field picture size, and thermal camera is through puppet
After color processing, each pixel needs the RGB component of 3 bytes, and the data volume of N roads camera transmissions is N times of single channel, therefore
No. 6 video camera splicings need to process the data of 1.8 hundred million byte per second.Various algorithms of different are employed in splicing and calculates multiple
Miscellaneous, the operand for calculating in real time is huge.
Embodiment one
As shown in figure 1, the present invention proposes a kind of infrared video joining method, methods described includes：
Step 101：Extract the characteristic point of each image in video；
Step 102：The descriptor of each characteristic point is calculated, descriptor and stochastical sampling unification algorism according to characteristic point
Image two-by-two is matched, the matching characteristic point of image two-by-two is set up to list；
Step 103：The affine matrix of video camera is calculated according to matching characteristic point, the affine matrix according to video camera is with most
Small intrinsic method solves upward vector, and carrying out level to image according to upward vector stretches treatment；
Step 104：The summation of all images light intensity and gain product difference in overlapping region in video is calculated, and is solved
The gain coefficient of each image is obtained, compensation is exposed to each image according to gain coefficient；
Step 105：During each image projected into same spheric coordinate system, in overlapping region, multilayer hybrid algorithm enters
Row is seamless spliced, generates spliced panorama sketch；
Wherein, in practical application, exposure compensating is carried out after treatment can first being stretched to described image level, or to the figure
As the first laggard water-filling horizontal drawing of exposure compensating is directly processed, that is to say, that above-mentioned steps 103 and step 104 are without sequencing.
The infrared video joining method that the present invention is provided, proposition carries out the side of corrective operations and real-time exposure compensating to video
Formula, for the problem of the strong and weak different and skewness of brightness of image caused by sensor sensitivity difference between multi-section thermal camera
Automatic straightening is carried out, it is uneven to change over time caused brightness of image simultaneously for intensity of illumination and object radiation infrared intensity
Problem carried out exposure compensating, be truly realized the multichannel infrared video with automatic straightening and real-time exposure compensating function complete
Scape splices.
Embodiment two
Referring to Fig. 2, a kind of infrared video joining method is the embodiment of the invention provides, methods described includes：
Step 201：Correction pretreatment is carried out to each image in video；
Due to the difference of sensitivity between sensor, the difference of monitoring background environment, and various object radiations and reflection are red
The power of outside line ability, these factors can cause not go the same way contrast between image and brightness has very big difference, or even picture occur
The uneven phenomenon in face, therefore the embodiment of the present invention pre-processed to original image before splicing.
For the problem of infrared brightness disproportionation, the present invention carries out brightness correction to each pixel, and the image after correction is bright
Spend for compensation coefficient is multiplied by original image brightness, correction formula is logarithm tangent type function：
(1)
It is wherein compensation coefficient, represents pixel position in the picture, represent Gaussian Profile center position, represents
In variance both horizontally and vertically, variance is bigger, and adjustment region is bigger.The center of image is generally set to, by original graph
Image brightness is multiplied by the brightness of image after compensation coefficient A is corrected.The brightness that Fig. 7 a and Fig. 7 b is shown respectively is for details, reference can be made to rectify
Panorama sketch after just preceding panorama sketch and brightness correction.
In addition, in order to improve arithmetic speed, the present invention sets the caching of a piece and image same size, in program initialization
When precompute the compensation coefficient of each pixel and be stored in caching, every two field picture below need to only be multiplied by compensation coefficient and nothing
Need to compute repeatedly, drastically increase arithmetic speed.Wherein, the compensation coefficient of each pixel is different, the correction system of pixel
Number and the distance dependent of Gaussian Profile central point, in practical application, are determined strong by (x-xc) and (y-yc) in above-mentioned formula 1
Positive coefficient.
Step 202：Extract the characteristic point of each image in infrared video；
Specifically, the present invention extracts the characteristic point of each image using rapid robust feature (SURF) algorithm, first with not
Convolution is carried out to image with the second-order partial differential coefficient of variance Gaussian function and obtains integral image, calculate each pixel Hessian of image
Determinant of a matrix value, to the extreme point more than thresholding as characteristic point, then calculates the principal direction of the point, and rotation original graph is arrived
64 dimension descriptor vectors and point coordinates are stored by 64 dimension descriptor vectors of principal direction generation characteristic point, are next step
Images match provides important evidence.
If Fig. 3 a and Fig. 3 b are the characteristic point schematic diagram that two images are extracted respectively using SURF algorithm.
Step 203：The matching characteristic point of image two-by-two is set up to list；
Specifically included in the step：The descriptor of each characteristic point is calculated, the descriptor according to characteristic point and is adopted at random
Consistent (RANSAC) algorithmic match of sample image two-by-two, set up described in image two-by-two matching characteristic point to list.
The principal direction of the characteristic point being calculated using above-mentioned steps 202, principal direction generation feature is rotated to by original image
64 dimension descriptor vectors of point, the feature point description symbol extracted using SURF algorithm sets up KD trees, and KD trees are k-dimensional
The abbreviation of tree, is a kind of data structure in segmentation k dimension datas space, then with BBF (Best-Bin-First) K-NN search
The characteristic point that method fast search is matched between going out two width figures, sets up the matching characteristic point between two width figures to list.
Preferably, for a small amount of erroneous matching situation, stochastical sampling of the present invention also using maximum quantity is consistent
(RANSAC) algorithm rejects erroneous matching, to obtain and match consistent characteristic point pair on maximum quantity.If Fig. 4 a are to be searched using KD trees
The Feature Points Matching figure that rope is obtained, Fig. 4 b are the Feature Points Matching figure after maximum quantity stochastical sampling is unanimously processed.
Step 204：The affine matrix of video camera is calculated according to matching characteristic point；
Specifically, matching characteristic point calculates the affine matrix of video camera in the step, specifically include：Obtain matching characteristic
The image center of point, obtains image in the horizontal direction or the focal length of vertical direction；Calculated according to image center and focal length
The eigenmatrix of video camera；The Eulerian angles of the image of matching characteristic point are obtained, the attitude matrix of video camera is obtained according to Eulerian angles；
The attitude matrix of eigenmatrix and video camera according to video camera, obtains the affine matrix of video camera.
Characteristic point according to matching is calculated including the camera matrix including eigenmatrix and attitude matrix to parameter, such as
Fruit does not consider under the conditions of camera lens deformity and optical axis deviation center etc. that image center is fixed as (u0,v0), both horizontally and vertically
Focal length be unanimously fi, then eigenmatrix be expressed as
If do not consider camera translation, three Eulerian angles of attitude matrix describe video camera rotation, then attitude matrix is
According to video camera affine matrixCalculate the position that k-th characteristic point is mapped to figure i from figure jThen subtract each other with the position of the figure i points and obtain residual errorFinally use Levenberg-
Marquardt algorithms are iterated adjustment camera parameters makes total residual error minimum, obtains optimal camera matrix parameter.
Step 205：Level is carried out to image and stretches treatment；
In the step, the affine matrix first according to video camera solves upward vector with minimum intrinsic method, then basis
Upward vector carries out level and stretches treatment to described image.Wherein, level is carried out to image according to upward vector and stretches treatment, had
Body includes：The spin matrix of the overall situation is calculated according to upward vector meter；The attitude matrix of each video camera is multiplied by spin matrix, is obtained
To the panoramic picture for stretching in the horizontal direction.
In practical application, if directly spliced according to camera matrix (KR), panorama sketch will occur heaving of the sea
Phenomenon, it is necessary to the X-axis of rotary camera make Y-axis keep straight up.
Assuming that amount of images is n, i-th camera horizon axial vector is Xi, then upward vector u must is fulfilled for condition：
(4)
Solved using least square method and obtain upward vector u, then the spin matrix of the overall situation, each video camera are calculated with u
Attitude matrix be multiplied by spin matrix, panoramic picture is stretched in the horizontal direction.Fig. 5 a are the panorama sketch before level is stretched, figure
5b is the panorama sketch after level is stretched.
Step 206：Compensation is exposed to image；
In the step, the summation of all images light intensity and gain product difference in overlapping region in video is calculated first,
And the gain coefficient for obtaining each image is solved, compensation is then exposed to each image according to gain coefficient.
All images light intensity in overlapping region is with the summation of gain product difference
(5)
Wherein it is the region of adjacent two images overlap, and is the gain of i-th and j width images, and is i-th and j width images
Average intensity.In order that obtaining error e minimum, the gain coefficient g of each image is solved using least square method, then will be every
The light intensity of width image is multiplied by corresponding gain coefficient, realizes exposure compensating function.
Elapse over time, the angle of sunlight changes, each brightness of image can significantly change, it is necessary to constantly
Ground adjust gain coefficient.Therefore, at interval of the gain coefficient for recalculating each image for several seconds, the automatic exposure of video-splicing is realized
Light compensation function, the panorama sketch brightness throughout after compensation is basically identical.Effect after exposure compensating treatment is as shown in Figure 6.
Step 207：Multilayer mixing concatenation is carried out to overlapping region, spliced panorama sketch is generated；
Specifically, during each image projected into same spheric coordinate system, (Multi- is mixed with multilayer in overlapping region
Band Blending) algorithm carries out seamless spliced, generates spliced panorama sketch.
Convolution is carried out using the Gaussian function of various criterion difference to image intensity in overlapping region and weight, multilayer is obtained
The blurred picture and weight of (difference), the mixing light intensity of pixel is in overlapping region
(6)
It is wherein the hybrid weight that the i-th width graphics standard difference is, is the mixing light intensity of the i-th width graphics standard difference and standard deviation
Difference：
(7)
In addition, the embodiment of the present invention is also cut out operation to the frame of panorama sketch.
Specifically, unification projects to spheric coordinate system Zhong Mei roads image and produces deformation, and the pitching in panorama sketch
Height is also different, so occurring in that the frame of black in picture, have impact on the overall visual effect of panorama.By contrasting each road
The most imperial palace that picture position obtains panorama connects rectangle, and the width and height that rectangle is then connect according in are cut, and obtain picture
The full panorama sketch being full of.
The infrared video joining method that the present invention is provided, proposition carries out the side of corrective operations and real-time exposure compensating to video
Formula, for the problem of the strong and weak different and skewness of brightness of image caused by sensor sensitivity difference between multi-section thermal camera
Automatic straightening is carried out, it is uneven to change over time caused brightness of image simultaneously for intensity of illumination and object radiation infrared intensity
Problem carried out exposure compensating, be truly realized the multichannel infrared video with automatic straightening and real-time exposure compensating function complete
Scape splices.
Embodiment three
Referring to Fig. 8, a kind of infrared video splicing apparatus is the embodiment of the invention provides, described device includes rectification module
301st, matching primitives module 302, level stretches module 303, exposure compensation module 304 and concatenation module 305；
Rectification module 301, the compensation coefficient for calculating each pixel, according to the compensation coefficient to every in video
Width image carries out correction pretreatment, and extracts the characteristic point of each image in video；
Matching primitives module 302, the descriptor of each characteristic point for obtaining is extracted for calculating rectification module 301, according to
The descriptor and stochastical sampling unification algorism of characteristic point match image two-by-two, set up the matching characteristic point of image two-by-two to list；
Level stretches module 303, and the matching characteristic point for being obtained according to matching primitives module 302 calculates video camera
Affine matrix, the affine matrix according to video camera solves upward vector with minimum intrinsic method, and image is entered according to upward vector
Water-filling horizontal drawing is directly processed；
Exposure compensation module 304, for calculating all images light intensity and gain product difference in overlapping region in video
Summation, and solve and obtain the gain coefficient of each image, compensation is exposed to each image according to gain coefficient；
Concatenation module 305, for level to be stretched into each image that module 303 and the treatment of exposure compensation module 305 are obtained
In projecting to same spheric coordinate system, carried out with multilayer hybrid algorithm in overlapping region seamless spliced, generated spliced complete
The spliced panorama sketch that Jing Tu, such as Fig. 9 show.
In addition, infrared video splicing apparatus also includes memory module, for storing each pixel that rectification module is calculated
The compensation coefficient of point, for other images in video, only need to be multiplied by compensation coefficient and need not compute repeatedly, and drastically increase
Arithmetic speed.
The infrared video splicing apparatus that the present invention is provided, correction behaviour is carried out by rectification module and light compensating module to video
Make and real-time exposure compensating, it is strong and weak different for brightness of image caused by sensor sensitivity difference between multi-section thermal camera and
The problem of skewness has carried out auto brightness correction, is changed over time simultaneously for intensity of illumination and object radiation infrared intensity
The uneven problem of caused brightness of image has carried out exposure compensating, is truly realized with automatic straightening and real-time exposure compensating work(
The multichannel infrared video panoramic mosaic of energy.
It is last it is to be noted that：The above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations.This
The those of ordinary skill in field should be understood：Technical scheme described in foregoing embodiments can be modified, or it is right
Which part technical characteristic carries out equivalent；These modifications are replaced, and the essence of appropriate technical solution is departed from this
Invent the spirit and scope of each embodiment technical scheme.
It should be understood by those skilled in the art that, embodiments of the invention can be provided as method, system or computer program
Product.Therefore, the present invention can be using the reality in terms of complete hardware embodiment, complete software embodiment or combination software and hardware
Apply the form of example.And, the present invention can be used and wherein include the computer of computer usable program code at one or more
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) is produced
The form of product.
The present invention is the flow with reference to method according to embodiments of the present invention, equipment (system) and computer program product
Figure and/or block diagram are described.It should be understood that every first-class during flow chart and/or block diagram can be realized by computer program instructions
The combination of flow and/or square frame in journey and/or square frame and flow chart and/or block diagram.These computer programs can be provided
The processor of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce
A raw machine so that produced for reality by the instruction of computer or the computing device of other programmable data processing devices
The device of the function of being specified in present one flow of flow chart or multiple one square frame of flow and/or block diagram or multiple square frames.
These computer program instructions may be alternatively stored in can guide computer or other programmable data processing devices with spy
In determining the computer-readable memory that mode works so that instruction of the storage in the computer-readable memory is produced and include finger
Make the manufacture of device, the command device realize in one flow of flow chart or multiple one square frame of flow and/or block diagram or
The function of being specified in multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that in meter
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented treatment, so as in computer or
The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in individual square frame or multiple square frames.
, but those skilled in the art once know basic creation although preferred embodiments of the present invention have been described
Property concept, then can make other change and modification to these embodiments.So, appended claims are intended to be construed to include excellent
Select embodiment and fall into having altered and changing for the scope of the invention.
Claims (10)
1. a kind of infrared video joining method, it is characterised in that methods described includes：
The characteristic point of each image in infrared video is extracted, the descriptor of each characteristic point is calculated；
Descriptor according to the characteristic point and the matching of stochastical sampling unification algorism image two-by-two, set up described in two-by-two image
With characteristic point to list；
The affine matrix of video camera is calculated according to the matching characteristic point, the affine matrix most small capital according to the video camera
The method of levying solves upward vector, and carrying out level to described image according to the upward vector stretches treatment；
The summation of all images light intensity and gain product difference in overlapping region in the video is calculated, and solution obtains every width
The gain coefficient of image, compensation is exposed according to the gain coefficient to each image；
During each image projected into same spheric coordinate system, carried out with multilayer hybrid algorithm in overlapping region it is seamless spliced,
Generate spliced panorama sketch；
Wherein, exposure compensating is carried out after treatment is stretched to described image level, or it is flat to the laggard water-filling of described image exposure compensating
Stretch treatment.
2. the method for claim 1, it is characterised in that in the extraction video before the characteristic point of each image, institute
Stating method also includes carrying out brightness correction pretreatment to each image in video, specially：
Brightness of image after correction is multiplied by compensation coefficient for original image brightness,
Wherein, the position for the pixel in the picture, is the center position of each image, is original image in level and hangs down
Nogata to variance.
3. method as claimed in claim 2, it is characterised in that the characteristic point of each image in the extraction video, specific bag
Include：
Convolution is carried out to each image using the second-order partial differential coefficient of multiple variance Gaussian functions, each image integrogram is obtained
Picture；
The determinant of each pixel matrix in the integral image is calculated, the extreme point for exceeding thresholding is carried out as characteristic point
Extract.
4. method as claimed in claim 2, it is characterised in that the descriptor and stochastical sampling one according to the characteristic point
Cause algorithmic match image two-by-two, set up described in image two-by-two matching characteristic point to list, specifically include：
The principal direction of the characteristic point is calculated, original image is rotated into the 64 dimension descriptor arrows that the principal direction obtains characteristic point
Amount；
KD trees are set up using the feature point description symbol for accelerating robust features algorithm to extract；
Go out the characteristic point of images match two-by-two using K-NN search method fast search in the KD trees, set up image two-by-two
Matching characteristic point is to list.
5. method as claimed in claim 2, it is characterised in that the affine square of video camera is calculated according to the matching characteristic point
Battle array, specifically includes：
The image center of the matching characteristic point is obtained, described image is obtained in the horizontal direction or the focal length of vertical direction；
The eigenmatrix of video camera is calculated according to described image central point and the focal length；
The Eulerian angles of the image of the matching characteristic point are obtained, the attitude matrix of the video camera is obtained according to the Eulerian angles；
The attitude matrix of eigenmatrix and the video camera according to the video camera, obtains the affine matrix of the video camera.
6. method as claimed in claim 5, it is characterised in that the affine matrix of the video camera is
Wherein,It is the eigenmatrix of video camera,It is the attitude of video camera
Matrix, (u0,v0) it is image center, fiIt is focal length both horizontally and vertically.
7. method as claimed in claim 2, it is characterised in that the upward vector of basis carries out level and stretches to described image
Treatment, specifically includes：
The spin matrix of the overall situation is calculated according to the upward vector meter；
The attitude matrix of each video camera is multiplied by spin matrix, the panoramic picture for being stretched in the horizontal direction.
8. method as claimed in claim 6, it is characterised in that described to carry out seamless spelling with multilayer hybrid algorithm in overlapping region
Connect, generate spliced panorama sketch and specifically include：
The mixing light intensity of pixel in overlapping region is calculated,
Wherein, wherein the hybrid weight for being for the i-th width graphics standard difference, is the mixed light of the i-th width graphics standard difference and standard deviation
It is strong poor
Multilayer mixing splicing is carried out according to the folded region of mixing light intensity counterweight, spliced panorama sketch is generated.
9. a kind of infrared video splicing apparatus, it is characterised in that described device includes rectification module, matching primitives module, level
Stretch module, exposure compensation module and concatenation module；
The rectification module, the compensation coefficient for calculating each pixel, according to the compensation coefficient to every width in video
Image carries out correction pretreatment, and extracts the characteristic point of each image in video；
The matching primitives module, the descriptor of each characteristic point for obtaining is extracted for calculating the extraction module, according to institute
State descriptor and stochastical sampling unification algorism the matching image two-by-two of characteristic point, set up described in image two-by-two matching characteristic point pair
List；
The level stretches module, and the matching characteristic point for being obtained according to the matching primitives module calculates video camera
Affine matrix, the affine matrix according to the video camera solves upward vector with minimum intrinsic method, according to the upward arrow
Amount carries out level and stretches treatment to described image；
The exposure compensation module, for calculating all images light intensity and gain product difference in overlapping region in the video
Summation, and solve and obtain the gain coefficient of each image, benefit is exposed to each image according to the gain coefficient
Repay；
The concatenation module, for the level to be stretched into each image throwing that module and exposure compensation module treatment are obtained
Shadow is seamless spliced in same spheric coordinate system, being carried out with multilayer hybrid algorithm in overlapping region, generates spliced panorama
Figure.
10. device as claimed in claim 9, it is characterised in that described device also includes the storage of the connection rectification module
Module, the compensation coefficient for storing each pixel that the rectification module is calculated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611259450.8A CN106851092B (en) | 2016-12-30 | 2016-12-30 | A kind of infrared video joining method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611259450.8A CN106851092B (en) | 2016-12-30 | 2016-12-30 | A kind of infrared video joining method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106851092A true CN106851092A (en) | 2017-06-13 |
| CN106851092B CN106851092B (en) | 2018-02-09 |
Family
ID=59113804
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611259450.8A Expired - Fee Related CN106851092B (en) | 2016-12-30 | 2016-12-30 | A kind of infrared video joining method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106851092B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107257443A (en) * | 2017-07-24 | 2017-10-17 | 中科创达软件科技（深圳）有限公司 | The method and its device, terminal device of a kind of anti-vignetting of stitching image |
| CN107341827A (en) * | 2017-07-27 | 2017-11-10 | 腾讯科技（深圳）有限公司 | A kind of method for processing video frequency, device and storage medium |
| CN109272442A (en) * | 2018-09-27 | 2019-01-25 | 百度在线网络技术（北京）有限公司 | Processing method, device, equipment and the storage medium of panorama spherical surface image |
| CN109981985A (en) * | 2019-03-29 | 2019-07-05 | 上海智觅智能科技有限公司 | A kind of continuous stitching algorithm of double vision frequency |
| CN110800283A (en) * | 2018-12-29 | 2020-02-14 | 深圳市大疆创新科技有限公司 | Panoramic image generation method, panoramic image generation device and unmanned aerial vehicle |
| CN110796597A (en) * | 2019-10-10 | 2020-02-14 | 武汉理工大学 | Vehicle-mounted all-round-view image splicing device based on space-time compensation |
| CN111445416A (en) * | 2020-03-30 | 2020-07-24 | 南京泓众电子科技有限公司 | Method and device for generating high-dynamic-range panoramic image |
| CN113222878A (en) * | 2021-06-04 | 2021-08-06 | 杭州海康威视数字技术股份有限公司 | Image splicing method |
| CN117670667A (en) * | 2023-11-08 | 2024-03-08 | 广州成至智能机器科技有限公司 | Unmanned aerial vehicle real-time infrared image panorama stitching method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103955888A (en) * | 2014-05-12 | 2014-07-30 | 中国人民解放军空军预警学院监控系统工程研究所 | High-definition video image mosaic method and device based on SIFT |
| US9196071B2 (en) * | 2013-12-03 | 2015-11-24 | Huawei Technologies Co., Ltd. | Image splicing method and apparatus |
-
2016
- 2016-12-30 CN CN201611259450.8A patent/CN106851092B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9196071B2 (en) * | 2013-12-03 | 2015-11-24 | Huawei Technologies Co., Ltd. | Image splicing method and apparatus |
| CN103955888A (en) * | 2014-05-12 | 2014-07-30 | 中国人民解放军空军预警学院监控系统工程研究所 | High-definition video image mosaic method and device based on SIFT |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107257443A (en) * | 2017-07-24 | 2017-10-17 | 中科创达软件科技（深圳）有限公司 | The method and its device, terminal device of a kind of anti-vignetting of stitching image |
| CN107341827A (en) * | 2017-07-27 | 2017-11-10 | 腾讯科技（深圳）有限公司 | A kind of method for processing video frequency, device and storage medium |
| CN107341827B (en) * | 2017-07-27 | 2023-01-24 | 腾讯科技（深圳）有限公司 | Video processing method, device and storage medium |
| CN109272442A (en) * | 2018-09-27 | 2019-01-25 | 百度在线网络技术（北京）有限公司 | Processing method, device, equipment and the storage medium of panorama spherical surface image |
| WO2020133412A1 (en) * | 2018-12-29 | 2020-07-02 | 深圳市大疆创新科技有限公司 | Panoramic image generation method, panoramic image generation device, and unmanned aerial vehicle |
| CN110800283A (en) * | 2018-12-29 | 2020-02-14 | 深圳市大疆创新科技有限公司 | Panoramic image generation method, panoramic image generation device and unmanned aerial vehicle |
| CN109981985A (en) * | 2019-03-29 | 2019-07-05 | 上海智觅智能科技有限公司 | A kind of continuous stitching algorithm of double vision frequency |
| CN110796597A (en) * | 2019-10-10 | 2020-02-14 | 武汉理工大学 | Vehicle-mounted all-round-view image splicing device based on space-time compensation |
| CN110796597B (en) * | 2019-10-10 | 2024-02-02 | 武汉理工大学 | Vehicle-mounted all-round image splicing device based on space-time compensation |
| CN111445416A (en) * | 2020-03-30 | 2020-07-24 | 南京泓众电子科技有限公司 | Method and device for generating high-dynamic-range panoramic image |
| CN111445416B (en) * | 2020-03-30 | 2022-04-26 | 南京泓众电子科技有限公司 | Method and device for generating high-dynamic-range panoramic image |
| CN113222878A (en) * | 2021-06-04 | 2021-08-06 | 杭州海康威视数字技术股份有限公司 | Image splicing method |
| CN113222878B (en) * | 2021-06-04 | 2023-09-05 | 杭州海康威视数字技术股份有限公司 | Image stitching method |
| CN117670667A (en) * | 2023-11-08 | 2024-03-08 | 广州成至智能机器科技有限公司 | Unmanned aerial vehicle real-time infrared image panorama stitching method |
| CN117670667B (en) * | 2023-11-08 | 2024-05-28 | 广州成至智能机器科技有限公司 | Unmanned aerial vehicle real-time infrared image panorama stitching method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106851092B (en) | 2018-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106851092B (en) | A kind of infrared video joining method and device | |
| CN109961399B (en) | Optimal suture line searching method based on image distance transformation | |
| CN110956661B (en) | Method for calculating dynamic pose of visible light and infrared camera based on bidirectional homography matrix | |
| CN111723801B (en) | Method and system for detecting and correcting target in fisheye camera picture | |
| CN110929566A (en) | Human face living body detection method based on visible light and near-infrared binocular camera | |
| CN103902953B (en) | A kind of screen detecting system and method | |
| CN109886883A (en) | Real-time polarization fog-penetrating imaging image enhancement processing method | |
| CN113191954B (en) | Panoramic image stitching method based on binocular camera | |
| CN103841298B (en) | Video image stabilization method based on color constant and geometry invariant features | |
| CN105787943B (en) | SAR image registration method based on multi-scale image block feature and rarefaction representation | |
| CN107038722A (en) | A kind of equipment localization method and device | |
| CN106952225A (en) | A kind of panorama mosaic method towards forest fire protection | |
| CN110189375A (en) | A kind of images steganalysis method based on monocular vision measurement | |
| CN110910456B (en) | Three-dimensional camera dynamic calibration method based on Harris angular point mutual information matching | |
| CN116681636B (en) | Light infrared and visible light image fusion method based on convolutional neural network | |
| CN108257094A (en) | The quick minimizing technology of remote sensing image mist based on dark | |
| CN112580434B (en) | Face false detection optimization method and system based on depth camera and face detection equipment | |
| CN106529556A (en) | Visual inspection system for instrument indicator lamp | |
| CN107680035A (en) | A kind of parameter calibration method and device, server and readable storage medium storing program for executing | |
| CN107067441A (en) | Camera marking method and device | |
| Lv et al. | Two adaptive enhancement algorithms for high gray-scale RAW infrared images based on multi-scale fusion and chromatographic remapping | |
| CN111915735B (en) | Depth optimization method for three-dimensional structure outline in video | |
| CN107194954A (en) | The sportsman's method for tracing and device of multi-angle video | |
| CN104794445B (en) | A kind of dynamic human face method for collecting iris based on ARM platforms | |
| CN109410308A (en) | Image processing method and device, electronic equipment, computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180209 Termination date: 20181230 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |