CN105023269A - Vehicle-mounted infrared image colorization method - Google Patents

Vehicle-mounted infrared image colorization method Download PDF

Info

Publication number
CN105023269A
CN105023269A CN201510206869.6A CN201510206869A CN105023269A CN 105023269 A CN105023269 A CN 105023269A CN 201510206869 A CN201510206869 A CN 201510206869A CN 105023269 A CN105023269 A CN 105023269A
Authority
CN
China
Prior art keywords
pixel
infrared image
mounted infrared
vehicle mounted
super
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201510206869.6A
Other languages
Chinese (zh)
Other versions
CN105023269B (en
Inventor
沈振一
孙韶媛
候俊杰
顾倩文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Donghua University
National Dong Hwa University
Original Assignee
Donghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Donghua University filed Critical Donghua University
Priority to CN201510206869.6A priority Critical patent/CN105023269B/en
Publication of CN105023269A publication Critical patent/CN105023269A/en
Application granted granted Critical
Publication of CN105023269B publication Critical patent/CN105023269B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The invention relates to a vehicle-mounted infrared image colorization method, which comprises the steps of: extracting features and training a random forest classifier; inputting test pictures into a trained random forest classifier for classification to obtain a classification result map; subjecting the classification result map to superpixel segmentation, carrying out histogram result statistics within superpixels, and determining the classifications to which superpixel blocks belong at last; and converting a color space into an HSV space, entrusting corresponding colors according to the corresponding classifications, and converting a gray level value of an original image into a V-layer value of a final color image. The vehicle-mounted infrared image colorization method is applicable to various kinds of vehicle-mounted infrared scenes, and the robustness of the algorithm is improved significantly.

Description

A kind of vehicle mounted infrared image colorization method
Technical field
The present invention relates to a kind of vehicle mounted infrared image colorization technology.
Background technology
In the situation such as night or haze, the visibility of naked eyes is very low, is very easy to the generation causing the problems such as traffic safety.Because vehicle mounted infrared camera has strong interference immunity, the visual field is stablized, and overcomes the problem that cannot obtain complete road information when the degree of being visible to the naked eye caused due to extraneous factor during vehicle travels has decline.In recent years based on the DAS (Driver Assistant System) of vehicle mounted infrared image research more and more pay close attention to by people.Research contents comprises infrared image target detection, estimation of Depth, colorize etc.The colorize technology of infrared image is one of gordian technique of research infrared image, can be single by color, contrast unconspicuous infrared image by the technological means of image procossing by more friendly for the information comprised in image, present to user intuitively, improve the understanding effect of user for infrared image.
Deepening continuously in recent years along with image colorization research, as based on the Images Classification colorize technology of label transfer algorithm and layering cutting techniques and the Images Classification colorize technology that uses local feature description's symbol and condition random field (CRF) to combine.But in the field of infrared image colorize research, mainly still rest on based on the infrared image colorize algorithm on Iamge Segmentation basis, cause final colorize result to be too dependent on the result of Iamge Segmentation.Colorize algorithm based on label transfer needs to set up a large amount of matching image storehouse, and increases along with the image in storehouse, and the time cost of consumption is then larger.
Summary of the invention
The technical problem to be solved in the present invention under the condition that vehicle mounted infrared road conditions is changeable, can correctly carry out the colorize work of vehicle mounted infrared image.
In order to solve the problems of the technologies described above, technical scheme of the present invention there is provided a kind of vehicle mounted infrared image colorization method, it is characterized in that, comprises the steps:
The first step, collect several vehicle mounted infrared images for training random forest to classify and the classification chart corresponding with vehicle mounted infrared image, extract the characteristic information of each pixel in every width vehicle mounted infrared image, the characteristic information of all pixels of each width vehicle mounted infrared image and the classification chart of correspondence are built training sample set together;
Second step, training sample set is utilized to train random forest sorter;
3rd step, random forest sorter vehicle mounted infrared image to be tested input trained, classified to each pixel in vehicle mounted infrared image to be tested by random forest sorter, obtain classification results figure;
4th step, super-pixel segmentation is carried out to classification results figure, and in each super-pixel inside, statistics with histogram is carried out to the classification of each pixel, be that class that the categorical attribute that counts in current super-pixel is maximum by the key words sorting of each super-pixel, obtain the classification results figure after final optimization pass;
5th step, establishment and the equirotal RGB image of vehicle mounted infrared image to be tested, the color space of RGB image is converted to HSV color space, according to the classification results figure after final optimization pass, corresponding tone is given by the pixel of correspondence, obtain final colo(u)r picture, and the gray-scale value of vehicle mounted infrared image to be tested is transferred to the value of the V layer of final colo(u)r picture.
Preferably, in the described first step, with Law ' s mask, the feature extraction of 3 yardsticks is carried out to each pixel in every width vehicle mounted infrared image.
Preferably, in described 4th step, SLIC super-pixel partitioning algorithm is utilized to carry out super-pixel segmentation to classification results figure.
The present invention is owing to taking above technical scheme, and it has the following advantages:
(1) colorize algorithm is applicable to multiple different vehicle mounted infrared scene herein, and the robustness of algorithm significantly improves.
(2) algorithm of text is based on the classification of pixel scale, and the classification of mistake only can affect this pixel, and the pixel around it can not be caused to go wrong.Again in conjunction with super-pixel segmentation and statistics with histogram algorithm, will effectively contain the situation of erroneous pixel mis-classification further, and then make the colorize of final infrared image more accurate.
Accompanying drawing explanation
Fig. 1 is the algorithm flow block diagram of a kind of vehicle mounted infrared image colorization method provided by the invention;
Fig. 2 is Law ' s mask;
3 different scales that Fig. 3 (a) to Fig. 3 (c) is training figure;
Fig. 4 (a) is original training image;
Fig. 4 (b) is corresponding classification results figure;
Fig. 5 (a) to Fig. 5 (d) is colorize arithmetic result figure.
Embodiment
For making the present invention become apparent, hereby with preferred embodiment, and accompanying drawing is coordinated to be described in detail below.
As shown in Figure 1, the present invention mainly comprises: 1, feature extraction train random forest sorter.2, test picture input the random forest sorter trained and classify, acquisition classification results figure.3, super-pixel segmentation is carried out to classification results figure, and carry out histogram results statistics in super-pixel inside, finally determine the affiliated classification of super-pixel block.4, transfer color space to HSV space, corresponding tone is given in the classification according to correspondence, and the gray-scale value of original image is transferred to the value of the V layer of final colo(u)r picture.Once will be described in detail respectively these 4 parts.
1, random forest
Random forests algorithm is proposed by Leo Breiman and Adele Cutler, and this algorithm combines " Bootstrap aggregating " thought of Breim-ans and " random subspace " method of Ho.Its essence is a sorter comprising multiple decision tree, these decision trees be formed by random method, be therefore also called stochastic decision tree, between the tree in random forest be do not have related.When test data enters random forest, namely allow each decision tree classify, finally getting that maximum class of classification results in all decision trees is final result.Therefore random forest is a sorter comprising multiple decision-making number, and the mode that its classification exported is the classification exported by indivedual tree is determined.It has very high predictablity rate, has good tolerance, and be not easy to occur over-fitting to exception and noise.
Random forests algorithm is the resampling based on Bootstrap method, produces multiple training set.Random forests algorithm have employed the method for random selecting Split Attribute collection when building decision tree.Detailed random forests algorithm flow process is as follows:
(1) resampling of Bootstrap method is utilized, random generation T training set S 1, S 2, S t.
(2) utilize each training set, generate corresponding decision tree C 1, C 2..., C t; Before each non-leaf nodes selects attribute, from M attribute, randomly draw the Split Attribute collection of m attribute as present node, and with divisional mode best in this m attribute, this node is divided.
(3) set all complete growth for every, and do not carry out beta pruning.
(4) for test sample book X, utilize each decision tree to test, obtain corresponding classification C 1(X), C 2(X) ..., C t(X).
(5) adopt ballot method, using export in T decision tree maximum classification as testing anxious sample class.
2, based on the Multi resolution feature extraction of Law ' s mask
For each pixel, needing to calculate the visual signature that a series of eigenwert contains to the pixel caught in each pixel and periphery certain limit thereof, needing the positional information etc. to also needing in conjunction with this pixel simultaneously.Use Law ' s mask image to be carried out to the feature extraction of 3 yardsticks, its feature as shown in Figure 2.
Train the image of three different scales of the image used as shown in Fig. 3 (a) to Fig. 3 (c).
Fig. 2 is Law ' s feature mask, and use Law ' s mask on three different scales of above-mentioned training image, to carry out convolution respectively when feature extraction, Output rusults is F n(x, y), n=1,2 ..., 9.Define texture energy in each super-pixel block such as formula shown in (3).
E i ( n ) = Σ ( x , y ) ∈ S i | I ( x , y ) * F n ( x , y ) | k - - - ( 3 )
Wherein, the pixel value that I (x, y) is original image, works as k=2, E when 4 in () represents energy and the kurtosis characteristic of pixel texture respectively, therefore each pixel has 9 × 3 × 2=54 feature, and finally again in conjunction with the x on pixel, y positional information, so for each pixel extraction, the feature of one 56 dimension is corresponding with it.
3, super-pixel segmentation and statistics with histogram optimized algorithm
Owing to containing the point of the discontinuous mis-classification of part in the Output rusults figure of random forest, so use super-pixel segmentation to split original classification chart, be divided into super-pixel block herein.Again statistics with histogram is carried out to super-pixel block inside.That class that the categorical attribute arrived in super-pixel internal statistical is maximum, is finally labeled as the final classification at super-pixel place.
So-called super-pixel, refers to the image block that the neighbor with features such as similar grain, color, brightness is formed.The segmentation of SLIC super-pixel is proposed by people such as Radhakrishna Achanta, and other super-pixel partitioning algorithm is compared, and this algorithm splitting speed is fast, and internal memory service efficiency is high, and algorithm effect is good, is applicable to very much the optimization process for vehicle mounted infrared image.Coloured image rgb space is converted to CIELAB color space by this algorithm, in conjunction with the positional information of pixel, generates [a l for each pixel i, a i, b i, x i, y i] tvector, wherein, l i, a i, b ibe respectively l ifor the brightness value of pixel, a i, b itwo Color Channels of respective pixel after conversion.Then to 5 dimensional feature vector structure modules, its concrete steps of process of image pixel being carried out to Local Clustering are as follows:
(1) determine algorithm parameter K, namely image needs to be divided into how many super-pixel block.After determining parameter, transfer the color space of image to CIELAB color space.The grid that spacing is S is divided into, wherein as image initial n is the number of the super-pixel block needing segmentation.Super-pixel center C i=[l i, a i, b i, x i, y i] tbe the center of grid.In order to avoid Seed Points is in the marginal position of image, and interference is caused to follow-up cluster process, need Seed Points in the window of centered by it 3 × 3, move to the minimum position of Grad.
(2) within the scope of the 2S of each super-pixel center, the distance of each pixel to center is calculated.In i-th super-pixel a jth pixel therewith the center of super-pixel distance calculate as shown in formula (4):
d c = ( l j - l i ) 2 + ( a j - a i ) 2 + ( b j - b i ) 2 , d s = ( x j - x i ) 2 + ( y j - y i ) 2 , D ′ = ( d c m ) 2 + ( d s S ) 2 - - - ( 4 )
Wherein d cfor distance on color, d sfor space length, m is used for adjusting d cand d sbetween proportionate relationship.
(3) each pixel is classified as the classification minimum with its distance D '.Recalculate the center of each super-pixel, repeat step (2).
4, super-pixel internal sorting result statistics with histogram
In order to remove in random forest classification results the situation that there is partial pixel point mis-classification, strengthen the continuity of classification results between pixel.After super-pixel segmentation, use statistics with histogram in super-pixel inside, the affiliated classification results of whole super-pixel will be marked as the maximum classification of the frequency of occurrences in statistics.If the affiliated classification of sky, ground, the woods represents with numerical value 1 ~ 3 respectively, in super-pixel, comprise the quantity N (i) of certain classification results, i ∈ 1,2,3, classification Sup belonging to certain super-pixel block j jrepresent that so super-pixel statistics with histogram formula is as shown in the formula shown in (5).
N(i)＝max N(t)，t∈1，2，3
(5)
Sup j＝i
5, infrared image colorize algorithm
First create the RGB image the same with original image size, the color space of image is converted to HSV color space.H is the Hue layer of coloured image, and S is the saturation degree layer of coloured image, and V is the brightness layer of coloured image.According to priori in colorize, according to the classification results that the classification after final optimization pass exports, its span of tone of the imparting correspondence of correspondence is between 0 ~ 1.Tone value as sky is 0.55, and the tone of trees is 0.32, and the tone on ground is 0.09, and the value of entire image saturation degree is 0.65.Value about tone and saturation degree can have multiple, can according to the corresponding adjustment of the visual custom of user after colorize.
Because infrared image is mainly monochrome information, therefore need in the image after colorize, to retain this vital information, therefore the brightness value of former infrared image is given to the V layer of the image after final colorize.The image after final colorize is made to remain the raw information of infrared image.
The size of the training image used is 344 × 132 pixels, and the size used after removing limit is 340 × 128 pixels.This experiment use 8 width image is as training image, and 700 width images are test pattern.8 width sample images and the corresponding classified image of training constitute the most original training set, as shown in Fig. 4 (a) to Fig. 4 (b).Original training image carries out feature extraction.According to characteristic extraction part, each pixel all has one 56 proper vector tieed up to represent this pixel and the feature of image in certain limit around it.
By in 700 width test pattern input random forest sorters, the classified image that random forest sorter exports is as shown in Fig. 4 (b).The Image Segmentation Using that super-pixel partitioning algorithm exports sorter also carries out statistics with histogram in super-pixel block, and optimum results is as shown in Fig. 5 (c).The result images optimized the most at last carries out the final design sketch of colorize process as shown in Fig. 5 (d).Fig. 5 (a) is corresponding infrared image.

Claims (3)

1. a vehicle mounted infrared image colorization method, is characterized in that, comprises the steps:
The first step, collect several vehicle mounted infrared images for training random forest to classify and the classification chart corresponding with vehicle mounted infrared image, extract the characteristic information of each pixel in every width vehicle mounted infrared image, the characteristic information of all pixels of each width vehicle mounted infrared image and the classification chart of correspondence are built training sample set together;
Second step, training sample set is utilized to train random forest sorter;
3rd step, random forest sorter vehicle mounted infrared image to be tested input trained, classified to each pixel in vehicle mounted infrared image to be tested by random forest sorter, obtain classification results figure;
4th step, super-pixel segmentation is carried out to classification results figure, and in each super-pixel inside, statistics with histogram is carried out to the classification of each pixel, be that class that the categorical attribute that counts in current super-pixel is maximum by the key words sorting of each super-pixel, obtain the classification results figure after final optimization pass;
5th step, establishment and the equirotal RGB image of vehicle mounted infrared image to be tested, the color space of RGB image is converted to HSV color space, according to the classification results figure after final optimization pass, corresponding tone is given by the pixel of correspondence, obtain final colo(u)r picture, and the gray-scale value of vehicle mounted infrared image to be tested is transferred to the value of the V layer of final colo(u)r picture.
2. a kind of vehicle mounted infrared image colorization method as claimed in claim 1, is characterized in that, in the described first step, carries out the feature extraction of 3 yardsticks with Law ' s mask to each pixel in every width vehicle mounted infrared image.
3. a kind of vehicle mounted infrared image colorization method as claimed in claim 1, is characterized in that, in described 4th step, utilizes SLIC super-pixel partitioning algorithm to carry out super-pixel segmentation to classification results figure.
CN201510206869.6A 2015-04-27 2015-04-27 A kind of vehicle mounted infrared image colorization method Expired - Fee Related CN105023269B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510206869.6A CN105023269B (en) 2015-04-27 2015-04-27 A kind of vehicle mounted infrared image colorization method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510206869.6A CN105023269B (en) 2015-04-27 2015-04-27 A kind of vehicle mounted infrared image colorization method

Publications (2)

Publication Number Publication Date
CN105023269A true CN105023269A (en) 2015-11-04
CN105023269B CN105023269B (en) 2018-01-19

Family

ID=54413210

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510206869.6A Expired - Fee Related CN105023269B (en) 2015-04-27 2015-04-27 A kind of vehicle mounted infrared image colorization method

Country Status (1)

Country Link
CN (1) CN105023269B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106951863A (en) * 2017-03-20 2017-07-14 贵州电网有限责任公司电力科学研究院 A kind of substation equipment infrared image change detecting method based on random forest
CN108446678A (en) * 2018-05-07 2018-08-24 同济大学 A kind of dangerous driving behavior recognition methods based on skeleton character
CN109017799A (en) * 2018-04-03 2018-12-18 张锐明 A kind of new-energy automobile driving behavior prediction technique
DE102018216806A1 (en) * 2018-09-28 2020-04-02 Volkswagen Aktiengesellschaft Concept for processing infrared images

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1996013806A1 (en) * 1994-11-01 1996-05-09 Massachusetts Institute Of Technology Method and apparatus for generating a synthetic image by the fusion of signals representative of different views of the same scene
CN101853492A (en) * 2010-05-05 2010-10-06 浙江理工大学 Method for fusing night-viewing twilight image and infrared image
CN102270355A (en) * 2011-04-28 2011-12-07 华中科技大学 Infrared scene image generation method based on scenery classification
CN102306295A (en) * 2011-05-31 2012-01-04 东华大学 Natural color night vision realization method based on single band infrared image

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1996013806A1 (en) * 1994-11-01 1996-05-09 Massachusetts Institute Of Technology Method and apparatus for generating a synthetic image by the fusion of signals representative of different views of the same scene
CN101853492A (en) * 2010-05-05 2010-10-06 浙江理工大学 Method for fusing night-viewing twilight image and infrared image
CN102270355A (en) * 2011-04-28 2011-12-07 华中科技大学 Infrared scene image generation method based on scenery classification
CN102306295A (en) * 2011-05-31 2012-01-04 东华大学 Natural color night vision realization method based on single band infrared image

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106951863A (en) * 2017-03-20 2017-07-14 贵州电网有限责任公司电力科学研究院 A kind of substation equipment infrared image change detecting method based on random forest
CN106951863B (en) * 2017-03-20 2023-09-26 贵州电网有限责任公司电力科学研究院 Method for detecting change of infrared image of substation equipment based on random forest
CN109017799A (en) * 2018-04-03 2018-12-18 张锐明 A kind of new-energy automobile driving behavior prediction technique
CN108446678A (en) * 2018-05-07 2018-08-24 同济大学 A kind of dangerous driving behavior recognition methods based on skeleton character
CN108446678B (en) * 2018-05-07 2021-05-11 同济大学 Dangerous driving behavior identification method based on skeletal features
DE102018216806A1 (en) * 2018-09-28 2020-04-02 Volkswagen Aktiengesellschaft Concept for processing infrared images
WO2020064521A1 (en) 2018-09-28 2020-04-02 Volkswagen Aktiengesellschaft Concept for preparing infrared images

Also Published As

Publication number Publication date
CN105023269B (en) 2018-01-19

Similar Documents

Publication Publication Date Title
CN108573276B (en) Change detection method based on high-resolution remote sensing image
CN102902956B (en) A kind of ground visible cloud image identifying processing method
CN104966085B (en) A kind of remote sensing images region of interest area detecting method based on the fusion of more notable features
CN110210475B (en) License plate character image segmentation method based on non-binarization and edge detection
CN107392130A (en) Classification of Multispectral Images method based on threshold adaptive and convolutional neural networks
CN106651872A (en) Prewitt operator-based pavement crack recognition method and system
CN101840581B (en) Method for extracting profile of building from satellite remote sensing image
CN109800736A (en) A kind of method for extracting roads based on remote sensing image and deep learning
CN107392925A (en) Remote sensing image terrain classification method based on super-pixel coding and convolutional neural networks
CN110619282A (en) Automatic extraction method for unmanned aerial vehicle orthoscopic image building
CN107122776A (en) A kind of road traffic sign detection and recognition methods based on convolutional neural networks
CN108229425A (en) A kind of identifying water boy method based on high-resolution remote sensing image
CN107832797B (en) Multispectral image classification method based on depth fusion residual error network
CN108734719A (en) Background automatic division method before a kind of lepidopterous insects image based on full convolutional neural networks
CN107784657A (en) A kind of unmanned aerial vehicle remote sensing image partition method based on color space classification
CN104598908A (en) Method for recognizing diseases of crop leaves
CN107025440A (en) A kind of remote sensing images method for extracting roads based on new convolutional neural networks
CN111639587B (en) Hyperspectral image classification method based on multi-scale spectrum space convolution neural network
CN104835196A (en) Vehicular infrared image colorization and three-dimensional reconstruction method
CN105427309A (en) Multiscale hierarchical processing method for extracting object-oriented high-spatial resolution remote sensing information
CN113160062B (en) Infrared image target detection method, device, equipment and storage medium
CN109684922A (en) A kind of recognition methods based on the multi-model of convolutional neural networks to finished product dish
CN105955708A (en) Sports video lens classification method based on deep convolutional neural networks
CN107358176A (en) Sorting technique based on high score remote sensing image area information and convolutional neural networks
CN110807485B (en) Method for fusing two-classification semantic segmentation maps into multi-classification semantic map based on high-resolution remote sensing image

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20180119

Termination date: 20200427