CN112766335B - Image processing method and device, electronic equipment and storage medium - Google Patents
Image processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112766335B CN112766335B CN202110027893.9A CN202110027893A CN112766335B CN 112766335 B CN112766335 B CN 112766335B CN 202110027893 A CN202110027893 A CN 202110027893A CN 112766335 B CN112766335 B CN 112766335B
- Authority
- CN
- China
- Prior art keywords
- pixel
- super
- image
- probability distribution
- classified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 11
- 238000010586 diagram Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 32
- 238000007637 random forest analysis Methods 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims description 24
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000000354 decomposition reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000010287 polarization Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000005388 cross polarization Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
Abstract
The application discloses an image processing method, an image processing device, electronic equipment and a storage medium, and relates to the technical field of image processing. Dividing an image to be classified into a plurality of super pixels, and extracting characteristic information of the image to be classified; the feature information is brought into a first random forest model trained in advance, and a single pixel probability distribution map is obtained; classifying each of the plurality of superpixels by using the pre-trained second random forest model to obtain a superpixel probability distribution map; aiming at each pixel point, based on the class probability of the pixel point in a single pixel probability distribution diagram and the class probability of the pixel point in the super pixel probability distribution diagram, obtaining the corrected class probability of the pixel point; and obtaining a result probability distribution diagram according to the corrected class probability of each pixel point. According to the scheme, the problems that in the prior art, spatial information of pixel points is not fully considered, so that image classification efficiency is low and classification accuracy is low are solved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to an image processing method, an image processing device, an electronic device, and a storage medium.
Background
Synthetic aperture radars (Synthetic Aperture Radar, SAR) have the characteristics of active operation, no influence of light and weather, and strong penetration, and can continuously acquire target data from the ground in severe weather. As a high-resolution imaging radar, since birth, rapid development has been achieved, and the radar has been widely used in military information, agricultural monitoring, ocean exploration, forest monitoring, geological exploration, disaster relief research and other fields.
At present, most polarized SAR image classification technologies use pixel points as classification units, so that the purpose of classifying ground features in remote sensing images is achieved. However, the classification target is not composed of one or several pixels, the single-pixel classification method does not fully consider the spatial information of the pixels, and the polarized SAR image also has more speckle noise, so that the problems result in low image classification efficiency and low classification precision.
Disclosure of Invention
The application provides an image processing method, an image processing device, electronic equipment and a storage medium, which are used for solving the problems of low image classification efficiency and low classification precision caused by insufficient consideration of spatial information of pixel points in the prior art.
In a first aspect, an embodiment of the present application provides an image processing method, including: dividing an image to be classified into a plurality of super pixels, and extracting characteristic information of the image to be classified; the characteristic information is brought into a first random forest model trained in advance to obtain a single-pixel probability distribution map, wherein the single-pixel probability distribution map comprises class probabilities of each pixel point in all pixel points in the image to be classified; classifying each of the plurality of superpixels by using a pre-trained second random forest model to obtain a superpixel probability distribution map, wherein the superpixel probability distribution map comprises class probabilities of random forest classification corresponding to each superpixel; aiming at each pixel point, based on the class probability of the pixel point in the single-pixel probability distribution diagram and the class probability of the pixel point in the super-pixel probability distribution diagram, obtaining the corrected class probability of the pixel point; and obtaining a result probability distribution map according to the corrected class probabilities of the pixel points, wherein the result probability distribution map comprises corrected class probabilities of each pixel point in all the pixel points in the image to be classified.
In the embodiment of the application, the corrected class probability of the pixel point is obtained based on the class probability of the pixel point in the single-pixel probability distribution diagram and the class probability of the pixel point in the super-pixel probability distribution diagram. Because the adjacent pixels in the image have a certain correlation, and the super-pixels are composed of a plurality of pixel points with higher similarity, the classification probability of the pixel points in the super-pixel probability distribution map is introduced in the classification probability after the correction of the pixel points, so that the spatial information of the pixel points is considered, and the accuracy of image classification is improved.
With reference to the foregoing technical solution provided in the first aspect, in some possible implementation manners, obtaining the corrected class probability of the pixel point based on the class probability of the pixel point in the single pixel probability distribution diagram and the class probability of the pixel point in the super pixel probability distribution diagram includes: acquiring a first weight representing that a super pixel to which the pixel point belongs and a neighboring super pixel of the super pixel belong to the same category; obtaining the product of the class probability of the pixel point in the single-pixel probability distribution diagram and the first weight to obtain a first product; obtaining the product of the class probability of the pixel point in the super-pixel probability distribution diagram and the second weight to obtain a second product, wherein the sum of the first weight and the second weight is 1; and obtaining the sum of the first product and the second product to obtain the corrected class probability of the pixel point.
In the embodiment of the application, a first product is obtained by calculating the product of the class probability of the pixel point in the single pixel probability distribution diagram and the first weight; and obtaining a second product by multiplying the class probability of the pixel point in the super-pixel probability distribution diagram by a second weight; and calculating the sum of the first product and the second product to obtain the corrected class probability of the pixel point. The classification probability obtained by combining the two classification methods is combined, and the first weight and the second weight with the sum of the weights being 1 are used for weighting, so that the final classification result is more accurate, and the image classification precision is improved.
With reference to the foregoing technical solution provided in the first aspect, in some possible implementation manners, obtaining a first weight that characterizes a superpixel to which the pixel point belongs and a neighboring superpixel of the superpixel belong to a same class includes: calculating the distance between the super pixel center of the super pixel to which the pixel point belongs and the super pixel center of each adjacent super pixel, and obtaining the average value of all the calculated distances; and obtaining the first weight based on the average value, wherein the first weight is the inverse of the sum of the average value and a preset threshold value.
In the embodiment of the application, the average value of the distances between the super pixel center to which the pixel point belongs and each adjacent super pixel center is calculated, and then the first weight representing that the current super pixel and the adjacent super pixel belong to the same category and the second weight with the sum of the first weights being 1 are obtained based on the preset threshold value and the average value. Because adjacent pixels in the image have a certain correlation and are not completely independent, the category attribution of each pixel is related to the neighborhood attribution of the point, and the correlation between the adjacent super pixels is fully considered by introducing the weight representing that the super pixel and the adjacent super pixel belong to the same category, so that the image classification precision is improved.
With reference to the foregoing technical solution of the first aspect, in some possible implementation manners, before dividing an image to be classified into a plurality of superpixels and extracting feature information of the image to be classified, the method further includes: the method comprises the steps of obtaining an original image to be classified, carrying out filtering treatment on the original image to be classified, and filtering speckle noise in the original image to be classified to obtain the image to be classified.
In the embodiment of the application, the speckle noise in the original image to be classified can be filtered by filtering the original image to be classified, so that the error generated by the subsequent super-pixel segmentation is reduced, and the classification precision is improved.
With reference to the foregoing technical solution provided in the first aspect, in some possible implementation manners, segmenting the image to be classified into a plurality of super pixels includes: decomposing the image to be classified into gradient images; according to the gradient image, selecting a seed point as an initial center point of the super pixel by using a step S, and then adjusting the seed point to the lowest point of the gradient image within a range of n to n, wherein both S and n are positive integers which are larger than 1, and n is smaller than S; calculating the distance from each pixel to the initial center point within the range of 2S multiplied by 2S of the initial center point of the super pixel, and dividing the pixel into the nearest super pixel; dividing all pixels in the gradient image into super pixels, updating the center point of each super pixel, calculating the distance from each pixel to the center point again, and dividing the pixel into the nearest super pixel until convergence or the maximum iteration number is reached; merging the super pixels with the pixel number smaller than the threshold value into the super pixels nearest to the super pixels to obtain the plurality of super pixels.
In the embodiment of the application, a seed point is selected from a gradient image by taking S as a step length as an initial center point of a super pixel, then the seed point is adjusted to the lowest point of the gradient image in a range of n steps, the distance from each pixel to the initial center point in the range of n steps is calculated, and the pixel is divided into the nearest super pixels; dividing all pixels in the gradient image into super pixels, updating the center point of each super pixel, calculating the distance from each pixel to the center point again, and dividing the pixel into the nearest super pixel until convergence or the maximum iteration number is reached; and merging the super pixels with the pixel number smaller than the threshold value into the super pixels nearest to the super pixels, and finally obtaining a plurality of super pixels. According to the scheme, accurate and fine superpixels can be generated to serve as classification units, and interference of speckle noise in images is effectively reduced.
In a second aspect, an embodiment of the present application provides an image processing apparatus, including a first processing module, a classification module, a second processing module, and a generation module. The first processing module is used for dividing the image to be classified into a plurality of super pixels and extracting characteristic information of the image to be classified; the classification module is used for bringing the characteristic information into a first random forest model trained in advance to obtain a single-pixel probability distribution map, wherein the single-pixel probability distribution map comprises class probabilities of each pixel point in all pixel points in the image to be classified; the classification module is further configured to classify each of the plurality of superpixels by using the pre-trained second random forest model, so as to obtain a superpixel probability distribution map, where the superpixel probability distribution map includes class probabilities of random forest classifications corresponding to each superpixel; the second processing module is used for obtaining the corrected class probability of each pixel point based on the class probability of the pixel point in the single-pixel probability distribution diagram and the class probability of the pixel point in the super-pixel probability distribution diagram; the generating module is used for obtaining a result probability distribution diagram according to the corrected category probabilities of the pixel points, wherein the result probability distribution diagram comprises corrected category probabilities of each pixel point in all the pixel points in the image to be classified.
With reference to the foregoing technical solution of the second aspect, in some possible implementation manners, the second processing module includes an obtaining module and a calculating module. The acquisition module is used for acquiring a first weight representing that the super pixel to which the pixel point belongs and the adjacent super pixel of the super pixel belong to the same category; the calculation module is used for obtaining the product of the class probability of the pixel point in the single-pixel probability distribution diagram and the first weight to obtain a first product; the calculation module is further configured to obtain a product of the class probability of the pixel point in the super-pixel probability distribution diagram and the second weight, so as to obtain a second product, where the sum of the first weight and the second weight is 1; the computing module is further configured to obtain a sum of the first product and the second product, and obtain a corrected class probability of the pixel.
With reference to the foregoing technical solution provided in the second aspect, in some possible implementation manners, the obtaining module is specifically configured to calculate a distance between a superpixel center of a superpixel to which the pixel point belongs and a superpixel center of each neighboring superpixel, and obtain an average value of all the calculated distances; and obtaining the first weight based on the average value, wherein the first weight is the inverse of the sum of the average value and a preset threshold value.
In a third aspect, an embodiment of the present application provides an electronic device, including: the device comprises a memory and a processor, wherein the memory is connected with the processor; the memory is used for storing programs; the processor is configured to invoke the program stored in the memory to perform the method as the above-described first aspect embodiment and/or any possible implementation manner in combination with the first aspect embodiment.
In a fourth aspect, embodiments of the present application provide a storage medium having stored thereon a computer program which, when executed by a computer, performs a method as in the embodiments of the first aspect and/or any of the possible implementations in combination with the embodiments of the first aspect.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a graphics processing method according to an embodiment of the present application;
FIG. 2 is a block diagram of a graphics processing apparatus according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The terms "first," "second," and the like are used merely for distinguishing between descriptions and not necessarily for indicating a sequential order, nor are they to be construed as indicating or implying relative importance.
The technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings.
In the prior art, pixels are usually used as classification units, but the classification target is not composed of one or more pixels, and the prior art does not fully consider the spatial information of the pixels, so that the image classification efficiency is low and the classification precision is not high. Therefore, the application provides an image processing method to solve the problems of low image classification efficiency and low classification precision caused by insufficient consideration of spatial information of pixel points in the prior art.
Referring to fig. 1, fig. 1 is a schematic diagram of an image processing method according to an embodiment of the present application, and the steps included in the method are described below with reference to fig. 1.
Step S100: dividing an image to be classified into a plurality of super pixels, and extracting characteristic information of the image to be classified.
After extracting the characteristic information of the image to be classified, the image to be classified is divided into a plurality of super pixels, wherein the characteristic information can be all or part of polarized total power, correlation coefficient, homopolar phase difference, homopolar ratio, cross polarization, circular polarization correlation coefficient, odd-order scattering, even-order scattering, volume scattering, statistical disorder degree and the like. It will be appreciated that, according to the requirements of practical applications, the selected feature information is not identical, and the above listed feature information types are only examples of embodiments of the present application, and may also include other types of feature information, and the above examples should not be taken as limiting the present application.
In one embodiment, the process of dividing the image to be classified into a plurality of super pixels may be: firstly, decomposing an image to be classified into gradient images; then according to the gradient image, selecting a seed point as an initial center point of the super pixel by a step S, and then adjusting the seed point to the lowest point of the gradient image within a range of n to n, wherein S and n are positive integers larger than 1, n is smaller than S, and the step S can be determined by the number of the seed points to be selected and the total number of pixels of the image to be classified, and can be expressed asn may be selected according to practical needs, for example, n may be selected from 3, 4, 5, etc., which are only examples herein, and are not limiting on the present application. Then, in the range of 2S multiplied by 2S of the initial center point of the super pixel, calculating the distance from each pixel to the initial center point, and dividing the pixel into the nearest super pixel; after all pixels in the gradient image are divided into super pixels, updating the center point of each super pixel, calculating the distance from each pixel to the updated center point again, and dividing the pixel into the nearest super pixel until convergence or the maximum iteration number is reached; and finally merging the super pixels with the pixel number smaller than the threshold value into the super pixels nearest to the super pixels to obtain a plurality of super pixels.
The image to be classified can be decomposed into Pauil RGB gradient images through a Pauil algorithm, and optionally, different algorithms can be selected to decompose the image to be classified according to different polarization characteristics, for example, based on statistical disorder, an H-a-A algorithm can be selected to decompose the image to be classified.
When the Pauil algorithm is adopted to decompose the polarized SAR image into the Pauil RGB gradient image, the Pauli characteristic can be utilized to replace the spectral characteristic in the traditional optical image, the distance from the pixel to the center point of the super pixel is redefined, and then the accurate and fine super pixel is generated. Whereas Pauil decomposition yields 3 components with distinct physical significance, odd, even and bulk scattering, respectively. Wherein, odd scattering represents spherical surface, plane, triangular surface, symmetrical angular ground object, even scattering represents ground object of dihedral angle scattering, and bulk scattering represents dihedral angle (including vegetation volume scattering) of pi/4 inclination. By usingAn average feature vector representing Pauli decomposition for the superpixel center i, where +.>Representing odd scattering, ++>Representing the even-order scattering and,representing volume scattering, using (x i ，y i ) Representing the spatial position of the superpixel i by +.>An average feature vector representing Pauli decomposition for pixel j, wherein +.>Show odd scattering, ++>Representing even scattering, ++>Representing volume scattering, using (x j ，y j ) Representing the spatial position of the superpixel j, the spatial distance d between the superpixel center i and the pixel j s And Pauli distance d p The method comprises the following steps of: the distance d of a pixel j to the superpixel center i can be expressed as: />Wherein max (d p ) Pauli distance d in the last iteration p And is used with step S for d p And d s Normalization. Normalized d p And d s The relative size of (2) affects the shape and size of the superpixel. The larger the spatial distance is, the more compact the superpixel is generated, whereas the more irregular the shape and size is.
By using the method for calculating the distance from the pixel to the super-pixel center, the distance from each pixel in the range of 2S multiplied by 2S of each super-pixel center to the super-pixel center is calculated in sequence, and the pixel is divided into the nearest super-pixels; after the calculation is completed, the center of each superpixel is updated. Repeating the steps until convergence or the maximum iteration number is reached, and further generating the super pixel with the size being approximate to the S center. Wherein, the pixel closest to the average value of the super pixels in each super pixel can be used as a new super pixel center, and the average value of the pixels is:wherein omega j Representing the average value of all pixels in the super pixel j; n represents the number of pixels in the super pixel; g j The region representing the super pixel j, i represents the i-th pixel point in the super pixel j.
In one embodiment, before the image to be classified is divided into a plurality of super pixels and the feature information of the image to be classified is extracted, the original image to be classified is acquired, filtering processing is performed on the original image to be classified, and speckle noise in the original image to be classified is filtered out, so that the image to be classified is obtained.
The original image to be classified can be obtained in advance and stored in a database or a disk, and can be obtained directly when needed, or can be obtained by shooting in real time when needed. In one embodiment, the Lee filtering is performed on the original image to be processed to filter out speckle noise in the original image to be classified.
Step S200: the extracted characteristic information is brought into a first random forest model trained in advance, and a single pixel probability distribution map is obtained; and classifying each of the plurality of superpixels by using a pre-trained second random forest model to obtain a superpixel probability distribution map.
In one embodiment, the first random forest model and the second random forest model may be the same random forest model, or the first random forest model and the second random forest model may be two random forest models.
Methods for training random forest models and classifying by applying random forest models are well known to those skilled in the art, and will not be described herein.
Step S300: and aiming at each pixel point, obtaining the corrected class probability of the pixel point based on the class probability of the pixel point in the single-pixel probability distribution diagram and the class probability of the pixel point in the super-pixel probability distribution diagram.
Because each pixel point in the image to be classified has a class probability in the single pixel probability distribution map and the super pixel probability distribution map respectively, and adjacent pixels in the image tend to have certain correlation instead of being completely independent, the class of each pixel point is also related to the adjacent pixel point, and therefore the class probability of the pixel point in the single pixel probability distribution map and the class probability of the pixel point in the super pixel probability distribution map are weighted by introducing the weight representing that the super pixel and the adjacent super pixel belong to the same class, so that the classification precision is improved.
In one embodiment, the method for obtaining the corrected class probability of the pixel point based on the class probability of the pixel point in the single-pixel probability distribution diagram and the class probability of the pixel point in the super-pixel probability distribution diagram may be: firstly, obtaining a first weight representing that a super pixel to which the pixel point belongs and a neighboring super pixel of the super pixel belong to the same category; then obtaining the product of the class probability of the pixel point in the single-pixel probability distribution diagram and the first weight, namely a first product, and the product of the class probability of the pixel point in the super-pixel probability distribution diagram and the second weight, namely a second product, wherein the sum of the first weight and the second weight is 1; and finally, calculating the sum of the first product and the second product to obtain the corrected class probability of the pixel point.
Wherein p represents the corrected class probability of the pixel point, and p is used for pixel Representing the class probability of the pixel point in a single pixel probability distribution diagram, and using p slic The class probability of the pixel point in the super-pixel probability distribution diagram is represented, wherein alpha represents a first weight, and beta represents a second weight. The above procedure can be expressed as p=αp slic +(1-α)p pixel Wherein α+β=1.
In one embodiment, the method for obtaining the first weight representing the super pixel belonging to the same category as the adjacent super pixel of the super pixel point may be: firstly, calculating the distance between the super pixel center of the super pixel to which the pixel point belongs and the super pixel center of each adjacent super pixel, and obtaining the average value of all the calculated distances; and then obtaining a first weight based on the average value, wherein the first weight is the inverse of the sum of the average value and a preset threshold value, and meanwhile, the second weight can be obtained because the sum of the first weight and the second weight is 1.
Wherein d is j Representing the distance between the super pixel center of the super pixel to which the pixel point belongs and the super pixel center of the adjacent j-th super pixel, and calculating by dThe average value of all distances to, when n superpixels are adjacent to the current superpixel, the average value of the n superpixel centers and the current superpixel center can be expressed as:correspondingly, the first weight is denoted by α, the second weight is denoted by β, and the first weight α is: />The second weight β is β=1- α. Wherein k is a preset threshold. In one embodiment, the preset threshold k may be 1, and when k is 1, the first weight α may be expressed as:for the embodiment provided herein, k may take other values, such as 2, 3, 2/3, etc., and is not limited thereto.
Step S400: and obtaining a result probability distribution diagram according to the corrected class probability of each pixel point.
After obtaining the corrected class probability for each pixel in the image to be classified, a resulting probability distribution map including the corrected class probability for each of all pixels in the image to be classified may be generated based on the corrected class probability for each pixel.
Referring to fig. 2, fig. 2 is an image processing apparatus 100 according to an embodiment of the present application, which includes a first processing module 110, a classification module 120, a second processing module 130, and a generation module 140.
The first processing module 110 is configured to divide an image to be classified into a plurality of super pixels, and extract feature information of the image to be classified;
the classification module 120 is configured to bring the feature information into a first random forest model trained in advance, so as to obtain a single-pixel probability distribution map, where the single-pixel probability distribution map includes a class probability of each pixel point in all the pixels in the image to be classified;
the classification module 120 is further configured to classify each of the plurality of superpixels by using the pre-trained second random forest model, so as to obtain a superpixel probability distribution map, where the superpixel probability distribution map includes a class probability of each superpixel corresponding to a random forest classification;
the second processing module 130 is configured to obtain, for each pixel, a corrected class probability of the pixel based on a class probability of the pixel in the single-pixel probability distribution map and a class probability of the pixel in the super-pixel probability distribution map;
the generating module 140 is configured to obtain a result probability distribution map according to the corrected class probabilities of the respective pixels, where the result probability distribution map includes the corrected class probabilities of each of all the pixels in the image to be classified.
The second processing module 130 includes an obtaining module and a calculating module, where the obtaining module is configured to obtain a first weight that characterizes a superpixel to which the pixel point belongs and a neighboring superpixel of the superpixel belong to a same class; the calculation module is used for obtaining the product of the class probability of the pixel point in the single-pixel probability distribution diagram and the first weight to obtain a first product; the calculation module is further configured to obtain a product of the class probability of the pixel point in the super-pixel probability distribution diagram and the second weight, so as to obtain a second product, where the sum of the first weight and the second weight is 1; the computing module is further configured to obtain a sum of the first product and the second product, and obtain a corrected class probability of the pixel.
The acquisition module is specifically used for calculating the distance between the super pixel center of the super pixel to which the pixel point belongs and the super pixel center of each adjacent super pixel, and acquiring the average value of all the calculated distances; and obtaining the first weight based on the average value, wherein the first weight is the inverse of the sum of the average value and a preset threshold value.
The first processing module 110 is further configured to obtain an original image to be classified, filter the original image to be classified, and filter speckle noise in the original image to be classified to obtain the image to be classified.
The first processing module 110 is specifically configured to decompose the image to be classified into gradient images; according to the gradient image, selecting a seed point as an initial center point of the super pixel by using a step S, and then adjusting the seed point to the lowest point of the gradient image within a range of n to n, wherein both S and n are positive integers which are larger than 1, and n is smaller than S; calculating the distance from each pixel to the initial center point within the range of 2S multiplied by 2S of the initial center point of the super pixel, and dividing the pixel into the nearest super pixel; dividing all pixels in the gradient image into super pixels, updating the center point of each super pixel, calculating the distance from each pixel to the center point again, and dividing the pixel into the nearest super pixel until convergence or the maximum iteration number is reached; and merging the super pixels with the pixel number smaller than the threshold value into the super pixels nearest to the super pixels to obtain a plurality of super pixels.
The embodiment of the device provided by the embodiment of the present application has the same implementation principle and effect as those of the embodiment of the method, and the reference is made to the corresponding parts in the embodiment of the method for reference.
Please refer to fig. 3, which illustrates an electronic device 200 according to an embodiment of the present application. The electronic device 200 includes: transceiver 210, memory 220, communication bus 230, processor 240.
The transceiver 210, the memory 220, and the processor 240 are electrically connected directly or indirectly to each other to realize data transmission or interaction. For example, the components may be electrically coupled to each other via one or more communication buses 230 or signal lines. Wherein the transceiver 210 is configured to transmit and receive data. The memory 220 is used to store a computer program such as the software functional modules shown in fig. 2, i.e., the image processing apparatus 100. The image processing apparatus 100 includes at least one software function module that may be stored in the memory 220 in the form of software or firmware (firmware) or cured in an Operating System (OS) of the electronic device 200. The processor 240 is configured to execute executable modules stored in the memory 220, such as software functional modules or computer programs included in the image processing apparatus 100. For example, dividing an image to be classified into a plurality of super pixels, and extracting characteristic information of the image to be classified; the characteristic information is brought into a first random forest model trained in advance to obtain a single-pixel probability distribution map, wherein the single-pixel probability distribution map comprises class probabilities of each pixel point in all pixel points in the image to be classified; classifying each of the plurality of superpixels by using the pre-trained second random forest model to obtain a superpixel probability distribution map, wherein the superpixel probability distribution map comprises class probabilities of random forest classification corresponding to each superpixel; aiming at each pixel point, based on the class probability of the pixel point in the single-pixel probability distribution diagram and the class probability of the pixel point in the super-pixel probability distribution diagram, obtaining the corrected class probability of the pixel point; and obtaining a result probability distribution map according to the corrected class probabilities of the pixel points, wherein the result probability distribution map comprises corrected class probabilities of each pixel point in all the pixel points in the image to be classified.
The Memory 220 may be, but is not limited to, a random access Memory (Random Access Memory, RAM), a Read Only Memory (ROM), a programmable Read Only Memory (Programmable Read-Only Memory, PROM), an erasable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), an electrically erasable Read Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.
The processor 240 may be an integrated circuit chip with signal processing capabilities. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor 240 may be any conventional processor or the like.
The electronic device 200 includes, but is not limited to, a personal computer, a server, and the like.
The embodiment of the present application further provides a non-volatile computer readable storage medium (hereinafter referred to as a storage medium) having a computer program stored thereon, which when executed by a computer such as the above-described electronic device 200, performs the above-described image processing method. Wherein the storage medium comprises: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. An image processing method, comprising:
dividing an image to be classified into a plurality of super pixels, and extracting characteristic information of the image to be classified;
the characteristic information is brought into a first random forest model trained in advance to obtain a single-pixel probability distribution map, wherein the single-pixel probability distribution map comprises class probabilities of each pixel point in all pixel points in the image to be classified;
classifying each of the plurality of superpixels by using a pre-trained second random forest model to obtain a superpixel probability distribution map, wherein the superpixel probability distribution map comprises class probabilities of random forest classification corresponding to each superpixel;
aiming at each pixel point, based on the class probability of the pixel point in the single-pixel probability distribution diagram and the class probability of the pixel point in the super-pixel probability distribution diagram, obtaining the corrected class probability of the pixel point;
and obtaining a result probability distribution map according to the corrected class probabilities of the pixel points, wherein the result probability distribution map comprises corrected class probabilities of each pixel point in all the pixel points in the image to be classified.
2. The method of claim 1, wherein deriving the corrected class probability for the pixel based on the class probability for the pixel in the single pixel probability distribution map and the class probability for the pixel in the super pixel probability distribution map comprises:
acquiring a first weight representing that a super pixel to which the pixel point belongs and a neighboring super pixel of the super pixel belong to the same category;
obtaining the product of the class probability of the pixel point in the single-pixel probability distribution diagram and the first weight to obtain a first product;
obtaining the product of the class probability of the pixel point in the super pixel probability distribution diagram and a second weight to obtain a second product, wherein the sum of the first weight and the second weight is 1;
and obtaining the sum of the first product and the second product to obtain the corrected class probability of the pixel point.
3. The method of claim 2, wherein obtaining a first weight that characterizes a superpixel to which the pixel belongs as belonging to a same class as a neighboring superpixel of the superpixel comprises:
calculating the distance between the super pixel center of the super pixel to which the pixel point belongs and the super pixel center of each adjacent super pixel, and obtaining the average value of all the calculated distances;
and obtaining the first weight based on the average value, wherein the first weight is the inverse of the sum of the average value and a preset threshold value.
4. The method according to claim 1, wherein before segmenting the image to be classified into a plurality of superpixels and extracting feature information of the image to be classified, the method further comprises:
acquiring an original image to be classified;
and filtering the original image to be classified, and filtering out speckle noise in the original image to be classified to obtain the image to be classified.
5. The method of claim 1, wherein segmenting the image to be classified into a plurality of superpixels, comprises:
decomposing the image to be classified into gradient images;
according to the gradient image, selecting a seed point as an initial center point of the super pixel by using a step S, and then adjusting the seed point to the lowest point of the gradient image within a range of n to n, wherein both S and n are positive integers which are larger than 1, and n is smaller than S;
calculating the distance from each pixel to the initial center point within the range of 2S multiplied by 2S of the initial center point of the super pixel, and dividing the pixel into the nearest super pixel;
dividing all pixels in the gradient image into super pixels, updating the center point of each super pixel, calculating the distance from each pixel to the center point again, and dividing the pixel into the nearest super pixel until convergence or the maximum iteration number is reached;
merging the super pixels with the pixel number smaller than the threshold value into the super pixels nearest to the super pixels to obtain the plurality of super pixels.
6. An image processing apparatus, comprising:
the first processing module is used for dividing the image to be classified into a plurality of super pixels and extracting characteristic information of the image to be classified;
the classification module is used for bringing the characteristic information into a first random forest model trained in advance to obtain a single-pixel probability distribution map, wherein the single-pixel probability distribution map comprises class probabilities of each pixel point in all pixel points in the image to be classified;
the classification module is further configured to classify each of the plurality of superpixels by using the pre-trained second random forest model, so as to obtain a superpixel probability distribution map, where the superpixel probability distribution map includes class probabilities of random forest classifications corresponding to each superpixel;
the second processing module is used for obtaining the corrected class probability of each pixel point based on the class probability of the pixel point in the single-pixel probability distribution diagram and the class probability of the pixel point in the super-pixel probability distribution diagram;
the generation module is used for obtaining a result probability distribution diagram according to the corrected category probability of each pixel point, wherein the result probability distribution diagram comprises corrected category probabilities of each pixel point in all the pixels in the image to be classified.
7. The image processing apparatus of claim 6, wherein the second processing module comprises:
the acquisition module is used for acquiring a first weight representing that the super pixel to which the pixel point belongs and the adjacent super pixel of the super pixel belong to the same category;
the calculation module is used for obtaining the product of the class probability of the pixel point in the single-pixel probability distribution diagram and the first weight to obtain a first product;
the calculation module is further configured to obtain a product of the class probability of the pixel point in the super-pixel probability distribution diagram and the second weight, to obtain a second product, where the sum of the first weight and the second weight is 1;
the calculation module is further configured to obtain a sum of the first product and the second product, and obtain a corrected class probability of the pixel.
8. The image processing apparatus according to claim 7, wherein the acquisition module is specifically configured to:
calculating the distance between the super pixel center of the super pixel to which the pixel point belongs and the super pixel center of each adjacent super pixel, and obtaining the average value of all the calculated distances;
and obtaining the first weight based on the average value, wherein the first weight is the inverse of the sum of the average value and a preset threshold value.
9. An electronic device, comprising: the device comprises a memory and a processor, wherein the memory is connected with the processor;
the memory is used for storing programs;
the processor is configured to invoke a program stored in the memory to perform the method of any of claims 1-5.
10. A storage medium having stored thereon a computer program which, when executed by a computer, performs the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110027893.9A CN112766335B (en) | 2021-01-08 | 2021-01-08 | Image processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110027893.9A CN112766335B (en) | 2021-01-08 | 2021-01-08 | Image processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112766335A CN112766335A (en) | 2021-05-07 |
| CN112766335B true CN112766335B (en) | 2023-12-01 |
Family
ID=75701096
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110027893.9A Active CN112766335B (en) | 2021-01-08 | 2021-01-08 | Image processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112766335B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113409338B (en) * | 2021-06-24 | 2023-04-25 | 西安交通大学 | Super-pixel method based on probability distribution |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2919162A1 (en) * | 2014-03-13 | 2015-09-16 | Omron Corporation | Image processing apparatus and image processing method |
| CN108154104A (en) * | 2017-12-21 | 2018-06-12 | 北京工业大学 | A kind of estimation method of human posture based on depth image super-pixel union feature |
| CN108734228A (en) * | 2018-06-14 | 2018-11-02 | 中交第二公路勘察设计研究院有限公司 | The polarimetric SAR image random forest classification method of comprehensive multiple features |
| WO2020062360A1 (en) * | 2018-09-29 | 2020-04-02 | 深圳大学 | Image fusion classification method and apparatus |
-
2021
- 2021-01-08 CN CN202110027893.9A patent/CN112766335B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2919162A1 (en) * | 2014-03-13 | 2015-09-16 | Omron Corporation | Image processing apparatus and image processing method |
| CN108154104A (en) * | 2017-12-21 | 2018-06-12 | 北京工业大学 | A kind of estimation method of human posture based on depth image super-pixel union feature |
| CN108734228A (en) * | 2018-06-14 | 2018-11-02 | 中交第二公路勘察设计研究院有限公司 | The polarimetric SAR image random forest classification method of comprehensive multiple features |
| WO2020062360A1 (en) * | 2018-09-29 | 2020-04-02 | 深圳大学 | Image fusion classification method and apparatus |
Non-Patent Citations (2)
| Title |
|---|
| 基于纹元森林和显著性先验的弱监督图像语义分割方法;韩铮;肖志涛;;电子与信息学报(03);全文 * |
| 概率加权测地距离的脑部MR图像超像素分割;赵晶;周元峰;魏广顺;辛士庆;高珊珊;;计算机辅助设计与图形学学报(05);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112766335A (en) | 2021-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108805828B (en) | Image processing method, device, computer equipment and storage medium | |
| Montazer et al. | An improved radial basis function neural network for object image retrieval | |
| CN115294409B (en) | Video processing method, system and medium for security monitoring | |
| CN111899163A (en) | Efficient structure preservation for generating single image super-resolution in countermeasure networks | |
| CN106338733B (en) | Forward-Looking Sonar method for tracking target based on frogeye visual characteristic | |
| CN112328715B (en) | Visual positioning method, training method of related model, related device and equipment | |
| CN111340077B (en) | Attention mechanism-based disparity map acquisition method and device | |
| CN110689043A (en) | Vehicle fine granularity identification method and device based on multiple attention mechanism | |
| WO2021120961A1 (en) | Brain addiction structure map evaluation method and apparatus | |
| CN110969619A (en) | Method and device for automatically identifying primary tumor of nasopharyngeal carcinoma | |
| WO2020168648A1 (en) | Image segmentation method and device, and computer-readable storage medium | |
| CN111223128A (en) | Target tracking method, device, equipment and storage medium | |
| CN111915015A (en) | Abnormal value detection method and device, terminal equipment and storage medium | |
| CN112819199A (en) | Precipitation prediction method, device, equipment and storage medium | |
| CN112766335B (en) | Image processing method and device, electronic equipment and storage medium | |
| Quan et al. | Scattering feature-driven superpixel segmentation for polarimetric SAR images | |
| Yang et al. | Polarimetric SAR image classification using geodesic distances and composite kernels | |
| Karine et al. | A novel statistical model for content-based stereo image retrieval in the complex wavelet domain | |
| Kumar et al. | Automatic image segmentation using wavelets | |
| Gu et al. | A classification method for polsar images using SLIC superpixel segmentation and deep convolution neural network | |
| Kekre et al. | SAR image segmentation using vector quantization technique on entropy images | |
| CN116843906A (en) | Target multi-angle intrinsic feature mining method based on Laplace feature mapping | |
| Gong et al. | A superpixel segmentation algorithm based on differential evolution | |
| Geng et al. | Robust core tensor dictionary learning with modified gaussian mixture model for multispectral image restoration | |
| CN113627324A (en) | Matching method and device of face images, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |