CN117437491B - Diffusion model fusion active learning-based deep space probe few-sample geological image category identification method - Google Patents

Diffusion model fusion active learning-based deep space probe few-sample geological image category identification method Download PDF

Info

Publication number
CN117437491B
CN117437491B CN202311693665.0A CN202311693665A CN117437491B CN 117437491 B CN117437491 B CN 117437491B CN 202311693665 A CN202311693665 A CN 202311693665A CN 117437491 B CN117437491 B CN 117437491B
Authority
CN
China
Prior art keywords
image
sample
deep space
geological
few
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311693665.0A
Other languages
Chinese (zh)
Other versions
CN117437491A (en
Inventor
王彬
黄鹏程
冯哲
孔祥晨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kunming University of Science and Technology
Original Assignee
Kunming University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kunming University of Science and Technology filed Critical Kunming University of Science and Technology
Priority to CN202311693665.0A priority Critical patent/CN117437491B/en
Publication of CN117437491A publication Critical patent/CN117437491A/en
Application granted granted Critical
Publication of CN117437491B publication Critical patent/CN117437491B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/091—Active learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)
  • Analysing Materials By The Use Of Radiation (AREA)

Abstract

The invention discloses a deep space probe few-sample geological image category identification method based on diffusion model fusion active learning, which comprises the following steps: acquiring an original deep space detector geological image; constructing a diffusion model for generating a low-resolution geologic image from a low-sample geologic image of a first deep space detector; obtaining a high-resolution extended geological image dataset according to the low-resolution geological image generated by generating the diffusion model; training the low-sample geological type recognition model of the active learning deep space detector according to the high-resolution extended geological image data set to obtain a trained low-sample geological type recognition model of the active learning deep space detector; and carrying out category identification on the geologic image/test set to be tested according to the trained active learning deep space probe few-sample geologic type identification model. The method can effectively solve the problem of unbalance among geologic data classes of few samples of the surface of the extraterrestrial celestial body, and can make up the problem of few sample data of the surface image of the extraterrestrial celestial body to a certain extent.

Description

Diffusion model fusion active learning-based deep space probe few-sample geological image category identification method
Technical Field
The invention relates to a deep space probe few-sample geological image category identification method based on diffusion model fusion active learning, and belongs to the technical application field of computer vision in deep space detection.
Background
The deep space exploration is an exploration activity of human beings on deep space environment and celestial bodies, and future extraterrestrial celestial body exploration tasks require that the detector has the capability of safely and accurately landing in a specific area with higher scientific value, which not only provides great challenges for a navigation guidance control system, but also provides higher requirements for landing area selection. The landing site is selected by considering not only the reduction of landing risk but also the characteristics of landform, geology and the like. Generally, in the task planning stage of the deep space probe, rough selection is carried out on a landing zone according to the existing conditions of terrain, illumination, communication and the like; however, after the detector approaches the target celestial body, the acquired navigation image can be utilized to acquire celestial body surface information, automatically identify different region characteristics and perform 'carefully choosing' on land points, so that the detector can be ensured to independently, accurately and safely land on a region of interest with few samples and high scientific value.
The scientific value of the potential landing zone is generally evaluated by considering scientific exploration diversity, determining geologic structures and years, and weather evolution evidence and preserving vital signs. The identification of the geologic features of the surface landing area of the extraterrestrial star based on the microscopic morphological features and the material component features has important significance, taking Mars surface image dataset HiRISE Orbital Data Set (v 3) as an example, researchers find that Mars surfaces can be divided into 8 geologic categories, and the scientific value and brief introduction of few sample categories are as follows: bright sand dunes (Bright dune): is a topography formed by the accumulation of fine particulate matter (such as dust and sand) on the Mars, which is known as brighter in color than the surrounding ground. The shape and position of the dune are often affected by wind force, and the direction and intensity of wind on the Mars can be reflected. Merle pit (Crater): meteorite craters are formed by asteroid or comet striking the surface of a spark. The number of merle pits on the Mars is very large, especially in the southern hemisphere, indicating that the Mars surface has undergone a long geological evolution process. The shape and size of the merle can also provide information about the impact material. Dark dune (Dark dune): similar to bright sand dunes, dark dunes are also formed from a pile of particulate matter, but they are generally composed of darker matter (e.g., volcanic sand grains) and thus appear darker than bright dunes. Impinging jet (Impact ejecta): is the material that is ejected from the pit by impact and falls into the surrounding area as the merle pit forms. The impinging jet may contain some raw material that is not melted by the impingement and is therefore important for understanding the geological history of the spark. Swiss cheese-like geology (SWISS CHEESE): this is a unique topography in the south-pole region of Mars, which is known as Swiss cheese. The geology is formed by melting carbon dioxide ice on the earth surface under solar radiation. Diagonal stripe (Slope): is a dark relief that appears on steep slopes on sparks, and may be formed by dust landslide or liquid flow. Although the mechanism of its formation is not clear, the diagonal stripes may reflect short-term geological activity on the Mars. Spider geology (Spider): is a unique landform in the south-pole area of Mars, and is named because of its shape similar to a spider. It is due to the spring time period, carbon dioxide ice in the subsurface melts and forms across the surface. Spider topography is an important sign of seasonal changes on sparks. Others (Other): there are many other types of geologic features on Mars, which in the dataset are characterized by preference for flat, broad, safe landing plains. Autonomous identification of few samples of the extraterrestrial celestial body to guide the probe to fall is important to scientific community in studying the conditions of the starry rock and soil under different geological conditions.
The existing few-sample autonomous identification method is mostly prone to being directly used for processing the few-sample image data of various extraterrestrial celestial bodies by utilizing the existing computer vision method, and is used for directly carrying out model learning on the few-sample image of the extraterrestrial celestial bodies, so that the application of identifying different extraterrestrial celestial body geological categories and the like is realized. However, unlike the general geologic structure image classification problem, the celestial body surface features are various but are uneven in distribution and quantity, so that the problem of data imbalance among classes is inevitably existed in samples for training an automatic classification model, the autonomous recognition capability and detection precision of the samples on the satellite are severely limited, and the accuracy requirement and timeliness requirement of the existing landing site geologic recognition model which depends on celestial body surface image data in autonomous landing can not be met.
Disclosure of Invention
The invention provides a deep space probe few-sample geological image category identification method based on diffusion model fusion active learning, which at least solves the problem of data unbalance among few-sample categories caused by uneven distribution and quantity of various surface features of the existing celestial body.
The technical scheme of the invention is as follows:
According to a first aspect of the invention, a deep space probe few-sample geological image category identification method based on diffusion model fusion active learning is provided, comprising the following steps: acquiring an original deep space detector geological image; the original deep space detector geological image comprises a first deep space detector few-sample geological image and a first deep space detector multi-sample geological image; constructing a diffusion model for generating a low-resolution geologic image from a low-sample geologic image of a first deep space detector; obtaining a high-resolution extended geological image dataset according to the low-resolution geological image generated by generating the diffusion model; training the low-sample geological type recognition model of the active learning deep space detector according to the high-resolution extended geological image data set to obtain a trained low-sample geological type recognition model of the active learning deep space detector; and carrying out category identification on the geologic image/test set to be tested according to the trained active learning deep space probe few-sample geologic type identification model.
The deep space probe few-sample geological image condition generation diffusion model comprises the following steps: according to the UNet network, performing a training process of forward diffusion and noise adding to an original few-sample geological image and a process of reverse denoising to generate a low-resolution geological image; the training process of forward diffusion noise addition to the original few-sample geological image uses a classifier-free guiding method and an index moving average method.
The classifier-free guiding method comprises the following steps: when UNet is trained, geological category label information is added according to preset probability, and the geological category label information is used for guiding noise prediction together with the current time t added in the noise prediction process.
The index moving average method has the expression:
w＝β·wold+(1-β)·wnew；
Wherein w represents the total weight of the models of the current training round, w old represents the old weight of the models of the previous round, w new represents the new weight of the models of the current training round, and beta represents the proportion of the old model weights.
The obtaining a high resolution extended geologic image dataset from the low resolution geologic image generated by generating the diffusion model comprises: training an SR3 model according to the original deep space detector geological image to obtain a trained SR3 model; inputting a low-resolution geologic image generated by the first deep space probe few-sample geologic image into a trained SR3 model to obtain a first high-resolution few-sample geologic image; expanding the first high-resolution few-sample geological image to obtain a second high-resolution few-sample geological image; downsampling the multi-sample geological image of the first deep space detector to obtain a multi-sample geological image of the second deep space detector; and constructing a high-resolution extended geological image data set from the first deep space detector few-sample geological image, the second high-resolution few-sample geological image and the second deep space detector multi-sample geological image.
Downsampling the first deep space probe multisample-like geologic image satisfies a difference between a maximum duty cycle and a minimum duty cycle in the high resolution expanded geologic image dataset of less than 10%.
The active learning deep space probe few sample geologic type recognition model uses an active learning sampling strategy KCENTERGREEDY to select samples in each round of cycles in the recognition model.
According to a second aspect of the present invention, there is provided a deep space probe few-sample geologic image category recognition system based on diffusion model fusion active learning, comprising: the acquisition module is used for acquiring the geological image of the original deep space detector; the original deep space detector geological image comprises a first deep space detector few-sample geological image and a first deep space detector multi-sample geological image; the construction module is used for constructing a diffusion model for generating a low-resolution geologic image from the low-sample geologic image of the first deep space detector; the acquisition module is used for acquiring a high-resolution extended geological image data set according to the low-resolution geological image generated by the diffusion model; the recognition module is used for training the recognition model of the few-sample geological type of the active learning deep space probe according to the high-resolution extended geological image data set, and obtaining a trained recognition model of the few-sample geological type of the active learning deep space probe; and carrying out category identification on the geologic image/test set to be tested according to the trained active learning deep space probe few-sample geologic type identification model.
According to a third aspect of the present invention, there is provided a processor for running a program, wherein the program is executed to perform the deep space probe low sample geological image classification recognition method based on diffusion model fusion active learning as described in any one of the above.
According to a fourth aspect of the present invention, there is provided a computer readable storage medium, the computer readable storage medium including a stored program, wherein when the program is run, the computer readable storage medium is controlled to execute the deep space probe few-sample geological image classification recognition method based on diffusion model fusion active learning according to any one of the above.
The beneficial effects of the invention are as follows: the method can effectively solve the problem of unbalance among geologic data of few samples on the surface of the extraterrestrial celestial body, and can compensate the problem of few sample data of the surface image of the extraterrestrial celestial body to a certain extent, and simulation experiment results show that the method can improve the accuracy of identifying few samples based on machine vision and can better meet the requirements of autonomy, accuracy and instantaneity of the deep space probe in the process of carefully selecting landing points.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a schematic of a diffusion model;
FIG. 3 is a schematic diagram of training the geological data of the landing sites of the region of interest of the SR3 model;
FIG. 4 is a schematic diagram of a SR3 super-resolution few-sample geologic image generation process;
FIG. 5 is a schematic diagram of active learning for imbalances between geologic data classes for few samples of stars;
FIG. 6 is a schematic diagram of an imbalance between Hirise-map-proj-v3_2 Mars geological data classes;
FIG. 7 is a diagram of DDPM generating a super-resolution map of small sample class Mars region of interest landing site data (left) and DDPM-SR3 for small sample class Mars region of interest landing site data (right);
FIG. 8 is a class classification confusion matrix inferred from each model pair Hirise-map-proj-v3_2 Mars geological test dataset; wherein, fig. 8 (a) expands the dataset for ResNet18+ raw dataset, fig. 8 (b) expands the dataset for ResNet18+ DDPM-SR3, fig. 8 (c) actively learns +raw dataset for KCENTERGREEDY, fig. 8 (d) actively learns + DDPM-SR3 for KCENTERGREEDY;
FIG. 9Hirise-map-proj-v3_2 Mars geological training versus test dataset spider class diagram.
Detailed Description
The invention will be further described with reference to the drawings and examples, but the invention is not limited to the scope.
Example 1: 1-9, according to a first aspect of an embodiment of the present invention, there is provided a deep space probe few-sample geological image classification recognition method based on diffusion model fusion active learning, including: acquiring an original deep space detector geological image; the original deep space detector geological image comprises a first deep space detector few-sample geological image and a first deep space detector multi-sample geological image; the whole original deep space detector geological image is divided into a training set, a verification set and a test set; training the subsequent model, performing corresponding steps of processing by using a training set, and participating in training; constructing a diffusion model for generating a low-resolution geologic image from a low-sample geologic image of a first deep space detector; obtaining a high-resolution extended geological image dataset according to the low-resolution geological image generated by generating the diffusion model; training the low-sample geological type recognition model of the active learning deep space detector according to the high-resolution extended geological image data set to obtain a trained low-sample geological type recognition model of the active learning deep space detector; and carrying out category identification on the geologic image/test set to be tested according to the trained active learning deep space probe few-sample geologic type identification model.
Further, the deep space probe low sample geological image condition generation diffusion model comprises: according to the UNet network, performing a training process of forward diffusion and noise adding to an original few-sample geological image and a process of reverse denoising to generate a low-resolution geological image; wherein a classifier-less guide (CFG) method and an exponential moving average (Exponential Moving Average, EMA) method are used in the training process of forward diffusion plus noise to the original few-sample geologic image.
Further, the classifier-free guiding method comprises the following steps: when UNet is trained, adding geologic category label information according to a preset probability and adding the current time t in the noise prediction process is used for guiding the prediction of noise (in the embodiment of the invention, the preset probability is set to be 90%, so that the diffusion model is not limited to generate the diversity of few-sample geology on the 10% probability), namely, linear interpolation between conditional and unconditional prediction noise:
Wherein Z represents noise information of the UNet at the predicted time t under the action of CFG, The prediction noise unconditionally generated at time t is represented by Z t,c, the conditional prediction noise at time t under the guidance of the class label as c, and p is represented by the conditional scale.
Further, the index moving average method is as follows: the current updated model is given smaller weight, the previous model is given larger weight, so that the parameter update of the model is obtained by gradually stepping according to the previous model parameter, and the purpose of the method is to enable the training process of generating the diffusion model by using the geological image with fewer samples to be smoother. The implementation of EMA:
w＝β·wold+(1-β)·wnew，
wherein w represents the total weight of the model of the current training round, w old represents the old weight of the model of the previous round, w new represents the new weight of the model of the current training round, and beta represents the proportion of the old model weight, and in the embodiment of the invention, beta takes 0.995.
Specifically, for the forward process, the calculation method for obtaining X t from X t-1 includes that X t represents image data corresponding to time t:
Where β t is a parameter that increases over time, β in the method increases linearly from 1×10 -4 to 2×10 -2, and Z is noise sampled randomly from a standard normal distribution.
Can directly obtain X t from X 0 by deduction, lead 1-beta t＝αt, andComprising the following steps:
The training process of the diffusion model aiming at the few-sample geologic image is to randomly sample the batch size data in the input process, sample the time step t, add noise to the batch size data by using the above method, input the noise data together with the current time step t and the corresponding original data, namely the few-sample geologic image category label information label, into a UNet network by a classifier-free guiding method to obtain the added noise of network prediction And do/>The invention adopts the least square error L2 loss for updating the network weight, and comprises the following steps:
The process of generating the low-resolution geological image by reverse denoising specifically comprises the following steps: predicting noise to be subtracted in each image restoration process by using a UNet network, sampling X T from the initial Gaussian noise, gradually generating samples X T-1,XT-2 and … with smaller noise, and finally denoising to obtain a low-resolution geological image X 0, wherein each time step T corresponds to a specific noise level, and X T represents image data corresponding to the moment of a training process T;
For the reverse process, essentially, from X t, the relation of X t-1 (updating one-step, less-sample geologic image data), using the noise estimate from UNet, the following can be derived:
Xt-1～N(μ,σ2)；
Where N (μ, σ 2) represents the mean μ, and the variance is σ 2 normal distribution.
Further, the obtaining a high resolution extended geologic image dataset from the low resolution geologic image generated by generating the diffusion model includes: training an SR3 model according to the original deep space detector geological image to obtain a trained SR3 model; inputting a low-resolution geologic image generated by the first deep space probe few-sample geologic image into a trained SR3 model to obtain a first high-resolution few-sample geologic image; expanding the first high-resolution few-sample geological image to obtain a second high-resolution few-sample geological image; downsampling the multi-sample geological image of the first deep space detector to obtain a multi-sample geological image of the second deep space detector; constructing a high-resolution extended geological image data set from the first deep space detector few-sample geological image, the second high-resolution few-sample geological image and the second deep space detector multi-sample geological image; downsampling the first deep space probe multisample-like geologic image satisfies a difference between a maximum duty cycle and a minimum duty cycle in the high resolution expanded geologic image dataset of less than 10%.
Specifically, the performance of generating a diffusion model by using a super-Resolution model SR3 based on a diffusion model to promote the condition of a Low-sample geological image of a deep space detector, wherein the super-Resolution of a Low-Resolution geological image generated by the condition generating diffusion model of the Low-sample geological image of the deep space detector is a High-Resolution clear geological image, more and finer address features are obtained, and the SR3 method is used for guiding the diffusion model to generate a finer High-Resolution Low-sample geological image for the fuzzy High-Resolution Low-sample geological image by taking a Low-Resolution (LR) 64×64 Low-sample geological image as a condition guide through three linear interpolation.
The SR3 method takes the LR few sample geological image as a condition, and sends the LR few sample geological image into UNet reconstruction after being embedded with a noise image, namely 6 channels are input in SR3, and 3 channels are output; and no longer take out of SR3The values are taken to be uniform distribution/>And t is no longer input as UNet, but directly from a uniform distribution/>The input is the noise magnitude value for that time step. After the SR3 model has super resolution, 224×224 high-resolution few-sample geological images can be obtained.
Using the high resolution augmented geologic image dataset generated from the diffusion model, defining it as an unlabeled geologic data poolAnd always expecting to add a geological data budget value B, wherein the goal is to select B geological data samples aiming at each category in each cycle, acquire a real data tag set Query z, integrate with the data of the previous training set, input a classification model for continuous training, and realize higher identification precision for fewer sample categories while maximizing the performance of the classification model;
Samples in each round of the loop in the recognition model are selected using an active learning sampling strategy KCENTERGREEDY. Specifically, n samples with the greatest distance from the nearest marker sample are found using the active learning sampling strategy KCENTERGREEDY. l=epoch×n, b+l represents the total number of training samples per round, and b represents the number of training samples per round 0; epoch represents the number of rounds and n represents the number of new increments per round (set to 8000 in the embodiment of the present invention). Through the selection of the optimal characteristic ground particle strategy, a geological identification model is input, the processes of selecting and expanding characteristic geological data, inputting the optimal characteristic ground particle and learning new characteristic geological point characteristics are closed, the purpose of labeling the characteristic geological data quantity by a small amount is achieved, and the identification precision of the detector on fewer samples on the satellite is continuously improved.
According to a second aspect of the embodiment of the present invention, there is provided a deep space probe few-sample geological image classification recognition system based on diffusion model fusion active learning, including: the acquisition module is used for acquiring the geological image of the original deep space detector; the original deep space detector geological image comprises a first deep space detector few-sample geological image and a first deep space detector multi-sample geological image; the construction module is used for constructing a diffusion model for generating a low-resolution geologic image from the low-sample geologic image of the first deep space detector; the acquisition module is used for acquiring a high-resolution extended geological image data set according to the low-resolution geological image generated by the diffusion model; the recognition module is used for training the recognition model of the few-sample geological type of the active learning deep space probe according to the high-resolution extended geological image data set, and obtaining a trained recognition model of the few-sample geological type of the active learning deep space probe; and carrying out category identification on the geologic image/test set to be tested according to the trained active learning deep space probe few-sample geologic type identification model. For portions of the foregoing that are not detailed for each module, reference may be made to the relevant description of the embodiments.
According to a third aspect of an embodiment of the present invention, there is provided a processor, configured to run a program, where the program runs to execute the deep space probe few-sample geologic image classification identification method based on diffusion model fusion active learning as described in any one of the above.
According to a fourth aspect of the embodiment of the present invention, there is provided a computer readable storage medium, where the computer readable storage medium includes a stored program, and when the program runs, the apparatus where the computer readable storage medium is controlled to execute the method for identifying a low sample geological image class of a deep space probe based on diffusion model fusion active learning according to any one of the above.
The following is described in connection with experimental data:
Aiming at the problem of low recognition precision of a deep space probe caused by unbalanced geological image sample data, hirise-map-proj-v3_2 Mars geological data under an atlas of a NASA planetary data system (PLANETARY DATA SYSTEM, PDS) is selected as method frame verification data.
Hirise-map-proj-v3_2 Mars image data with number of marked images 10815 and total number of enhanced images 73,031, 6 additional images were extended using the following method: rotating by 90 degrees; rotating 180 degrees; rotating 270 degrees; horizontally overturning; vertically overturning; random brightness adjustment. The imbalance of the geological image sample data is extremely serious, as shown in fig. 6. Other types account for 81.39%, the ratio is at least Impact ejecta types, and the ratio is only 0.68%.
Aiming at the unbalanced small sample type of the Mars geological data type, the invention carries out the deep space detector small sample geological image generation diffusion model training on 7 small sample types except Other in the total image after Hirise-map-proj-v3_2 Mars image data is enhanced, the guiding model carries out image generation on the geological small sample types to supplement a data set, and a deep space detector small sample geological image generation diffusion model schematic diagram is shown in figure 2. 1000 pictures are generated for each geologic less sample category.
Carrying out training on SR3 models by 7 less sample classes except Other in the total image after Hirise-map-proj-v3_2 Mars image data are enhanced, and obtaining trained SR3 models; 1000 images generated by each geological few-sample class according to the deep space detector few-sample geological image are respectively input into a trained SR3 model, and a corresponding first high-resolution few-sample geological image is obtained; the geological image with the first high resolution and few samples is expanded through random cutting, rotation (rotation by 90 degrees; rotation by 180 degrees; rotation by 270 degrees) and brightness adjustment, in the embodiment of the invention, the generated Mars few samples of various 1000 sheets are expanded to be 5 times of the original Mars few samples, the total number of generation of each category is 5000, namely, 35000 geological images with the second high resolution and few samples are newly generated, and all generated images are clear large images with 224 multiplied by 224; in addition, aiming at most Other types, 10000 224 multiplied by 224 pictures are obtained by downsampling; the schematic diagram of SR3 is shown in FIG. 3 and FIG. 4.
The total number of data sets obtained after the diffusion model and the SR3 model are generated under the condition is 56914, the ratio of the original Hirise-map-proj-v3_2 Mars geological data to the high-resolution extended geological image data set generated by the invention is shown in table 1, the ratio of Other categories in the total data is reduced from 81.39% in the original data to 17.57% in the high-resolution extended geological image data set, and the difference between the maximum ratio and the minimum ratio in the high-resolution extended geological image data set is less than 10% (the maximum ratio is 17.57%, and the minimum ratio is 9.61%) is satisfied.
TABLE 1
As shown in Table 1, the unbalance between the Mars geological image classes is greatly improved.
In order to improve the feature fineness performance of the Mars few-sample geologic image, the intrinsic features of the original Mars few-sample landing site geologic features are better restored, and the condition generation diffusion model DDPM is utilized to generate a high-resolution image which is super-divided into 224×224 high-resolution images consistent with the original Mars few-sample geologic data. The effect of generating the Mars sample-less geological data for the diffusion model (left panel) and the effect of generating the high-resolution Mars sample-less geological data for SR3 (right panel) are shown in FIG. 7.
TABLE 2
Evaluation index The obtained score
FID 1.49↓
IS 2.55↑
Table 2 shows objective index evaluation and data results of images generated by using the geological data of few Mars samples, wherein Table 2 is the objective evaluation index (+.gtoreq.s.the smaller and better the representative value, +.gtoreq.s.the larger and better the representative value) of the image quality generated by using the conditional diffusion model DDPM-SR 3. Wherein, the FID measures the difference between the generated image and the real image, and the IS measures the diversity and the authenticity of the generated image.
An active learning method for unbalance among geologic data classes of few stars samples is shown in figure 5.
In experimental settings, the recognition models were each chosen ResNet-18 (He K et al 2016), resNet-18 being a deep residual network (Deep Residual Network), which is a smaller model in the ResNet family, comprising 18 layers of neural networks. Experiments for KCENTERGREEDY active learning were set as follows: the initial training set is total 8000 pieces of extraterrestrial celestial body image data of 8 categories, 1000 samples and the data sample with the largest distance from the nearest marked sample are respectively selected for geological few sample categories except other categories in each subsequent training round, and are filled into the new round training set, and training is carried out for 5 rounds in total.
TABLE 3 Table 3
Table 3 shows the framework proposed herein: the diffusion model is generated by the geological image condition of the deep space probe with few samples, and the recognition model of the deep space probe with few samples is fused, namely, the KCENTERGREEDY active learning+ DDPM-SR3 expansion data set method has the superiority compared with other ablation methods. The accuracy of the method reaches 92.303% on the overall Mars geological data identification accuracy, and experiments show that the model identification Mars sample geological data category performance is greatly improved (from 39.311% to 66.238% and 26.927%) after the data set is expanded by DDPM-SR3, so that the DDPM-SR3 method has proved effectiveness in solving the Mars sample category imbalance sample problem.
And under the addition of KCENTERGREEDY active learning, the total recognition accuracy of the model is further improved. The KCENTERGREEDY active learning and original dataset method has low identification accuracy because the unbalanced small sample number in the original spark geological dataset is too small, so that the sampling capability of the KCENTERGREEDY method on new image data is limited. Specifically, for example, impact ejecta types have a ratio of only 0.68% in the original Mars geological dataset, and when the greedy distance measurement is performed on the raw Mars geological dataset by the KCENTERGREEDY active learning method, the raw Mars geological dataset contains a small amount of data, and a certain number of geological feature salient samples cannot be selected. However, from the experimental result, the KCENTERGREEDY active learning method improves the identification precision of the model on the small sample class of the Mars geological data class imbalance from 39.311% to 47.267%, and improves 7.956%. The applicability of KCENTERGREEDY active learning method to the solution of the problem of unbalanced Mars few sample data is proved.
To further demonstrate the ability of the proposed method to identify each Mars geological data category, FIG. 8 is a matrix of classification confusion for each class, derived by reasoning about Hirise-map-proj-v3_2 Mars geological test data sets for each model in Table 3.
As can be taken from fig. 8 (a), the method using only ResNet18+ raw dataset has high recognition accuracy for the Mars geological other class, but has rather poor recognition accuracy for the remaining classes excluding the other class, for example: for geologic class data with a true tag impact ejecta, the model classifies it as an other class, and this phenomenon is further explained by the fact that the other class in the original dataset is over-rated, so that the model tends to classify all classes as other for smaller training errors, however such training is over-fitted to the other class with the most occupancy in the original dataset.
Comparing FIG. 8 (a) with FIG. 8 (b), the method of ResNet + DDPM-SR3 extended dataset had a reduced accuracy in identifying other classes, but it was significantly better to classify the remaining geologic classes from other classes than the method using ResNet18 +original dataset alone. For example: comparing the classification conditions of the two methods on the slope structurally eak geological classes, the ResNet18+ original data set method divides the 49 slope structurally eak geological class pictures into 39 pieces to other classes in an error mode, and the ResNet18+ DDPM/SR3 extended data set method divides the 49 slope structurally eak geological class pictures into only 20 pieces to other classes in an error mode.
As can be seen by comparing fig. 8 (a), 8 (b) and 8 (c), the KCENTERGREEDY active learning+raw dataset method has classification accuracy for each category between the first two methods, and the reason for the lower recognition accuracy for the overall geologic test dataset is the same as mentioned above: the unbalanced small sample count in the original spark geological dataset is too small, limiting the ability of the KCENTERGREEDY method to sample new image data.
The method can be obtained by comparing KCENTERGREEDY active learning + DDPM-SR3 expansion data set of fig. 8 (d) with other methods, so that the identification precision of the other class with the largest data quantity in the Mars geological test set is reserved, and the identification precision of the unbalanced few sample class in the Mars geological data set is improved compared with other methods. It should be noted that, as can be seen from fig. 8, the number of the identification of the spider in the Hirise-map-proj-v3_2 Mars geological test data set is 0 in all the methods, the identification is obtained after the image data of the Hirise-map-proj-v3_2 Mars geological training set and the image data of the spider in the test set are checked, the image data of the spider in the test set has obvious differences from the characteristic distribution of the image in the training set, visually, the brightness of the spider picture in the training set is far higher than that of the spider picture in the test set, and the geological image mechanism is also obviously different, which is an important reason for limiting the identification precision of the spider geological image data by the model herein, thereby indicating that the invention is limited reasonably. FIG. 9 is a chart showing the comparison of Hirise-map-proj-v3_2 Mars geological training with the test dataset spider.
The KCENTERGREEDY active learning + DDPM-SR3 data set expansion method provided by the invention can be used for carrying out high-precision identification on other classes in a test set and simultaneously carrying out better identification on unbalanced few sample classes in a spark geological data set under the condition that the other classes in the original spark geological data set have overlarge proportion.
While the present invention has been described in detail with reference to the drawings, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention within the knowledge of those skilled in the art.

Claims (7)

1. A deep space probe few-sample geological image type identification method based on diffusion model fusion active learning is characterized by comprising the following steps:
acquiring an original deep space detector geological image; the original deep space detector geological image comprises a first deep space detector few-sample geological image and a first deep space detector multi-sample geological image;
Constructing a diffusion model for generating a low-resolution geologic image from a low-sample geologic image of a first deep space detector;
generating a low-resolution geologic image generated by the diffusion model according to the conditions, and obtaining a high-resolution extended geologic image dataset;
Training the low-sample geological type recognition model of the active learning deep space detector according to the high-resolution extended geological image data set to obtain a trained low-sample geological type recognition model of the active learning deep space detector; performing category identification on the geologic image/test set to be tested according to the trained active learning deep space probe few-sample geologic type identification model;
The deep space probe few-sample geological image condition generation diffusion model comprises the following steps: according to the UNet network, performing a training process of forward diffusion and noise adding to an original few-sample geological image and a process of reverse denoising to generate a low-resolution geological image; the forward diffusion noise adding method is used for training the original few-sample geological image, and a classifier-free guiding method and an index moving average method are used in the training process;
The generating the low-resolution geologic image generated by the diffusion model according to the condition to obtain a high-resolution extended geologic image data set comprises the following steps:
training an SR3 model according to the original deep space detector geological image to obtain a trained SR3 model;
inputting a low-resolution geologic image generated by the first deep space probe few-sample geologic image into a trained SR3 model to obtain a first high-resolution few-sample geologic image;
Expanding the first high-resolution few-sample geological image to obtain a second high-resolution few-sample geological image;
Downsampling the multi-sample geological image of the first deep space detector to obtain a multi-sample geological image of the second deep space detector;
Constructing a high-resolution extended geological image data set from the first deep space probe few-sample geological image, the second high-resolution few-sample geological image and the second deep space probe multi-sample geological image;
the active learning deep space probe few sample geologic type recognition model uses an active learning sampling strategy KCENTERGREEDY to select samples in each round of cycles in the recognition model.
2. The deep space probe few-sample geologic image category identification method based on diffusion model fusion active learning of claim 1, wherein the classifier-free guiding method is as follows: when UNet is trained, geological category label information is added according to preset probability, and the geological category label information is used for guiding noise prediction together with the current time t added in the noise prediction process.
3. The diffusion model fusion active learning-based deep space probe few-sample geologic image category identification method of claim 2, wherein the exponential moving average method is expressed as follows:
w＝β·wold+(1-β)·wnew；
Wherein w represents the total weight of the models of the current training round, w old represents the old weight of the models of the previous round, w new represents the new weight of the models of the current training round, and beta represents the proportion of the old model weights.
4. The diffusion model fusion active learning-based deep space probe few-sample geologic image category identification method of claim 1, wherein downsampling the first deep space probe multi-sample geologic image satisfies that a difference between a maximum duty ratio and a minimum duty ratio in the high-resolution extended geologic image dataset is less than 10%.
5. A kind of recognition system of the geological image of little sample of deep space probe based on diffusion model fuses the initiative study, characterized by, comprising:
the acquisition module is used for acquiring the geological image of the original deep space detector; the original deep space detector geological image comprises a first deep space detector few-sample geological image and a first deep space detector multi-sample geological image;
the construction module is used for constructing a diffusion model for generating a low-resolution geologic image from the low-sample geologic image of the first deep space detector;
the obtaining module is used for generating a low-resolution geologic image generated by the diffusion model according to the conditions and obtaining a high-resolution extended geologic image data set;
the recognition module is used for training the recognition model of the few-sample geological type of the active learning deep space probe according to the high-resolution extended geological image data set, and obtaining a trained recognition model of the few-sample geological type of the active learning deep space probe; performing category identification on the geologic image/test set to be tested according to the trained active learning deep space probe few-sample geologic type identification model;
The deep space probe few-sample geological image condition generation diffusion model comprises the following steps: according to the UNet network, performing a training process of forward diffusion and noise adding to an original few-sample geological image and a process of reverse denoising to generate a low-resolution geological image; the forward diffusion noise adding method is used for training the original few-sample geological image, and a classifier-free guiding method and an index moving average method are used in the training process;
The generating the low-resolution geologic image generated by the diffusion model according to the condition to obtain a high-resolution extended geologic image data set comprises the following steps:
training an SR3 model according to the original deep space detector geological image to obtain a trained SR3 model;
inputting a low-resolution geologic image generated by the first deep space probe few-sample geologic image into a trained SR3 model to obtain a first high-resolution few-sample geologic image;
Expanding the first high-resolution few-sample geological image to obtain a second high-resolution few-sample geological image;
Downsampling the multi-sample geological image of the first deep space detector to obtain a multi-sample geological image of the second deep space detector;
Constructing a high-resolution extended geological image data set from the first deep space probe few-sample geological image, the second high-resolution few-sample geological image and the second deep space probe multi-sample geological image;
the active learning deep space probe few sample geologic type recognition model uses an active learning sampling strategy KCENTERGREEDY to select samples in each round of cycles in the recognition model.
6. A processor, wherein the processor is configured to run a program, and wherein the program is configured to perform the deep space probe low sample geologic image classification method based on diffusion model fusion active learning of any of claims 1-4 when run.
7. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored program, wherein the program when run controls a device in which the computer readable storage medium is located to execute the deep space probe low sample geologic image category identification method based on diffusion model fusion active learning according to any one of claims 1-4.
CN202311693665.0A 2023-12-11 2023-12-11 Diffusion model fusion active learning-based deep space probe few-sample geological image category identification method Active CN117437491B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311693665.0A CN117437491B (en) 2023-12-11 2023-12-11 Diffusion model fusion active learning-based deep space probe few-sample geological image category identification method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311693665.0A CN117437491B (en) 2023-12-11 2023-12-11 Diffusion model fusion active learning-based deep space probe few-sample geological image category identification method

Publications (2)

Publication Number Publication Date
CN117437491A CN117437491A (en) 2024-01-23
CN117437491B true CN117437491B (en) 2024-06-14

Family

ID=89558535

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311693665.0A Active CN117437491B (en) 2023-12-11 2023-12-11 Diffusion model fusion active learning-based deep space probe few-sample geological image category identification method

Country Status (1)

Country Link
CN (1) CN117437491B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112819794A (en) * 2021-02-04 2021-05-18 青岛科技大学 Small celestial body meteorite crater detection method based on deep learning
CN116740340A (en) * 2023-07-11 2023-09-12 西安电子科技大学 Calculation spectrum imaging error correction method based on deep learning

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DK3121622T3 (en) * 2015-07-24 2021-09-13 Bergen Teknologioverforing As Method for predicting parameters of a geological formation
CN117197744A (en) * 2023-09-14 2023-12-08 雅砻江流域水电开发有限公司 Hydropower engineering high-steep side slope dangerous rock falling data set expansion method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112819794A (en) * 2021-02-04 2021-05-18 青岛科技大学 Small celestial body meteorite crater detection method based on deep learning
CN116740340A (en) * 2023-07-11 2023-09-12 西安电子科技大学 Calculation spectrum imaging error correction method based on deep learning

Also Published As

Publication number Publication date
CN117437491A (en) 2024-01-23

Similar Documents

Publication Publication Date Title
Du et al. Landslide susceptibility assessment based on an incomplete landslide inventory in the Jilong Valley, Tibet, Chinese Himalayas
Hodgson et al. Unmanned aerial vehicles for surveying marine fauna: assessing detection probability
Hartmann et al. Martian cratering 11. Utilizing decameter scale crater populations to study Martian history
Yalcin GIS-based landslide susceptibility mapping using analytical hierarchy process and bivariate statistics in Ardesen (Turkey): comparisons of results and confirmations
Li et al. GIS‐Based Landslide Susceptibility Mapping Using Information, Frequency Ratio, and Artificial Neural Network Methods in Qinghai Province, Northwestern China
CN109598243B (en) Moon surface safe landing area selection method and system
Banks et al. Crater population and resurfacing of the Martian north polar layered deposits
CN109597415B (en) Patrol device path planning method and system based on lunar surface safe landing area
CN108416820B (en) Remote sensing data-based geological map filling method for moon surface impact pit
Henney et al. Automated coronal hole detection using He I 1083 nm spectroheliograms and photospheric magnetograms
Greeley et al. Wind‐related features in Gusev crater, Mars
CN117437491B (en) Diffusion model fusion active learning-based deep space probe few-sample geological image category identification method
Anderson et al. Assessing the Martian surface distribution of aeolian sand using a Mars general circulation model
Smit et al. Downscaling legacy soil information for hydrological soil mapping using multinomial logistic regression
Hamlington et al. The emerging golden age of satellite altimetry to prepare humanity for rising seas
Xiao et al. A new approach to study terrestrial yardang geomorphology based on high‐resolution data acquired by unmanned aerial vehicles (UAVs): A showcase of whaleback yardangs in Qaidam Basin, NW China
Verkhoglyadova et al. Classification of high density regions in global ionospheric maps with neural networks
CN113469512B (en) Natural protected area determining method and device, electronic equipment and storage medium
Korycansky et al. Titan impacts and escape
CN114415254B (en) Multi-case weak supervision mars surface morphology detection method based on online learning
Visick Using remote sensing to track complex sinuous ripple migration on a barchan dune in Herschel Crater, Mars
Pillay et al. A locational analysis of rock art in the North Island, Aotearoa New Zealand
Miller et al. Engineering geomorphological mapping
Chojnacki et al. Surficial properties of landslides and surrounding units in Ophir Chasma, Mars
Pratama et al. Algorithm design for wind prediction in Berakit Bay, Bintan Island using Long Short-Term Memory (LSTM) method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant