CN114415254B - Multi-case weak supervision mars surface morphology detection method based on online learning - Google Patents
Multi-case weak supervision mars surface morphology detection method based on online learning Download PDFInfo
- Publication number
- CN114415254B CN114415254B CN202210072417.3A CN202210072417A CN114415254B CN 114415254 B CN114415254 B CN 114415254B CN 202210072417 A CN202210072417 A CN 202210072417A CN 114415254 B CN114415254 B CN 114415254B
- Authority
- CN
- China
- Prior art keywords
- case
- mars
- level
- candidate
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01V—GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS
- G01V8/00—Prospecting or detecting by optical means
- G01V8/10—Detecting, e.g. by using light barriers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Immunology (AREA)
- Pathology (AREA)
- General Life Sciences & Earth Sciences (AREA)
- Geophysics (AREA)
- Image Analysis (AREA)
Abstract
A multi-case weak supervision mars surface morphology detection method based on online learning belongs to the field of machine vision object detection. The method aims at the problems that the existing weak supervision object detection method is low in detection precision and converges on a local optimal solution. Generating a plurality of candidate boxes by adopting a selective search algorithm on the Mars long-shot image; extracting image features of the Mars perspective image by adopting a VGG16 network model; further obtaining the full connection characteristic of each candidate frame; in the weak supervision detection network, inputting the full connection characteristics of the candidate frames, judging the category of each candidate frame through classification and detection branches, scoring the position information of the candidate frames, and finally multiplying the scores of the two branches to obtain the score of the candidate frame as a case level label; and the K-grade refined network layer takes the score of each candidate box of the multi-case learning network or the previous-grade branch as supervision information, trains other optimized branches of the network, and performs backward propagation calculation. The method is used for detecting the surface topography and the target of the mars.
Description
Technical Field
The invention relates to a multi-case weak supervision mars surface morphology detection method based on online learning, and belongs to the technical field of machine vision object detection.
Background
The mars are the geostationary planets in the solar system, which are close to the earth and are the most similar to the earth in the solar system; the discovery of water makes mars one of the most likely life-gesting stars recognized and one of the main targets for developing space exploration. At present, space missions have attracted attention in earth satellites and manned space engineering, and the development of deep space exploration is a follow-up focus and has great significance for technological progress and social development. Because the mars environment is complicated and changeable and is often accompanied by weathers such as sand and dust, the key technology of the mars detection at present is to acquire a high-resolution mars image and detect objects/terrains on the surface of the mars by using the high-resolution image, so that the mars can stably and safely land on the surface of the mars, and further develop subsequent related scientific research tasks.
The visual task is a key link for realizing autonomous work of the Mars train, and the Mars train carries binocular visual environment perception and autonomous path planning technology at present; as the detection of mars advances, more vision techniques will be applied to the mars car to accomplish more complex detection tasks. The Mars detector carries on the Mars tour car and carries out the planet and survey the task at the Mars, and the detector is for better messenger's Mars tour car steadily, land on the Mars surface safely, and it is crucial to obtain the topography and geomorphology condition on Mars surface.
At present, a classical mainstream framework for object detection includes a YOLO algorithm based on bounding box regression, an RCNN series algorithm based on candidate region generation, and the like, but the classical object detection algorithms all adopt a fully supervised training mode, and case-level labeling is required on a training set (case-level labeling means that not only image types are given during network training, but also a target positioning frame is given with a center coordinate and a length and a width of a bounding box to represent an accurate position), case-level training is performed through a regression idea and a candidate region-based idea, and a result of the type and the position of a detected object is further obtained. However, case-level labeling information does not exist in the acquired mars topographic and topographic images at present, and if a manual labeling mode is adopted, the subjectivity of case-level labeling is too strong, and manual labeling is a time-consuming and labor-consuming process, and the case-level labeling conditions are difficult to meet.
Under the condition that the case-level labeling information is missing, the object detection task is usually realized by adopting a weak supervised learning mode at present. In the object detection task based on the weak supervised learning, the object detection can be realized only by labeling the object type information (without labeling the position information). However, in the field of weakly supervised object detection, there are two main problems: (1) The model is sensitive to initialization, and the detection precision is low; (2) The method is easy to converge to a local optimal solution, and the visual expression is that only the most characteristic region of the object can be detected, but not all object regions, so that the object positioning fails.
Disclosure of Invention
Aiming at the problems of low detection precision and convergence to a local optimal solution of the existing weak supervision object detection method, the invention provides a multi-case weak supervision mars surface morphology detection method based on online learning.
The invention relates to a multi-case weak supervision mars surface morphology detection method based on online learning, which comprises the following steps,
forming a training set by using Mars perspective images; configuring a terrain category label for each Mars long shot image;
the method comprises the steps that an online network is set to comprise a candidate box generating unit, a VGG16 network model and a weak supervision detection network;
a candidate frame generating unit generates a plurality of candidate frames of target objects or target terrains for each Mars long shot image by adopting a selective search algorithm; extracting image features of each Mars perspective image by adopting a VGG16 network model pre-trained on ImageNet; combining the image characteristics and the position information of each candidate frame to obtain the full-connection characteristics of each candidate frame;
detecting the full-connection characteristics of each candidate frame by adopting a weak supervision detection network, and obtaining a position detection score of the candidate frame including a target object or a target terrain according to a detection result; classifying the full-connection characteristics of each candidate frame, and obtaining a classification score according to a classification result; obtaining a primary case-level label of the candidate box according to the dot product result of the position detection score and the category score of the candidate box;
taking the primary case-level label as supervision information, and performing optimization processing on the full connection characteristics of each candidate frame step by adopting a K-grade fine network layer to obtain a final case-level label of the candidate frame;
comparing the case-level label output by each refinement network layer with the case-level label or the primary case-level label output by the adjacent first refinement network layer, comparing the classification result of each candidate box with the terrain category true value to obtain a loss function, and optimizing the weak supervision detection network based on the loss function to obtain a final weak supervision detection network;
and carrying out Mars surface morphology detection on the Mars long shot image acquired in real time by adopting a final weak supervision detection network.
According to the multi-case weak supervision Mars surface morphology detection method based on online learning, the terrain category labels comprise hills, ravines, stone regions and flat regions.
According to the multi-case weak supervision Mars surface morphology detection method based on online learning, the process of obtaining the final case-level label of the candidate box by adopting the K-level refined network layer comprises the following steps:
in a first-level refinement network layer, a softmax classifier is adopted to carry out case-level classification on the full-connection features of each candidate frame, and case-level classification results are adopted to correct the elementary case-level labels to obtain first-level corrected case-level labels;
in a second-level refinement network layer, performing case-level classification on the full-connection features of each candidate frame by adopting a softmax classifier, and correcting a first-level correction case-level label by adopting a case-level classification result to obtain a second-level correction case-level label;
……
in a K-level refinement network layer, a softmax classifier is adopted to perform case-level classification on the full-connection features of each candidate frame, and a case-level classification result is adopted to correct a K-1-level correction case-level label to obtain a K-level correction case-level label as a final case-level label.
According to the multi-case weak supervision mars surface morphology detection method based on online learning, the VGG16 network model obtains the image characteristics of the mars long-range images through convolution operation; and combining the image characteristics to obtain the full connection characteristics of the candidate frames for each candidate frame through the RoI pooling operation.
According to the multi-case weak supervision mars surface morphology detection method based on online learning, 1000 candidate frames of target objects or target terrains are generated in each mars long-shot image.
According to the multi-case weak supervision mars surface morphology detection method based on online learning, the VGG16 network model is propagated reversely through weights with 16 layers of parameters.
According to the multi-case weak supervision mars surface morphology detection method based on online learning, the mars distant view image is provided by a mars32k data set and a GMSRI data set of NASA.
According to the multi-case weak supervision mars surface morphology detection method based on online learning, after an online network is trained, the online network is verified by adopting a verification set, and the online network is tested by adopting a test set; the proportion of the training set, the verification set and the test set is 3:1:1.
the invention has the beneficial effects that: the method can be used for detecting the objects on the surface of the Mars and the terrain under the condition of scarce training data labeling information in relevant tasks of Mars detection. The method is used for executing the detection task of landform and landform when the Mars detector lands in the actual detection process, so that the Mars vehicle can better find the position which is most suitable for the landing of the patrol vehicle.
The method only finishes the detection of the landform and the landform of the Mars through image-level marking information, and provides a chance for a detector to land on the Mars and select the surface position of the celestial sphere with the most scientific value in subsequent planning tasks. In the multi-case weak supervision network, the full connection characteristic of each candidate frame is input, the functions of two parallel classification and detection branches are respectively used for judging the category of each candidate frame and scoring the position information of each candidate frame, and finally the scores of the classification branches and the detection branches are multiplied to obtain the score of the candidate frame as the case level label of the candidate frame. In a K-grade refined network layer, the scores of each candidate box of a multi-case learning network or a previous-grade branch are used as supervision information, other optimized branches of the network are trained, backward propagation calculation is carried out, and the detection precision is further improved.
The method breaks through the limitation that the fully supervised object detection method needs case-level labeling information, solves the problems of low detection precision and convergence to the local optimal solution in the weakly supervised object detection method, and can achieve the purpose of Mars surface object and terrain detection under the condition of only having image-level supervision information (namely giving terrain and landform types existing in each image) by utilizing the method without manually labeling case-level labels.
The method provided by the invention is used as basic technical research work in a Mars detection task, can promote a subsequent planning task to a certain extent, and provides a certain technical support for the subsequent planning task.
Drawings
FIG. 1 is a schematic diagram illustrating the principle of the online learning-based multi-case weakly supervised Mars surface morphology detection method of the present invention;
FIG. 2 is a diagram of Mars perspective image comparison for four terrain categories;
FIG. 3 is a diagram of a package and example of a Mars perspective image;
FIG. 4 is a schematic structural diagram of a VGG16 network model;
FIG. 5 is a schematic diagram of a level K refinement network layer;
FIG. 6 is a graph comparing the results of experiments in the examples.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The invention is further described with reference to the following drawings and specific examples, which are not intended to be limiting.
In a first embodiment, referring to fig. 1 to 5, the present invention provides a method for detecting the surface morphology of a multi-case weak supervised mars based on online learning, including,
forming a training set by using a Mars perspective image; configuring a terrain category label for each Mars long shot image;
the online network is arranged to comprise a candidate box generation unit, a VGG16 network model and a weak supervision detection network;
a candidate frame generating unit generates a plurality of candidate frames of target objects or target terrains for each Mars long shot image by adopting a selective search algorithm; extracting image features of each Mars perspective image by adopting a VGG16 network model pre-trained on ImageNet; combining the image characteristics and the position information of each candidate frame to obtain the full-connection characteristics of each candidate frame;
detecting the full-connection characteristics of each candidate frame by adopting a weak supervision detection network, and obtaining a position detection score of the candidate frame including a target object or a target terrain according to a detection result; classifying the full-connection characteristics of each candidate frame, and obtaining a classification score according to a classification result; obtaining a primary case-level label of the candidate box according to the dot product result of the position detection score and the category score of the candidate box;
taking the primary case-level label as supervision information, and performing optimization processing on the full connection characteristics of each candidate frame step by adopting a K-grade fine network layer to obtain a final case-level label of the candidate frame;
comparing the case-level label output by each refinement network layer with the case-level label or the primary case-level label output by the adjacent first refinement network layer, comparing the classification result of each candidate box with the terrain category true value to obtain a loss function, and optimizing the weak supervision detection network based on the loss function to obtain a final weak supervision detection network;
and carrying out Mars surface morphology detection on the Mars long shot image acquired in real time by adopting a final weak supervision detection network.
The implementation mode can realize the detection of the objects on the surface of the mars and the terrain in the mars image, particularly, the detection technology of the objects on the surface of the mars and the terrain does not depend on a large database with mark information any more, the model can be trained only by knowing the simple object or terrain category information in the image and without marking the complex object position information, and the purpose of detecting the objects on the surface of the mars and the terrain is further achieved. The method mainly realizes the detection of the landform and the landform of the mars through the characteristic identification of objects (such as stones and fine sands) on the surface of the mars, and creates conditions for selecting the most scientific landing point in the mars landing or the subsequent detection tasks of the mars.
As an example, as shown in connection with fig. 2, the terrain category labels include hills, ravines, stone regions, and flat regions. These four class labels will be the image level labels for each Mars image to train the online network.
Further, with reference to fig. 1 and 5, the process of obtaining the final case-level label of the candidate box by using the K-level refinement network layer includes:
in a first-level refinement network layer, performing case-level classification on the full-connection features of each candidate frame by adopting a softmax classifier, and correcting the primary case-level labels by adopting case-level classification results to obtain first-level corrected case-level labels;
in a second-level refinement network layer, performing case-level classification on the full-connection features of each candidate frame by adopting a softmax classifier, and correcting a first-level correction case-level label by adopting a case-level classification result to obtain a second-level correction case-level label;
……
in a K-level refinement network layer, a softmax classifier is adopted to perform case-level classification on the full-connection features of each candidate frame, and a case-level classification result is adopted to correct a K-1-level correction case-level label to obtain a K-level correction case-level label as a final case-level label.
The present embodiment utilizes a multiple case learning approach to implement weakly supervised object and terrain detectors. Since there is no artificially labeled location information as the true value of the object location regression, weakly supervised objects and terrain detectors will usually converge to a locally optimal solution, resulting in a phenomenon of local focusing, such as highlighting only the locally most discriminating regions of the object and terrain. In order to improve the recognition rate, the present invention proposes an online optimization strategy, which includes several optimization branches in parallel with the multiple-instance learning detection network, as shown in fig. 1. During training, the output result of the weak supervision detection network or the previous fine network layer is used as supervision information to supervise and train the optimization branches of the subsequent fine network layer, and further the purpose of accurately detecting objects and landforms on the surface of the mars is achieved.
In summary, the method of the present invention includes an object detection part (based on a multi-case weak supervision detection network) and an optimization part (K-level fine network layer), where the object detection part actually performs coarse positioning on a target object, and performs element-wise operation after detecting and classifying the network to obtain a matrix of C × R, where C is the type of the target, i.e., dan Kuaiou, ravines, hills, and flat regions, and R is the number of candidate frames. Then, the situation of local focusing of the target frame is reduced through K times of fine networks, so that the output position of object detection is more accurate and meets the actual requirement.
The K-grade refined network layer is not alternately trained, the trained data are labeled again, the thought of repeated iteration and repeated training is further carried out, the K-grade refined network layer is repeatedly refined by a method of intersecting with the candidate area with the highest score and carrying out similarity comparison on the candidate area with the area larger than a certain threshold value, and the case-grade label of each candidate area is further corrected, so that a multi-case weak supervision mars surface object and terrain detection network based on online learning can be obtained, and the problems that the position of the existing multi-case weak supervision detector is inaccurate and the detection precision is low are solved.
Still further, as shown in fig. 1 and 4, the VGG16 network model obtains the image features of the Mars perspective image through convolution operation; and then combining the image characteristics to each candidate frame to obtain the full connection characteristics of the candidate frames through the RoI pooling operation.
By way of example, as shown in connection with fig. 1, each Mars perspective image generates 1000 candidate boxes of target objects or target terrain.
In a weak supervision detection network, for each Mars image data only having Mars landform category labels, firstly, a selective search algorithm is adopted to generate about 1000 possible positions of target objects and areas, the possible positions of each target object and area are called candidate areas (spots), and the candidate areas are marked by candidate boxes; and then, obtaining full connection features of each candidate region by using a VGG16 network model, and inputting the extracted full connection features into a subsequent detection classification network to realize object and terrain detection tasks in Mars images.
The VGG16 network model is back-propagated by weight with 16-layer parameters.
By way of example, the Mars perspective image is provided by the mars32k dataset and the GMRI dataset of NASA.
In practical use of the present embodiment, the specific type of the detection object may be determined according to practical problems of the user.
In the training process, the data set can be formed by the real Mars images shot by the curio Mars train and the Mars images generated by the learning method. NASA provides a data set that contains 32368 color images collected by a curio (Curiosity) mars train on mars and approximately 30000 color mars images generated during months 8 and 2018 and 11 months. These images show various geographical and geological features of mars, such as mountains and valleys, craters, sand dunes and rock terrain, etc., with an image resolution of 560 x 500px. Images of mars32k and GMSRI data sets comprise Mars near view, distant view, large rocks, fine sand particles and images shot in dim light, and because a detector needs to search for a landing flat area when the detector is away from the surface of the Mars by kilometers, the Mars distant view images are used as training sets, and the image categories are divided into four categories, namely hills, ravines, stone areas and flat areas. The only category labels are used for training the online network, the position information is not required to be marked, a large amount of manpower and material resources are not required to mark the database, and the deviation caused by the subjective property of manual marking can be avoided.
Furthermore, after the on-line network is trained, the on-line network is verified by adopting a verification set, and the on-line network is tested by adopting a test set; the proportion of the training set, the verification set and the test set is 3:1:1.
in conclusion, the method and the device can obtain higher detection rate of objects and terrains on the surface of the mars under the condition that position marking information is not needed. The development of a Mars image-based object and terrain detection technology can be promoted, and a certain technical support is provided for the related tasks of follow-up Mars detection.
The specific embodiment is as follows:
firstly, training samples are prepared according to the actual requirements of users, and then a weakly supervised object/terrain detector is trained according to a multiple case learning (MIL) method. And then, the detection precision of the weakly supervised object and the terrain is further improved by utilizing an online optimization strategy, and a more accurate Mars surface terrain detection result is obtained. Each section is described in detail below:
first, a training sample is prepared.
The multi-case learning based weakly supervised detection network is then trained.
The weakly supervised detection network is an end-to-end detection method, wherein in multiple case learning, the label of a packet is known, the label of a case is unknown, and the label of the packet only describes what kind of target exists in the image, but the position of the target is unknown. Taking the mars image as an example, each 560 × 500 mars image is a packet, and some regions (Patches) in the image are instances, as shown in fig. 3. As can be seen from fig. 3, the bag has hills and Dan Kuaiou, no ravines; there are only class labels for packets:
y＝[y 1 ,y 2 ,y 3 ,…，y n ]∈R C×l ，
wherein y is i A label indicating that the target is of type i, i =1,2,3, …, n; n is the number of regions (Patches) in the image, and C is the number of category labels;
y i the value is +1 or-1.
If the labeled type label is {1, -1,1}, it indicates that there are instances of a hill region, no instances of a ravine region, and instances of a stone region, and there is no specific location information of the hill region and the stone region in the label of the training sample. Next, for each input image of mars, 1000 locations where the target object/terrain is likely to appear are generated using a selective search algorithm, and each location where the target object/terrain is likely to appear is referred to as a candidate region (spots). And then, extracting features by using a pre-trained VGG16 network model, finally acquiring the features of each candidate region by using a RoI pooling method, further obtaining the full-link features of each candidate region, inputting the extracted full-link features into a subsequent detection classification network, and realizing an object/terrain detection task in the Mars image. The VGG16 network structure is named by back propagation through the weight with 16-layer parameters, and comprises the following parts: the feature extraction network, the full connection network and the parallel two-branch detection and identification network are shown in fig. 4, which is a structure of a weakly supervised mars surface object/terrain detector network based on multi-case learning.
And designing an online optimization strategy. In order to avoid that the output result of the bounding box is likely to appear on an object region with strong local information and is not an accurate and complete position, the embodiment proposes a K-time fine correction strategy of online learning on the basis of the weak supervision detector, and aims to realize the capability of transmitting the label information from the small region to the large region of the high-resolution bounding box, as shown in fig. 5. The weak supervision detection only has image-level labels and does not have case-level labels, and the idea of the invention is to calculate the cross entropy loss function through case set labels and the score matrix and reversely propagate and update the score matrix, so that case-level labels are particularly important to obtain. Firstly, the candidate area of { C +1} dimension is the probability of each class, the matrix output secondly is the matrix of R multiplied by C, the scoring matrix is optimized by calculating loss back propagation on the matrix through the case level label output in the previous refinement, each candidate area generates a vector of C +1 dimension, and K represents the current optimization times, wherein K = {0,1 …, K-1}.
Selecting NASA mThe ars32k and GMSRI data sets are used as training, verifying and verifying data of the invention, 60% of the data sets are used as training sets, loss is reduced through iteration times, 20% of the data sets are used as verifying sets, the generalization ability of the model at the moment is verified after certain iteration times, and the remaining 20% of the data sets are used as testing sets to prove the generalization ability of the network structure and the effectiveness of the algorithm in the detection of Mars terrain and landform. Pre-trained VGG16 is selected to extract the topographic features of mars, 20 epoch iteration times are adopted when a weak supervision basic detector is trained, and the learning rate of the first 10 epochs is 10 -5 And the learning rate of the last 10 epochs is 10 -6 . The optimizer chooses Adam, which introduces a momentum method that leaves the local optimum at parameter update, with momentum and weight decay set to 0.9 and 0.0005, respectively. After an online optimization strategy is introduced, a random gradient descent optimizer is adopted in network training, the mini-batch of an input image is optimized to be 2 (mini-batch = 2), the momentum and weight decay is still set to be 0.9 and 0.0005, and the threshold value of Iou during refinement is It =0.5. Finally, the non-maximum inhibition NMS was set to 0.3 to calculate the final evaluation indices mapp and CorLoc. The number of refinements was experimentally selected to be K =4. In order to increase the training data sample, a data augmentation mode is adopted, the probability p =0.5 of the random horizontal inversion of the input Mars image is set, the ratio of the length and the width of the image is fixed between 0.5 and 2, if the above conditions are not met, the shortest edges of the training set, the verification set and the test set are set to be {480, 576, 688, 864 and 1200}, and the longest edges are not more than 2000 pixels.
The multi-case weak supervision mars surface object and terrain detection network based on online learning trained in the steps can realize the mars surface object and terrain detection function without case level marking information, can be applied to selection of mars car landing points according to specific requirements and selection of related tasks with scientific value floorpoints in subsequent planning tasks of mars detection, is not limited by the requirement of a complete information marking database of the existing object detection method, and does not need to spend manpower and material resources on marking each training sample image.
Experiments prove that the method has high precision and accurate positioning for Mars surface terrain detection, and the data shown in the table 1 and the standard 2 are experimental result comparison data:
TABLE 1 comparative data table of experimental results with mAP (%) as evaluation index
TABLE 2 comparative data table of experimental results using CorLoc (%) as evaluation index
The mAP is an Average accuracy (mean Average Precision) which is an index for evaluating a test sample, and the Corloc is a Correct positioning rate (Correct positioning) which is an index for evaluating the positioning effect of a training sample in the training process. As can be seen from the comparative data, the mAP/Corloc detected by only carrying out weak supervision is 50.6%/69.4%, and compared with the strategy of introducing online optimization, the mAP/Corloc is improved to be 58.3%/78.3, so that the problem of weak supervision local focusing is obviously improved, and the effectiveness of the online optimization strategy is proved. Furthermore, the generated false true value is used as the fully supervised detector of the case level label of the training, and the final position coordinate information of the object is more accurate compared with the output result of the weakly supervised object detector by fully utilizing the strong regression capability of the fully supervised learning, thereby further laterally proving the effectiveness of the method. FIG. 6 is a diagram of experimental results, during which the positions of the Mars' surface topography are detected and candidate bounding boxes select objects and topographical frames from the Mars surface; the detector is used for finding a stone region, a hill region and a gully region, so that the selection of a flat region in the falling task is further completed, and a large stone region, a stone dense region, the hill region and the gully region can be clearly observed from the detection region.
Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that features described in different dependent claims and herein may be combined in ways different from those described in the original claims. It is also to be understood that features described in connection with individual embodiments may be used in other described embodiments.
Claims (4)
1. A multi-case weak supervision mars surface morphology detection method based on online learning is characterized by comprising the following steps,
forming a training set by using a Mars perspective image; configuring a terrain category label for each Mars long shot image;
the online network is arranged to comprise a candidate box generation unit, a VGG16 network model and a weak supervision detection network;
a candidate frame generating unit generates a plurality of candidate frames of target objects or target terrains for each Mars long shot image by adopting a selective search algorithm; extracting image features of each Mars perspective image by adopting a VGG16 network model pre-trained on ImageNet; combining the image characteristics and the position information of each candidate frame to obtain the full-connection characteristics of each candidate frame;
detecting the full-connection characteristics of each candidate frame by adopting a weak supervision detection network, and obtaining a position detection score of the candidate frame including a target object or a target terrain according to a detection result; classifying the full-connection characteristics of each candidate frame, and obtaining a classification score according to a classification result; obtaining a primary case-level label of the candidate box according to the dot product result of the position detection score and the category score of the candidate box;
taking the primary case-level label as supervision information, and performing optimization processing on the full connection characteristics of each candidate frame step by adopting a K-grade fine network layer to obtain a final case-level label of the candidate frame;
comparing the case-level label output by each refinement network layer with the case-level label or the primary case-level label output by the adjacent previous refinement network layer, comparing the classification result of each candidate box with the terrain category true value to obtain a loss function, and optimizing the weak supervision detection network based on the loss function to obtain a final weak supervision detection network;
carrying out Mars surface morphology detection on the Mars long shot image acquired in real time by adopting a final weak supervision detection network;
the process of obtaining the final case level label of the candidate box by adopting the K-level refinement network layer comprises the following steps:
in a first-level refinement network layer, performing case-level classification on the full-connection features of each candidate frame by adopting a softmax classifier, and correcting the primary case-level labels by adopting case-level classification results to obtain first-level corrected case-level labels;
in a second-level refinement network layer, performing case-level classification on the full-connection features of each candidate frame by adopting a softmax classifier, and correcting a first-level correction case-level label by adopting a case-level classification result to obtain a second-level correction case-level label;
……
in a K-level refinement network layer, performing case-level classification on the full-connection features of each candidate frame by adopting a softmax classifier, and correcting a K-1-level correction case-level label by adopting a case-level classification result to obtain a K-level correction case-level label as a final case-level label;
generating 1000 candidate frames of target objects or target terrains by each Mars long shot image;
the VGG16 network model performs back propagation through weights with 16-layer parameters;
the Mars perspective image is provided by the mars32k dataset and the GMRI dataset of NASA;
the VGG16 network model adopts 20 epoch iteration times when training the weak supervision detection network, and the learning rate of the first 10 epochs is 10 -5 And the learning rate of the last 10 epochs is 10 -6 (ii) a The optimizer selects Adam.
2. The method of claim 1, wherein the terrain category labels comprise hills, ravines, stone regions, and flat regions.
3. The multi-case weak supervision mars surface morphology detection method based on online learning of claim 2, wherein the VGG16 network model obtains image features of mars perspective images through convolution operation; and combining the image characteristics to obtain the full connection characteristics of the candidate frames for each candidate frame through the RoI pooling operation.
4. The multi-case weak supervision mars surface morphology detection method based on online learning of claim 3, wherein after the online network is trained, the online network is verified by using a verification set, and is tested by using a test set; the proportion of the training set, the verification set and the test set is 3:1:1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210072417.3A CN114415254B (en) | 2022-01-21 | 2022-01-21 | Multi-case weak supervision mars surface morphology detection method based on online learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210072417.3A CN114415254B (en) | 2022-01-21 | 2022-01-21 | Multi-case weak supervision mars surface morphology detection method based on online learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114415254A CN114415254A (en) | 2022-04-29 |
| CN114415254B true CN114415254B (en) | 2023-02-07 |
Family
ID=81275302
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210072417.3A Active CN114415254B (en) | 2022-01-21 | 2022-01-21 | Multi-case weak supervision mars surface morphology detection method based on online learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114415254B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107833213A (en) * | 2017-11-02 | 2018-03-23 | 哈尔滨工业大学 | A kind of Weakly supervised object detecting method based on pseudo- true value adaptive method |
| CN112464877A (en) * | 2020-12-10 | 2021-03-09 | 哈尔滨工业大学（深圳） | Weak supervision target detection method and system based on self-adaptive instance classifier refinement |
| CN113378829A (en) * | 2020-12-15 | 2021-09-10 | 浙江大学 | Weak supervision target detection method based on positive and negative sample balance |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170330059A1 (en) * | 2016-05-11 | 2017-11-16 | Xerox Corporation | Joint object and object part detection using web supervision |
| CA3061717A1 (en) * | 2018-11-16 | 2020-05-16 | Royal Bank Of Canada | System and method for a convolutional neural network for multi-label classification with partial annotations |
| US11810312B2 (en) * | 2020-04-21 | 2023-11-07 | Daegu Gyeongbuk Institute Of Science And Technology | Multiple instance learning method |
-
2022
- 2022-01-21 CN CN202210072417.3A patent/CN114415254B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107833213A (en) * | 2017-11-02 | 2018-03-23 | 哈尔滨工业大学 | A kind of Weakly supervised object detecting method based on pseudo- true value adaptive method |
| CN112464877A (en) * | 2020-12-10 | 2021-03-09 | 哈尔滨工业大学（深圳） | Weak supervision target detection method and system based on self-adaptive instance classifier refinement |
| CN113378829A (en) * | 2020-12-15 | 2021-09-10 | 浙江大学 | Weak supervision target detection method based on positive and negative sample balance |
Non-Patent Citations (2)
| Title |
|---|
| Multiple Instance Detection Network with Online Instance Classifier Refinement;Peng Tang et al.;《2017IEEE Conference on Computer Vision and Pattern Recognition》;20171231;第3059-3066页 * |
| 基于深度卷积神经网络的真实场景物体检测算法研究;张永强;《中国博士学位论文全文数据库 信息科技辑》;20210115(第01期);第26-32、40页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114415254A (en) | 2022-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111091105B (en) | Remote sensing image target detection method based on new frame regression loss function | |
| CN109614985B (en) | Target detection method based on densely connected feature pyramid network | |
| Atkinson | Sub-pixel target mapping from soft-classified, remotely sensed imagery | |
| Gorsevski et al. | Landslide detection and susceptibility mapping using LiDAR and an artificial neural network approach: a case study in the Cuyahoga Valley National Park, Ohio | |
| CN110084195B (en) | Remote sensing image target detection method based on convolutional neural network | |
| Chauhan et al. | Landslide susceptibility zonation through ratings derived from artificial neural network | |
| CN109709603A (en) | Seismic horizon identification and method for tracing, system | |
| CN107066995A (en) | A kind of remote sensing images Bridges Detection based on convolutional neural networks | |
| CN110334705A (en) | A kind of Language Identification of the scene text image of the global and local information of combination | |
| CN108830188A (en) | Vehicle checking method based on deep learning | |
| CN109598243B (en) | Moon surface safe landing area selection method and system | |
| Pan et al. | A generic FCN-based approach for the road-network extraction from VHR remote sensing images–using openstreetmap as benchmarks | |
| CN113033520A (en) | Tree nematode disease wood identification method and system based on deep learning | |
| CN104156943B (en) | Multi objective fuzzy cluster image change detection method based on non-dominant neighborhood immune algorithm | |
| CN112396039A (en) | Mars grid terrain map generation method based on neighborhood relationship | |
| CN111383273B (en) | High-speed rail contact net part positioning method based on improved structure reasoning network | |
| CN114913434B (en) | High-resolution remote sensing image change detection method based on global relation reasoning | |
| Marcer | Rock glaciers automatic mapping using optical imagery and convolutional neural networks | |
| CN113936222A (en) | Mars terrain segmentation method based on double-branch input neural network | |
| Buyukdemircioglu et al. | Deep learning based building footprint extraction from very high resolution true orthophotos and nDSM | |
| Malinowski et al. | Mapping rill soil erosion in agricultural fields with UAV‐borne remote sensing data | |
| Pramoedyo et al. | Comparison of Random Forest and Naïve Bayes Methods for Classifying and Forecasting Soil Texture in the Area Around Das Kalikonto, East Java | |
| Nourzad et al. | Ensemble methods for binary classifications of airborne LiDAR data | |
| CN114415254B (en) | Multi-case weak supervision mars surface morphology detection method based on online learning | |
| CN112529837A (en) | Remote sensing image change detection algorithm based on coupling discrimination feature self-learning network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |