CN117876840A - Remote sensing basic model rapid training method and system based on template editing - Google Patents
Remote sensing basic model rapid training method and system based on template editing Download PDFInfo
- Publication number
- CN117876840A CN117876840A CN202311623353.2A CN202311623353A CN117876840A CN 117876840 A CN117876840 A CN 117876840A CN 202311623353 A CN202311623353 A CN 202311623353A CN 117876840 A CN117876840 A CN 117876840A
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- basic model
- framework
- training
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 103
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000008569 process Effects 0.000 claims abstract description 29
- 230000008859 change Effects 0.000 claims abstract description 13
- 238000012544 monitoring process Methods 0.000 claims abstract description 10
- 238000005457 optimization Methods 0.000 claims description 18
- 238000007726 management method Methods 0.000 claims description 14
- 230000000007 visual effect Effects 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 10
- 238000013523 data management Methods 0.000 claims description 8
- 238000011156 evaluation Methods 0.000 claims description 7
- 238000012795 verification Methods 0.000 claims description 5
- 239000010881 fly ash Substances 0.000 claims description 3
- 230000010354 integration Effects 0.000 abstract description 7
- 238000012545 processing Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000012800 visualization Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a rapid training method and system for a remote sensing basic model based on template editing, relates to the technical field of remote sensing image processing, and aims to solve the technical problem that algorithm integration is difficult in the existing remote sensing basic model training process. The method comprises the following steps: acquiring a remote sensing sample data set marked in advance; determining a remote sensing basic model framework corresponding to task demands according to the task demands of the users; editing a corresponding algorithm frame and a mirror image environment for the remote sensing basic model framework based on task requirements; inputting the remote sensing sample data set into a remote sensing basic model framework, and training the remote sensing basic model framework by utilizing the edited algorithm framework and the mirror image environment; and visually monitoring the progress, parameter change and performance index of the remote sensing basic model framework in the training process.
Description
Technical Field
The invention relates to the technical field of remote sensing image processing, in particular to a rapid remote sensing basic model training method and system based on template editing.
Background
At present, remote sensing image processing is widely applied in various fields, wherein a model-based algorithm becomes one of key technologies in remote sensing image processing, however, the existing basic model training method has the problem that model algorithm integration is difficult, and efficiency and accuracy in practical application are limited.
In the prior art, a rapid training method based on template editing needs to integrate a plurality of algorithms and technologies, however, the integration and coordination between the plurality of algorithms and technologies are not easy, especially when they come from different research fields, which may lead to difficulties and complexity of algorithm integration, limit the practical application and popularization of the method, and on the other hand, the template selection, editing, training and the like need to be performed through a plurality of steps and operations, however, the current technology often lacks visual support for the whole training process, so that a user is difficult to understand and master the specific steps and operation processes of the method, and is also difficult to monitor and adjust each link in the training process. Therefore, the problems that an integrated algorithm is difficult and a process visualization is lacking are solved by the current remote sensing basic model rapid training method and system based on template editing.
Disclosure of Invention
In view of the above problems, the invention provides a rapid training method and system for a remote sensing basic model based on template editing.
According to a first aspect of the present invention, there is provided a rapid training method for a remote sensing base model based on template editing, including: acquiring a remote sensing sample data set marked in advance; determining a remote sensing basic model framework corresponding to task demands according to the task demands of the users; editing a corresponding algorithm frame and a mirror image environment for the remote sensing basic model framework based on task requirements; inputting the remote sensing sample data set into a remote sensing basic model framework, and training the remote sensing basic model framework by utilizing the edited algorithm framework and the mirror image environment; and visually monitoring the progress, parameter change and performance index of the remote sensing basic model framework in the training process.
According to the embodiment of the invention, the remote sensing sample data set is input into the remote sensing basic model framework, and before the remote sensing basic model framework is trained by utilizing the edited algorithm framework and the mirror image environment, the remote sensing basic model framework further comprises: parameter initialization setting is carried out on a remote sensing basic model framework; and creating a training task according to the remote sensing sample data set and the algorithm frame and the mirror image environment.
According to an embodiment of the present invention, inputting a remote sensing sample dataset into a remote sensing infrastructure model architecture, and training the remote sensing infrastructure model architecture using an edited algorithm framework and a mirrored environment includes: inputting the remote sensing sample data set into a remote sensing basic model framework for training according to the training task; and iteratively updating parameters of the algorithm framework by using an intelligent parameter adjustment optimization method.
According to an embodiment of the present invention, iteratively updating parameters of an algorithm framework using an intelligent parameter tuning optimization method includes: determining the hyper-parameters of an algorithm framework; iterative optimization of super parameters is carried out according to an intelligent parameter adjustment optimization algorithm; acquiring an evaluation index of the remote sensing basic model architecture on the verification set; and taking the part of the super parameter reaching the index threshold value in the iterative optimization process as the starting point of the next iteration according to the evaluation index until reaching the iteration termination condition.
According to the embodiment of the invention, the progress, parameter change and performance index of the remote sensing basic model framework in the training process are visually monitored: determining a visual dynamic library matched with the remote sensing basic model architecture; parameters and loss functions of the remote sensing basic model framework in the training process are transferred to a visual dynamic library in a dynamic mounting mode; and monitoring the progress, parameter change and performance index in the training process in real time according to the visual dynamic library.
According to an embodiment of the present invention, compiling a corresponding algorithm framework and a mirror image environment for a remote sensing infrastructure model architecture based on task requirements includes: creating a template, wherein the template is pre-integrated with at least one of a tensor flow framework, a Pytorch, a flyash framework, MXNet, mindSpore, caffe, caffe, and a just-in-time compilation framework; and selecting an algorithm framework corresponding to the task requirement from the template and configuring the algorithm framework into the remote sensing basic model framework.
According to an embodiment of the present invention, the template is further integrated with at least one of an x86_64 mirror architecture, a mips_64 mirror architecture, and an arm_64 mirror architecture.
According to an embodiment of the invention, creating training tasks from a remote sensing sample dataset and an algorithmic framework with mirrored environments comprises: and creating a stand-alone training task or a distributed training task according to the remote sensing sample data set and the algorithm frame and the mirror image environment.
According to an embodiment of the invention, a remote sensing sample dataset comprises: one of a remote sensing image dataset, a remote sensing text dataset, a remote sensing voice dataset, a remote sensing video dataset, a remote sensing electronic dataset and a remote sensing situation dataset.
The second aspect of the invention provides a remote sensing basic model rapid training system based on template editing, comprising: the sample data management module is used for acquiring a remote sensing sample data set marked in advance; the template management module is used for determining a remote sensing basic model framework corresponding to the task demand according to the task demand of the user; editing a corresponding algorithm frame and a mirror image environment for the remote sensing basic model framework based on task requirements; the training management module is used for inputting the remote sensing sample data set into the remote sensing basic model framework, and training the remote sensing basic model framework by utilizing the edited algorithm framework and the mirror image environment; and visually monitoring the progress, parameter change and performance index of the remote sensing basic model framework in the training process.
According to the rapid training method and system for the remote sensing basic model based on template editing, the algorithm framework and the mirror image environment are edited on the remote sensing basic model framework template according to task requirements, so that the integration between the algorithm and the technology is realized, and the problem that the algorithm and the technology corresponding to different fields are difficult to integrate and coordinate is solved. And meanwhile, the training process of the remote sensing basic model framework is visually monitored, so that a user can understand and master the progress, parameter change and performance index in the training process in real time, and the remote sensing basic model framework is beneficial to rapid training and optimization.
Drawings
The foregoing and other objects, features and advantages of the invention will be apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:
FIG. 1 schematically illustrates a flow chart of a remote sensing base model rapid training method based on template editing according to an embodiment of the invention;
FIG. 2 schematically illustrates a block diagram of a remote sensing base model rapid training system based on template editing according to an embodiment of the invention.
Detailed Description
The present invention will be further described in detail below with reference to specific embodiments and with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; may be mechanically connected, may be electrically connected or may communicate with each other; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In the description of the present invention, it should be understood that the terms "longitudinal," "length," "circumferential," "front," "rear," "left," "right," "top," "bottom," "inner," "outer," and the like indicate an orientation or a positional relationship based on that shown in the drawings, merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the subsystem or element in question must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention.
Like elements are denoted by like or similar reference numerals throughout the drawings. Conventional structures or constructions will be omitted when they may cause confusion in the understanding of the invention. And the shape, size and position relation of each component in the figure do not reflect the actual size, proportion and actual position relation.
Similarly, in the description of exemplary embodiments of the invention above, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. The description of the terms "one embodiment," "some embodiments," "example," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
In the technical scheme of the invention, the related data (such as including but not limited to personal information of a user) are collected, stored, used, processed, transmitted, provided, disclosed, applied and the like, all meet the requirements of related laws and regulations, necessary security measures are adopted, and the public welcome is not violated.
FIG. 1 schematically illustrates a flow chart of a remote sensing base model rapid training method based on template editing according to an embodiment of the invention.
As shown in FIG. 1, the remote sensing basic model rapid training method based on template editing in the embodiment includes operations S1 to S5.
In operation S1, a pre-labeled remote sensing sample dataset is obtained.
In this embodiment, the remote sensing sample dataset may be, for example, one of a remote sensing image dataset, a remote sensing text dataset, a remote sensing voice dataset, a remote sensing video dataset, a remote sensing electronic dataset, and a remote sensing situation dataset.
In operation S2, a remote sensing infrastructure corresponding to the task demand is determined according to the task demand of the user.
In this embodiment, an appropriate infrastructure is selected according to task requirements, and may be, for example, an infrastructure such as a Convolutional Neural Network (CNN) or a depth residual network (res net).
In operation S3, the remote sensing infrastructure model architecture is edited with a corresponding algorithm framework and mirroring environment based on the task requirements.
In this embodiment, the template structure of the remote sensing basic model is defined and edited according to the selected remote sensing basic model frame. And selecting an algorithm framework and a version according to task requirements, adding a mirror image environment required by algorithm training, inputting a training command, uploading an algorithm code file, and completing template integration after adjusting resource information.
In operation S4, the remote sensing sample data set is input into the remote sensing infrastructure model architecture, and the remote sensing infrastructure model architecture is trained by using the edited algorithm frame and the mirror image environment.
According to an embodiment of the present disclosure, before operation S4, operations S400 to S401 are further included:
in operation S400, parameter initialization setting is performed on the remote sensing infrastructure model architecture.
In this embodiment, default parameters may be set for the initial model using template editing, or the model may be initialized by uploading the pre-training model to shorten the training time.
In operation S401, a training task is created from the remote sensing sample dataset and the algorithmic framework and the mirrored environment.
In this embodiment, stand-alone or distributed training tasks are created using the prepared remote sensing sample data set and the algorithmic framework and mirrored environment.
According to an embodiment of the present disclosure, operation S4 may further include operation S402 to operation S403:
in operation S402, a remote sensing sample dataset is input into a remote sensing infrastructure model architecture for training according to a training task.
In this embodiment, according to the created stand-alone or distributed training task, the remote sensing sample dataset is input into the remote sensing basic model architecture for training, and the proper training batch size and training round number are set.
In operation S403, parameters of the algorithm framework are iteratively updated using the intelligent parameter tuning optimization method.
According to an embodiment of the present disclosure, operation S403 may further include the following operations:
and determining the super-parameters of the algorithm framework, wherein the super-parameters comprise learning rate, training round number and the like.
Selecting a proper intelligent parameter-adjusting optimization algorithm, such as grid Search, random Search (Random Search), bayesian optimization Search (Bayesian Optimization Search), genetic algorithm and the like, wherein different algorithms have different Search strategies and optimization performances, and selecting a proper algorithm according to specific situations to iteratively optimize super parameters according to the intelligent parameter-adjusting optimization algorithm.
And acquiring an evaluation index of the remote sensing basic model architecture on the verification set.
And taking the part of the hyper-parameters reaching the index threshold value in the iterative optimization process as the starting point of the next iteration until reaching the iteration termination condition according to the evaluation index, namely adjusting the hyper-parameters in the training process according to the performance of the model on the verification set so as to further improve the performance of the model.
In operation S5, the progress, parameter variation and performance index of the remote sensing infrastructure in the training process are visually monitored.
According to an embodiment of the present disclosure, operation S5 may further include operations S500 to S502:
in operation S500, a visualization dynamic library matching the remote sensing infrastructure model architecture is determined.
In this embodiment, a suitable visualization dynamic library, such as TensorBoard, matplotlib, is selected, which provides visualization tools and functions that can be used to monitor and display the progress, parameter changes, and performance metrics during the training process in real time.
In operation S501, parameters and loss functions of the remote sensing infrastructure model architecture in the training process are transferred to the visual dynamic library by means of dynamic mounting.
In operation S502, the progress, parameter variation and performance index in the training process are monitored in real time according to the visual dynamic library.
In the training process, functions and tools provided by the visual dynamic library are used for monitoring the progress, parameter change and performance index of training in real time, and for example, a loss function curve can be drawn, the training accuracy and the change of visual parameters can be displayed.
In another embodiment, operation S3 may further include the following operations:
a template is created, wherein the template is pre-integrated with at least one of a tensor flow framework (TensorFlow), a Pytorch, a flyby framework, MXNet, mindSpore, caffe, caffe, and a just-in-time compilation framework (jitter), and at least one of an x86_64 mirror architecture, a mips_64 mirror architecture, and an arm_64 mirror architecture.
And selecting an algorithm framework corresponding to the task requirement from the template and configuring the algorithm framework into the remote sensing basic model framework.
Because the tensor flow framework (TensorFlow), the Pytorch, the flyash framework, the MXNet, mindSpore, caffe, caffe, the just-in-time compiling framework (Jittor) and other main flow frameworks, the X86_64 mirror framework, the MIPS_64 mirror framework and the ARM_64 mirror framework are integrated on the template in advance, a user does not need to edit corresponding algorithms and environments in a time-consuming manner, but can directly select the corresponding algorithms and environments in the template according to task requirements, and great convenience is brought to the user when the algorithms are integrated.
FIG. 2 schematically illustrates a block diagram of a remote sensing base model rapid training system based on template editing according to an embodiment of the invention.
As shown in fig. 2, the remote sensing basic model rapid training system based on template editing of this embodiment includes: sample data management module 301, template management module 302, training management module 303.
The sample data management module 301 is configured to obtain a pre-labeled remote sensing sample data set. The sample data management module 301 has sample integration capability and labeling capability, wherein the remote sensing sample data set includes: one of a remote sensing image dataset, a remote sensing text dataset, a remote sensing voice dataset, a remote sensing video dataset, a remote sensing electronic dataset and a remote sensing situation dataset.
The template management module 302 is configured to determine a remote sensing basic model architecture corresponding to a task requirement according to the task requirement of a user; and editing the corresponding algorithm framework and mirror image environment for the remote sensing basic model framework based on the task requirement.
The training management module 303 is configured to input the remote sensing sample data set into a remote sensing infrastructure model architecture, and train the remote sensing infrastructure model architecture by using the edited algorithm frame and the mirror image environment; and visually monitoring the progress, parameter change and performance index of the remote sensing basic model framework in the training process.
In another embodiment, the remote sensing basic model rapid training system based on template editing further comprises: task scheduling module 304.
The task scheduling module 304 is configured to provide resource load balancing and task traceability.
In the training of the remote sensing base model, on the one hand, a large amount of computational resources are required to handle large-scale data sets and complex model structures. The goal of resource load balancing is to ensure that these resources can be fully utilized, avoiding excessive or insufficient use of resources, to increase the efficiency and speed of training. On the other hand, the training of the remote sensing basic model involves a plurality of tasks, such as sample data preprocessing, model training, verification, evaluation and the like. The task tracing aims at tracing and recording the starting time, the finishing time, the execution progress, the execution result and other information of each task. By task tracing, the execution condition of the task can be known, and problems in task execution, such as errors, anomalies or performance degradation, can be found and solved in time. Task traceability can also be used to analyze and optimize the training process, helping to improve the training effect and performance of the model.
Any of the sample data management module 301, the template management module 302, and the training management module 303 may be combined in one module to be implemented, or any of the modules may be split into a plurality of modules according to an embodiment of the present invention. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the sample data management module 301, the template management module 302, and the training management module 303 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware in any other reasonable manner of integrating or packaging the circuits, or in any one of or a suitable combination of any of the three. Alternatively, at least one of the sample data management module 301, the template management module 302, the training management module 303 may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.
It should be noted that, in the embodiment of the present invention, the remote sensing basic model rapid training system based on template editing corresponds to the remote sensing basic model rapid training method based on template editing, and specific implementation details and technical effects thereof are the same, which are not described herein.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the invention can be combined in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the present invention. In particular, the features recited in the various embodiments of the invention can be combined and/or combined in various ways without departing from the spirit and teachings of the invention. All such combinations and/or combinations fall within the scope of the invention.
The embodiments of the present invention are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the invention, and such alternatives and modifications are intended to fall within the scope of the invention.
Claims (10)
1. A remote sensing basic model rapid training method based on template editing is characterized by comprising the following steps:
acquiring a remote sensing sample data set marked in advance;
determining a remote sensing basic model architecture corresponding to task demands according to the task demands of a user;
editing a corresponding algorithm frame and a mirror image environment for the remote sensing basic model framework based on the task demand;
inputting the remote sensing sample data set into the remote sensing basic model framework, and training the remote sensing basic model framework by using an edited algorithm framework and a mirror image environment;
and visually monitoring the progress, parameter change and performance index of the remote sensing basic model framework in the training process.
2. The rapid training method of remote sensing basic model based on template editing according to claim 1, wherein before inputting the remote sensing sample dataset into the remote sensing basic model architecture and training the remote sensing basic model architecture by using the edited algorithm frame and mirror image environment, the method further comprises:
carrying out parameter initialization setting on the remote sensing basic model framework;
and creating a training task according to the remote sensing sample data set, the algorithm frame and the mirror image environment.
3. The rapid training method of remote sensing basic model based on template editing according to claim 2, wherein inputting the remote sensing sample dataset into the remote sensing basic model architecture, training the remote sensing basic model architecture by using an edited algorithm frame and a mirror image environment comprises:
inputting the remote sensing sample data set into the remote sensing basic model framework for training according to the training task;
and iteratively updating parameters of the algorithm framework by using an intelligent parameter adjustment optimization method.
4. The rapid training method of remote sensing basic model based on template editing according to claim 3, wherein the iterative updating of parameters of the algorithm framework using the intelligent parameter tuning optimization method comprises:
determining hyper-parameters of the algorithm framework;
iterative optimization of the super parameters is carried out according to an intelligent parameter adjustment optimization algorithm;
acquiring an evaluation index of the remote sensing basic model architecture on a verification set;
and taking the part of the super parameter reaching the index threshold value in the iterative optimization process as the starting point of the next iteration according to the evaluation index until reaching the iteration termination condition.
5. The rapid training method of remote sensing basic model based on template editing according to claim 1, wherein the visually monitoring the progress, parameter change and performance index of the remote sensing basic model architecture in the training process comprises:
determining a visual dynamic library matched with the remote sensing basic model architecture;
parameters and loss functions of the remote sensing basic model framework in the training process are transferred to the visual dynamic library in a dynamic mounting mode;
and monitoring the progress, parameter change and performance index in the training process in real time according to the visual dynamic library.
6. The rapid training method of remote sensing basic model based on template editing according to claim 1, wherein the compiling the corresponding algorithm frame and mirror image environment for the remote sensing basic model architecture based on the task requirements comprises:
creating a template, wherein the template is pre-integrated with at least one of a tensor flow framework, a Pytorch, a flyash framework, MXNet, mindSpore, caffe, caffe, and a just-in-time compilation framework;
and selecting an algorithm framework corresponding to the task demand from the template to be configured into the remote sensing basic model framework.
7. The template editing-based remote sensing base model rapid training method of claim 6, wherein the template is further integrated with at least one of an x86_64 mirror architecture, a mips_64 mirror architecture, and an arm_64 mirror architecture.
8. The rapid training method of remote sensing basic model based on template editing according to claim 2, wherein the creating training task according to the remote sensing sample data set and the algorithm frame and mirror image environment comprises:
and creating a stand-alone training task or a distributed training task according to the remote sensing sample data set and the algorithm frame and the mirror image environment.
9. The template-editing-based remote sensing basic model rapid training method according to claim 1, wherein the remote sensing sample data set comprises: one of a remote sensing image dataset, a remote sensing text dataset, a remote sensing voice dataset, a remote sensing video dataset, a remote sensing electronic dataset and a remote sensing situation dataset.
10. A remote sensing basic model rapid training system based on template editing comprises:
the sample data management module is used for acquiring a remote sensing sample data set marked in advance;
the template management module is used for determining a remote sensing basic model framework corresponding to the task demand according to the task demand of a user; editing a corresponding algorithm frame and a mirror image environment for the remote sensing basic model framework based on the task demand;
the training management module is used for inputting the remote sensing sample data set into the remote sensing basic model framework, and training the remote sensing basic model framework by utilizing the edited algorithm framework and the mirror image environment; and visually monitoring the progress, parameter change and performance index of the remote sensing basic model framework in the training process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311623353.2A CN117876840A (en) | 2023-11-30 | 2023-11-30 | Remote sensing basic model rapid training method and system based on template editing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311623353.2A CN117876840A (en) | 2023-11-30 | 2023-11-30 | Remote sensing basic model rapid training method and system based on template editing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117876840A true CN117876840A (en) | 2024-04-12 |
Family
ID=90578021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311623353.2A Pending CN117876840A (en) | 2023-11-30 | 2023-11-30 | Remote sensing basic model rapid training method and system based on template editing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117876840A (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7356559B1 (en) * | 1999-07-01 | 2008-04-08 | Affinity Internet, Inc. | Integrated platform for developing and maintaining a distributed multiapplication online presence |
| CN103019671A (en) * | 2012-10-08 | 2013-04-03 | 中国科学院对地观测与数字地球科学中心 | Data-intensive remote sensing image processing-oriented generic programmed framework programming method |
| CN107665111A (en) * | 2017-09-05 | 2018-02-06 | 北京空间飞行器总体设计部 | A kind of remote sensing satellite load task parametric control method and system |
| CN109189380A (en) * | 2018-09-12 | 2019-01-11 | 湖南创智艾泰克科技有限公司 | Algorithm integration frame and method |
| CN109857462A (en) * | 2019-01-25 | 2019-06-07 | 东莞理工学院 | The backstage Docker duty mapping method of remote sensing images visual editor |
| US20200242400A1 (en) * | 2019-01-25 | 2020-07-30 | Oath Inc. | Systems and methods for hyper parameter optimization for improved machine learning ensembles |
| CN111754073A (en) * | 2020-05-19 | 2020-10-09 | 北京吉威空间信息股份有限公司 | Centralized processing and distributed operation framework construction method for spatial data service |
| CN111882059A (en) * | 2020-07-17 | 2020-11-03 | 北京市商汤科技开发有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113158807A (en) * | 2021-03-24 | 2021-07-23 | 中科北纬(北京)科技有限公司 | Model self-training and optimizing system for remote sensing image |
| CN113886026A (en) * | 2021-12-07 | 2022-01-04 | 中国电子科技集团公司第二十八研究所 | Intelligent modeling method and system based on dynamic parameter configuration and process supervision |
| CN114064025A (en) * | 2021-11-10 | 2022-02-18 | 北京百度网讯科技有限公司 | Generation method, device, equipment and storage medium of training pipeline model |
| CN115202729A (en) * | 2022-07-30 | 2022-10-18 | 重庆长安汽车股份有限公司 | Container service-based mirror image generation method, device, equipment and medium |
| CN116107730A (en) * | 2021-11-09 | 2023-05-12 | 中国移动通信有限公司研究院 | Flow control method, device and equipment |
| CN116594608A (en) * | 2023-04-04 | 2023-08-15 | 中国科学院自动化研究所 | Method and device for generating and training visual neural network model |
-
2023
- 2023-11-30 CN CN202311623353.2A patent/CN117876840A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7356559B1 (en) * | 1999-07-01 | 2008-04-08 | Affinity Internet, Inc. | Integrated platform for developing and maintaining a distributed multiapplication online presence |
| CN103019671A (en) * | 2012-10-08 | 2013-04-03 | 中国科学院对地观测与数字地球科学中心 | Data-intensive remote sensing image processing-oriented generic programmed framework programming method |
| CN107665111A (en) * | 2017-09-05 | 2018-02-06 | 北京空间飞行器总体设计部 | A kind of remote sensing satellite load task parametric control method and system |
| CN109189380A (en) * | 2018-09-12 | 2019-01-11 | 湖南创智艾泰克科技有限公司 | Algorithm integration frame and method |
| CN109857462A (en) * | 2019-01-25 | 2019-06-07 | 东莞理工学院 | The backstage Docker duty mapping method of remote sensing images visual editor |
| US20200242400A1 (en) * | 2019-01-25 | 2020-07-30 | Oath Inc. | Systems and methods for hyper parameter optimization for improved machine learning ensembles |
| CN111754073A (en) * | 2020-05-19 | 2020-10-09 | 北京吉威空间信息股份有限公司 | Centralized processing and distributed operation framework construction method for spatial data service |
| CN111882059A (en) * | 2020-07-17 | 2020-11-03 | 北京市商汤科技开发有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113158807A (en) * | 2021-03-24 | 2021-07-23 | 中科北纬(北京)科技有限公司 | Model self-training and optimizing system for remote sensing image |
| CN116107730A (en) * | 2021-11-09 | 2023-05-12 | 中国移动通信有限公司研究院 | Flow control method, device and equipment |
| CN114064025A (en) * | 2021-11-10 | 2022-02-18 | 北京百度网讯科技有限公司 | Generation method, device, equipment and storage medium of training pipeline model |
| CN113886026A (en) * | 2021-12-07 | 2022-01-04 | 中国电子科技集团公司第二十八研究所 | Intelligent modeling method and system based on dynamic parameter configuration and process supervision |
| CN115202729A (en) * | 2022-07-30 | 2022-10-18 | 重庆长安汽车股份有限公司 | Container service-based mirror image generation method, device, equipment and medium |
| CN116594608A (en) * | 2023-04-04 | 2023-08-15 | 中国科学院自动化研究所 | Method and device for generating and training visual neural network model |
Non-Patent Citations (1)
| Title |
|---|
| 曲海成;张钧萍;陈雨时;: "多源遥感资源协同调度系统架构技术研究", 沈阳师范大学学报(自然科学版), no. 01, 15 January 2013 (2013-01-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102257939B1 (en) | System for automated generative design synthesis using data from design tools and knowledge from digital twin graphs | |
| US8983936B2 (en) | Incremental visualization for structured data in an enterprise-level data store | |
| EP4120199A1 (en) | Image rendering method and apparatus, and electronic device and storage medium | |
| CN108229085A (en) | computer simulation analysis system and method | |
| CN110114771A (en) | The system and method that lattice structure for increasing material manufacturing designs | |
| CN107292042A (en) | A kind of BIM modeling methods | |
| US10430529B1 (en) | Directed design updates in engineering methods for systems | |
| US20220026191A1 (en) | Methods and systems for an in-line automated inspection of a mechanical part | |
| CN111078094A (en) | Distributed machine learning visualization device | |
| WO2022030041A1 (en) | Prediction system, information processing device, and information processing program | |
| EP4089493A1 (en) | Development of a product using a process control plan digital twin | |
| CN115657467A (en) | Wind power plant wake flow control optimization method, device, equipment and storage medium | |
| US20070046695A1 (en) | System and method for computer aided design | |
| US20230252204A1 (en) | Method for generating a virtual model of a component, computer system, and computer program product | |
| US20220027739A1 (en) | Search space exploration for deep learning | |
| US10331808B1 (en) | Feature recognition in engineering methods and systems | |
| JP2007317096A (en) | Verification scenario producing program, recording medium, verification scenario producing system, and verification scenario producing method | |
| CN117876840A (en) | Remote sensing basic model rapid training method and system based on template editing | |
| US11250058B2 (en) | Providing an easily navigable visual representation of a graph | |
| CN116484545A (en) | Method for establishing data twin body based on photovoltaic flexible support | |
| Piotrowski et al. | Enhanced interactive parallel coordinates using machine learning and uncertainty propagation for engineering design | |
| US20210240873A1 (en) | Cad systems using rule-driven product and manufacturing information | |
| Bailey et al. | FIPER: an intelligent system for the optimal design of highly engineered products | |
| KR102364390B1 (en) | Apparatus and method for auto-processing modeling using matlab | |
| CN113641396B (en) | Method and device for modifying parameterized models in batches |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |