CN117216507A - Deep neural network model mobility measurement method based on geographic partition - Google Patents
Deep neural network model mobility measurement method based on geographic partition Download PDFInfo
- Publication number
- CN117216507A CN117216507A CN202311304916.1A CN202311304916A CN117216507A CN 117216507 A CN117216507 A CN 117216507A CN 202311304916 A CN202311304916 A CN 202311304916A CN 117216507 A CN117216507 A CN 117216507A
- Authority
- CN
- China
- Prior art keywords
- domain
- cross
- evaluation index
- source
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003062 neural network model Methods 0.000 title claims abstract description 60
- 238000005192 partition Methods 0.000 title claims abstract description 16
- 238000000691 measurement method Methods 0.000 title claims abstract description 12
- 238000012360 testing method Methods 0.000 claims abstract description 83
- 238000012549 training Methods 0.000 claims abstract description 35
- 238000011156 evaluation Methods 0.000 claims description 113
- 238000000034 method Methods 0.000 claims description 29
- 238000013508 migration Methods 0.000 claims description 29
- 230000005012 migration Effects 0.000 claims description 28
- 238000012795 verification Methods 0.000 claims description 16
- 230000001419 dependent effect Effects 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000003993 interaction Effects 0.000 claims description 7
- 238000000638 solvent extraction Methods 0.000 claims description 5
- 101001095088 Homo sapiens Melanoma antigen preferentially expressed in tumors Proteins 0.000 claims description 3
- 102100037020 Melanoma antigen preferentially expressed in tumors Human genes 0.000 claims description 3
- 230000000694 effects Effects 0.000 description 8
- 238000013136 deep learning model Methods 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 230000004927 fusion Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011158 quantitative evaluation Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a depth neural network model mobility measurement method based on geographic partition, which comprises the following steps: obtaining geospatial data and an initial neural network model, and extracting source domain data and target domain data from the geospatial data according to the division of the geospatial data; extracting a source domain training set from the source domain data, training to obtain a source domain model, extracting a target domain training set from the target domain data, and training to obtain a target domain model; verifying the source domain model to obtain source domain test data, verifying the source domain model to obtain cross-domain test data, and verifying the target domain model to obtain target domain test data; calculating the cross-domain stability and/or the cross-domain fitness of the source domain model; judging whether the source domain model is a neural network model capable of being used in a cross-domain mode according to the cross-domain stability and/or the cross-domain adaptability of the source domain model, and displaying a judging result.
Description
Technical Field
The application relates to the technical field of deep neural network models, in particular to a geographic partition-based deep neural network model mobility measurement method.
Background
Cross-domain model sharing is carried out among cities, so that the time and cost of model training are greatly reduced, and the application range of the model is improved. Successful application of deep learning models relies on a large amount of useful tag data. However, many geographic areas have small amounts of data and high quality tagged data is always in demand. Deep neural network models often do not generalize well into new geographic areas because limited geospatial annotation data creates an "overfitting" problem. While depending on the deep learning model generated for a particular geographic region, the recognition or prediction effect varies from geographic location to geographic location as the migration proceeds.
The traditional deep learning model only uses a training set under the same geographic area to carry out model training, and the performance of the model is checked through a verification set to carry out parameter adjustment and optimization. After the model is completed, the model is evaluated by using a local test set, and the measurement of the cross-domain application effect of the model is not realized.
Therefore, the lack of basis for the person skilled in the art when selecting the neural network model for cross-domain application often causes that the cross-domain application effect of the selected neural network model is not ideal.
Disclosure of Invention
The application aims to provide a depth neural network model mobility measurement method based on geographical partitions, and aims to solve the problem that the application effect of the prior art is not ideal when the geographical migration of the neural network model is carried out.
The technical scheme adopted for solving the technical problems is as follows:
the application provides a depth neural network model mobility measurement method based on geographic partition, which comprises the following steps:
obtaining geospatial data and an initial neural network model, and extracting source domain data and target domain data from the geospatial data according to the division of the geospatial data;
extracting a source domain training set from the source domain data, training a test neural network model according to the source domain training set to obtain a source domain model, extracting a target domain training set from the target domain data, and training the test neural network model according to the target domain training set to obtain a target domain model;
extracting a source domain verification set from the source domain data, verifying the source domain model to obtain source domain test data, extracting a target domain verification set and a cross-domain test set from the target domain data, verifying the source domain model according to the cross-domain test set to obtain cross-domain test data, and verifying the target domain model according to the target domain verification set to obtain target domain test data;
calculating the cross-domain stability of the source domain model according to the cross-domain test data and the source domain test data, and calculating the cross-domain fitness of the source domain model according to the cross-domain test data and the target domain test data;
judging whether the source domain model is a neural network model capable of being used in a cross-domain mode according to the cross-domain stability and/or the cross-domain adaptability of the source domain model, and displaying a judging result.
Further, calculating the domain-crossing stability of the source domain model according to the domain-crossing test data and the source domain test data, and calculating the domain-crossing adaptability of the source domain model according to the domain-crossing test data and the target domain test data specifically includes:
calculating a cross-domain basic evaluation index according to the cross-domain test data, calculating a source domain basic evaluation index according to the source domain test data, and calculating a target domain basic evaluation index according to the target domain test data;
calculating the cross-domain stability of the source domain model according to the cross-domain basic evaluation index and the source domain basic evaluation index, and calculating the cross-domain fitness of the source domain model according to the cross-domain basic evaluation index and the target domain basic evaluation index.
Further, calculating the cross-domain stability of the source domain model according to the source domain basic evaluation index and the cross-domain basic evaluation index, specifically subtracting the difference value of the source domain basic evaluation index from the cross-domain basic evaluation index to serve as the cross-domain fitness of the source domain model;
and calculating the cross-domain fitness of the source domain model according to the cross-domain basic evaluation index and the target domain basic evaluation index, specifically subtracting the difference value of the target domain basic evaluation index from the cross-domain basic evaluation index to serve as the cross-domain fitness of the source domain model.
Further, the base evaluation metrics each include one or more of accuracy, precision, recall, interaction ratio, kappa coefficient, mean absolute error, mean square error, root mean square error, mean absolute percentage error, and decision coefficient.
Further, the Accuracy Accuracy is calculated in the following manner:
the calculation mode of the Precision is as follows:
the Recall rate Recall is calculated in the following manner:
the interaction ratio IoU is calculated by:
the Kappa coefficient is calculated by:
the accuracy rate Po is specifically:
the occasional consistency error Pe is specifically:
the number of real cases of the test data corresponding to the TP-based evaluation index, the number of real negative cases of the test data corresponding to the TN-based evaluation index, the number of false positive cases of the test data corresponding to the FP-based evaluation index and the number of false negative cases of the test data corresponding to the FN-based evaluation index are all determined; po is the accuracy and Pe is the occasional consistency error.
The mean absolute error MAE is calculated by:
the mean square error MSE is calculated as:
the root mean square error RMSE is calculated as:
the mean absolute percentage error MAPE is calculated by:
determining the coefficient R 2 The calculation mode of (a) is as follows:
where N represents the number of samples, i represents the ith dependent variable, y i Represents the i-th dependent variable, and,representing the regression value of the dependent variable, i.e. the predicted value, < ->Represents the regression value of the ith dependent variable, +.>Representing the mean of the dependent variable.
Further, the judging whether the source domain model is a neural network model capable of being used in a cross-domain mode according to the cross-domain fitness and/or the cross-domain stability of the source domain model specifically comprises:
judging the excellent degree of the geographic migration of the source domain model according to the cross-domain fitness and/or the cross-domain stability;
judging whether the source domain model is a neural network model which can be used in a cross-domain mode according to the excellent degree of the geographic mobility of the source domain model.
Further, the determining the excellent degree of the geographic mobility of the source domain model according to the cross-domain fitness and/or the cross-domain stability specifically includes:
if the target domain sample data are enough to train a target domain model, judging the excellent degree of the geographic migration of the source domain model according to the cross-domain fitness;
and if the target domain sample data is insufficient to train the target domain model, judging the excellent degree of the geographic migration of the source domain model according to the cross-domain stability.
Further, the judging the excellent degree of the geographic mobility of the source domain model according to the cross-domain fitness and/or the cross-domain stability specifically comprises the following steps:
if the cross-domain fitness and/or the cross-domain stability are positive, judging that the excellent degree of the geographic mobility of the source domain model is excellent;
if the cross-domain fitness and/or the cross-domain stability are negative values with absolute values of 0-0.1, judging that the excellent degree of the geographic migration of the source domain model is good;
if the cross-domain fitness and/or the cross-domain stability are negative values with absolute values of 0.2-0.3, judging that the excellent degree of the geographic migration of the source domain model is medium;
and if the cross-domain fitness and/or the cross-domain stability are negative values with absolute values larger than 0.3, judging that the excellent degree of the geographic migration of the source domain model is poor.
The technical scheme adopted by the application has the following effects:
according to the method, a geographical partition is used as a source domain and a target domain, a source domain model is migrated to the target domain in a cross-geographical way, the effect of the source domain model before and after migration is tested, tested data are obtained as a basis, the cross-domain fitness and/or the cross-domain stability of the source domain model are evaluated, the geographical migration of the source domain model is evaluated, whether the source domain model is suitable for being migrated to the target domain is judged according to the geographical migration, so that a tester can select a proper source domain model to perform geographical migration application, cross-domain model sharing among different geographical areas is facilitated, the training cost of the model is reduced, and data fusion and knowledge flow which are 'available and invisible' are realized.
Drawings
FIG. 1 is a flowchart illustrating the steps of a method for measuring the mobility of a deep neural network model based on geographical partition in a preferred embodiment of the present application;
FIG. 2 is a schematic diagram of a model and data structure of a deep neural network model migration metric method based on geographical partitions in a preferred embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clear and clear, the present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
Example 1
Aiming at the problem that no clear method for measuring the geographic mobility of the deep neural network model exists at present, the application aims to provide a geographic partition-based method for measuring the geographic mobility of the deep neural network model. Firstly, geographic space partitioning is carried out, and a depth neural network model mobility measurement method based on geographic partitioning is invented for a geographic depth neural network model constructed by utilizing multi-source city big data such as remote sensing images, city road networks, interest point data, taxi tracks and the like so as to measure the geographic mobility of the depth neural network. The measurement method can realize quantitative evaluation of the migration capability of the deep neural network across the geographic region, is suitable for any data set with spatial attribute and different types of deep neural network model structures, and has certain comparability and expandability. This provides objective evaluation criteria and a general metric for the geomigration study of the subsequent deep learning model.
Specifically, referring to fig. 1 and 2, a first embodiment of the present application provides a method for testing mobility of a deep neural network model, which includes the steps of:
s1, acquiring geospatial data and an initial neural network model, and extracting source domain data and target domain data from the geospatial data according to the division of the geospatial data.
The geographic space is divided into a source domain and a target domain. The source domain refers to the geographic area in which the data set used in training the source domain model is located, and the target domain refers to the geographic area to which the trained source domain model is applied. Geographic region partitioning is an important step in verifying whether a trained source domain model can be effectively generalized in different geographic environments after migration. As shown in fig. 1, one geographical area is selected as a Source Domain (S), and geospatial data within the area is referred to as Source Domain data, and another geographical area is selected as a Target Domain (T), and geospatial data within the area is referred to as Target Domain data. In geographic region division, the geographic space may be divided into regions based on a variety of methods, such as methods based on administrative boundaries, based on geographic locations, based on spatial clustering, and the like. The selection of the source domain and the target domain may take into account the geographic characteristics and data distribution between the different domains.
And selects an appropriate deep neural network and deep learning framework to construct a deep neural network model as the initial neural network model in this embodiment.
In this embodiment, the geographical space data may be partitioned according to administrative division or may be partitioned according to grid, which is not limited to a specific partitioning method.
S2, extracting a source domain training set from the source domain data, training the test neural network model according to the source domain training set to obtain a source domain model, extracting a target domain training set from the target domain data, and training the test neural network model according to the target domain training set to obtain a target domain model.
Specifically, in each geographic region, the embodiment prepares a geospatial data set according to an application scene and requirements, and divides source domain data and target domain data into a training set and a verification set respectively. The training set is used to train the model, and the validation set and the test set are used for performance evaluation of the model.
And respectively training a source domain model and a target domain model in the source domain and the target domain according to the acquired training set.
S3, extracting a source domain verification set from the source domain data, verifying the source domain model to obtain source domain test data, extracting a target domain verification set and a cross-domain test set from the target domain data, verifying the source domain model according to the cross-domain test set to obtain cross-domain test data, and verifying the target domain model according to the target domain verification set to obtain target domain test data.
In this embodiment, the target domain test data and the cross-domain test data each include a true case (True Positive Sample), a true negative case (True Negative Sample), a false positive case (False Positive Sample), and a false negative case (False Negative Sample).
In this embodiment, a source domain verification set is also extracted from the source domain data, and the source domain model is verified by the source domain verification set to obtain source domain test data. In this embodiment, the cross-domain test set specifically includes all sample data of the target domain training set and the target domain verification set.
In the real world, the tag data in a part of the target domain may not be enough to train out a target domain model, so in this embodiment, if the tag data in the target domain is not enough to train out a target domain model, the source domain model is tested by using the verification set of the source domain to obtain source domain test data, so that a basis is provided for further calculating the cross-domain stability to measure the geographic mobility of the model.
S4, calculating the cross-domain stability of the source domain model according to the cross-domain test data and the source domain test data, and calculating the cross-domain fitness of the source domain model according to the cross-domain test data and the target domain test data.
Specifically, a cross-domain basic evaluation index is calculated according to the cross-domain test data, a target domain basic evaluation index is calculated according to the target domain test data, and then the cross-domain fitness of a source domain model is calculated according to the cross-domain basic evaluation index and the target domain basic evaluation index.
According to the application, the performance evaluation Index of the model is associated with a deep neural network model trained in a specific geographic region, and the source domain model and the target domain model calculate basic evaluation indexes in the model performance evaluation system on a test set and a verification set of the target domain respectively to obtain a cross-domain basic evaluation Index Cmodel And target domain base evaluation Index Tmodel 。
Specifically, in this embodiment, five basic evaluation indexes including an Accuracy (Acc), a Precision (Pre), a Recall (Recall), an interaction ratio (Intersection over Union, ioU) and a Kappa coefficient are selected for an application scenario of a classification problem, and a performance evaluation system of a deep neural network model is constructed by the five indexes. And constructing a Confusion Matrix (fusion Matrix) by using the recognition result of the deep neural network model, and calculating the index. The confusion matrix is used for comparing the actual category with the predicted category in a crossing way and is used for counting the correctness, the error and the type of the classification result.
Specifically, assuming that the number of real cases in the confusion matrix is TP, the number of real cases is TN, the number of false positive cases is FP, and the number of false negative cases is FN.
In this embodiment, the Accuracy is calculated by:
the calculation mode of the Precision is as follows:
the Recall rate Recall is calculated in the following manner:
the interaction ratio IoU is calculated by:
the Kappa coefficient is calculated by:
wherein Kappa coefficient is an index for measuring consistency in statistics, the value of the Kappa coefficient is between-1 and 1, namely [ -1,1], the Kappa coefficient is calculated based on a confusion matrix, and is usually larger than 0, and for classification problems, consistency is whether a model prediction result and an actual classification result are consistent or not. Po is accuracy and Pe is occasional consistency error. Po is the accuracy of the prediction, which is the sum of the number of correctly classified samples of each class divided by the total number of samples, i.e. the overall classification accuracy, and is understood as the consistency of the prediction.
Pe denotes the occasional consistency error, i.e. the sum of the "product of actual and predicted number" for all categories, respectively, divided by the "square of the total number of samples". Pe is the expected value of the consistency in the random case.
Specifically, the overall accuracy Po is calculated by:
the accidental consistency error Pe is calculated by:
it should be noted that, in other embodiments, the basic evaluation index is not limited to the listed 5 indexes, and other indexes that can be used to evaluate the deep learning model, such as mean square error, mean absolute error, root mean square error, mean absolute percentage error, decision coefficient, etc., may also be used as the evaluation index.
The mean absolute error MAE is calculated by:
the mean square error MSE is calculated as:
the root mean square error RMSE is calculated as:
the mean absolute percentage error MAPE is calculated by:
determining the coefficient R 2 The calculation mode of (a) is as follows:
where N represents the number of samples, i represents the ith dependent variable, y i Represents the i-th dependent variable, and,representing the regression value of the dependent variable, i.e. the predicted value, < ->Represents the regression value of the ith dependent variable, +.>Representing the mean of the dependent variable.
In practical use, when the geographic mobility of the model is measured, a proper basic evaluation index can be selected in a performance evaluation index system of the deep learning model according to the characteristics of an application scene and a data set, then the entropy weight method is used for carrying out weight distribution on a plurality of basic evaluation indexes, and finally all index values are weighted and summed to calculate a comprehensive evaluation index.
In this embodiment, an entropy weight method is used to perform weight distribution on five basic evaluation index values of Accuracy (Acc), precision (Pre), recall (Recall), interaction ratio (Intersection over Union, ioU) and Kappa coefficient, and then each index value is weighted and summed to obtain a comprehensive value;
specifically, the value of each index is normalized and scaled to the same scale range for comparison. The normalization can be performed by adopting methods such as linear transformation, minimum-maximum normalization and the like;
for each index, its entropy value is calculated. Entropy is an indicator for measuring the dispersion or uncertainty of information. In multi-criterion decision, entropy values are generally used to calculate the importance of the respective index, with higher entropy values indicating greater variability of the index values, and weights are determined based on the distribution of the index values.
Specifically, the normalized value of each index is divided into a plurality of intervals, typically equally distributed within the range of [0,1] or [ -1,1 ]. For each index, the frequency of occurrence of its normalized value in each interval is calculated, the frequency representing how many data points are within the interval. The frequency number within each interval is divided by the total number of samples to calculate the probability for each interval, which probability values represent the distribution of data points within each interval. For each index, the information entropy of each interval is calculated using the following formula:
information entropy= - Σ (probability log) 2 Probability (S)
Where Σ represents the sum over all intervals and the probability represents the probability for each interval.
And carrying out entropy weighted summation on the information of each section of each index to obtain the entropy value of the index. In general, the weight is a ratio of the number of samples (frequency) per section to the total number of samples. Normalizing the calculated entropy values to ensure that they range between [0,1 ];
the weight of each index is calculated using the entropy value. Generally, the weight is inversely related to the entropy value, i.e. the higher the entropy value, the lower its weight. Weights were calculated using the following formula:
where Σ represents the sum of all indices.
Normalizing the calculated weights to ensure that the sum of the weights is equal to 1, using the sum as a weight coefficient, and carrying out weighted summation on all index values;
multiplying each index value by a corresponding weight, and then adding the results to obtain a comprehensive evaluation index value. The comprehensive evaluation index is used as the final evaluation result of the test data on the model.
The source domain comprehensive evaluation index S of the source domain model is obtained Index Cross-domain comprehensive evaluation index C of source domain model Index And a target domain comprehensive evaluation index T of a target domain model Index Then, the index S can be comprehensively evaluated according to the source domain of the source domain model Index Cross-domain comprehensive evaluation index C of source domain model Index And a target domain comprehensive evaluation index T of a target domain model Index The cross-domain fitness and/or the cross-domain stability of the geographic mobility of the neural network is calculated.
In this embodiment, the Cross-Domain Stability (CDS) and the Cross-Domain fitness (CDA) of the main calculation model are used as the test results of the geographic mobility.
Wherein the source domain model is migrated across the geographic region to synthesize an evaluation index value (C Index ) The integrated evaluation index value (T Index ) The Cross-domain fitness (CDA) of the model is obtained. That is, the calculation formula of the cross-domain fitness is as follows:
CDA＝C Index -T Index ；
optionally, in the case that the tag data in a part of the target domains may not be enough to train a target domain model, the present application further calculates a source domain integrated evaluation index according to the source domain test data, and calculates a cross-domain stability of the source domain model according to the source domain integrated evaluation index and the cross-domain integrated evaluation index.
Specifically, the source domain model migrates across geographic domains with a comprehensive evaluation Index value (Index) Cmodel ) Correspondingly subtractComprehensive evaluation Index value (Index) of source domain model Smodel ) And the cross-domain stability CDS of the model is obtained. That is, the calculation formula of the cross-domain stability is as follows:
CDS＝C Index -S Index ；
when the source domain model is applied to a target domain different from the training environment, performance of the model changes, and differences exist between the cross-domain evaluation index of the source domain model, the source domain evaluation index of the source domain model and the target domain evaluation index of the target domain model. Performance-preserving stability indicates that the source domain model can exploit the geographic features of the source domain to assist in understanding the data in the new domain, thereby better migrating to the target domain. The significant decrease in performance indicates that no effective geomigration can be performed between the two domains, resulting in a negative migration phenomenon of the source domain model.
In the application, two indexes of the cross-domain stability and the cross-domain adaptability are used for evaluating the geographic mobility of the source domain model, and the difference between the cross-domain stability and the cross-domain adaptability is that the evaluation angles of the geographic mobility of the models are different. After the source domain model is migrated to the target domain, the evaluation index of the model is different from the value in the source domain, and the geographical migration of the model is concerned, namely whether the model can keep relatively stable performance under different geographical environments. The cross-domain fitness is calculated by the numerical difference between the source domain model and the target domain model evaluation index which are migrated from the cross-geographic domain, so that the actual application effect of the model in the target domain is emphasized more. The former focuses on the geographic mobility of the model itself, and the latter focuses more on the performance of the model in practical applications. The two metrics complement each other, helping to more fully measure the geographic mobility of the deep neural network.
Quantitative differences exist between the cross-domain stability and the cross-domain fitness of the different source domain-target domain pair models. The degree of geographic mobility of the deep neural network model in each source domain-target domain pair can be estimated through data classification. When the cross-domain stability and the cross-domain fitness of the model are positive values, the geographic migration level of the model is excellent; when the model is negative 0-0.1, the geographic migration grade of the model is good; at negative values of 0.2-0.3, the model has a moderate grade of geomigration; below a negative value of 0.3, the model has a poor grade of geomigration.
If the two conclusions are contradictory, in this embodiment, if the tag data in the target domain can train a target domain model, the cross-domain fitness is preferentially calculated to measure the geographic mobility of the model.
This is because the model may have a low degree of cross-domain stability (e.g., -0.2) after geo-migration, but a high degree of cross-domain adaptation (e.g., 0.1), which indicates that the source domain model shows a higher accuracy than the target domain model despite a large loss in accuracy after the source domain model is migrated, and that using the source domain model in the target domain is a more suitable choice than the target domain model. Therefore, when both metric values can be calculated in the present application, the cross-domain fitness is prioritized.
S5, judging whether the source domain model is a neural network model capable of being used in a cross-domain mode according to the cross-domain fitness and/or the cross-domain stability of the source domain model, and displaying a judging result.
Specifically, the model can be judged to be better or more by the cross-domain fitness and/or the cross-domain stability, namely, the source domain model with the cross-domain fitness and the cross-domain stability being positive values or negative values of 0-0.1 is used as the neural network model capable of being used in a cross-domain mode, so that testers can select and use the neural network model from various neural network models capable of being used in the cross-domain mode according to the actual requirement of a target domain.
The method can also test a plurality of neural network models which are selected and properly used as source domain models, and take the neural network model with the best geographic mobility as a neural network model which can be used in a cross-domain mode.
The application discloses a geographic partition-based deep neural network model mobility measurement method, which is used for dividing geographic space data into a source domain and a target domain. The source domain model is migrated to the target domain in a cross-geographic domain mode, and the geographic migration of the deep neural network model is measured by two measurement indexes of the cross-domain stability and the cross-domain adaptability. The method is beneficial to cross-domain model sharing among different geographic areas, so that the training cost of the model is reduced, and data fusion and knowledge flow of 'available invisible' are realized.
In summary, in the method, the geographic partition is a source domain and a target domain, the source domain model is migrated to the target domain in a cross-geographic manner, and then the effect of the source domain model before and after migration is tested, so that tested data is obtained as a basis, and further the cross-domain fitness and/or the cross-domain stability of the source domain model are evaluated to evaluate the geographic mobility of the source domain model, and whether the source domain model is suitable for being migrated to the target domain is judged according to the geographic mobility, so that testers can select the proper source domain model to perform geographic migration application, and cross-domain model sharing among different geographic regions is facilitated, so that the training cost of the model is reduced, and data fusion and knowledge flow of 'available invisible' are realized.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal comprising the element.
Of course, those skilled in the art will appreciate that implementing all or part of the above described embodiment methods may be accomplished by computer programs to instruct related hardware (e.g., processors, controllers, etc.).
It is to be understood that the application is not limited in its application to the examples described above, but is capable of modification and variation in light of the above teachings by those skilled in the art, and that all such modifications and variations are intended to be included within the scope of the appended claims.
Claims (9)
1. A deep neural network model mobility measurement method based on geographical partitioning, comprising:
obtaining geospatial data and an initial neural network model, and extracting source domain data and target domain data from the geospatial data according to the division of the geospatial data;
extracting a source domain training set from the source domain data, training a test neural network model according to the source domain training set to obtain a source domain model, extracting a target domain training set from the target domain data, and training the test neural network model according to the target domain training set to obtain a target domain model;
extracting a source domain verification set from the source domain data, verifying the source domain model to obtain source domain test data, extracting a target domain verification set and a cross-domain test set from the target domain data, verifying the source domain model according to the cross-domain test set to obtain cross-domain test data, and verifying the target domain model according to the target domain verification set to obtain target domain test data;
calculating the cross-domain stability of a source domain model according to the cross-domain test data and the source domain test data, and calculating the cross-domain fitness of the source domain model according to the cross-domain test data and the target domain test data;
judging whether the source domain model is a neural network model capable of being used in a cross-domain mode according to the cross-domain stability and/or the cross-domain adaptability of the source domain model, and displaying a judging result.
2. The geographical-partition-based deep neural network model mobility measurement method according to claim 1, wherein calculating the domain-crossing stability of the source domain model according to the domain-crossing test data and the source domain test data, and calculating the domain-crossing fitness of the source domain model according to the domain-crossing test data and the target domain test data specifically comprises:
calculating a cross-domain evaluation index according to the cross-domain test data, calculating a source domain evaluation index according to the source domain test data, and calculating a target domain evaluation index according to the target domain test data; and calculating the cross-domain stability of the source domain model according to the cross-domain evaluation index and the source domain evaluation index, and calculating the cross-domain fitness of the source domain model according to the cross-domain evaluation index and the target domain evaluation index.
3. The geographic partition-based deep neural network model mobility measurement method according to claim 2, wherein the calculating the domain-crossing stability of the source domain model according to the domain-crossing evaluation index and the source domain evaluation index is performed, specifically, subtracting the difference value of the source domain evaluation index from the domain-crossing evaluation index as the domain-crossing stability of the source domain model;
and calculating the cross-domain fitness of the source domain model according to the cross-domain evaluation index and the target domain evaluation index, specifically, subtracting the difference value of the target domain evaluation index from the cross-domain evaluation index to serve as the cross-domain fitness of the source domain model.
4. A geographical-partition-based deep neural network model mobility metric method as recited in claim 3, wherein said computing a cross-domain evaluation index from said cross-domain test data, computing a source-domain evaluation index from source-domain test data, and computing a target-domain evaluation index from said target-domain test data; according to the cross-domain evaluation index and the source domain evaluation index, the method specifically comprises the following steps:
calculating a plurality of cross-domain basic evaluation indexes according to the cross-domain test data, calculating a plurality of source domain basic evaluation indexes according to the source domain test data, and calculating a plurality of target domain basic evaluation indexes according to the target domain test data;
normalizing each of the cross-domain base assessment index, the source domain base assessment index and the target domain base assessment index;
calculating the information entropy of the corresponding cross-domain basic evaluation index according to each normalized cross-domain basic evaluation index, calculating the information entropy of the corresponding source domain basic evaluation index according to each normalized source domain basic evaluation index, and calculating the information entropy of the corresponding target domain basic evaluation index according to each normalized target domain basic evaluation index;
calculating the weight of each basic evaluation index according to the information entropy of each cross-domain basic evaluation index, the source domain basic evaluation index and the target domain basic evaluation index;
calculating a cross-domain comprehensive evaluation index according to each cross-domain basic evaluation index and the corresponding weight, calculating a source domain comprehensive evaluation index according to each source domain basic evaluation index and the corresponding weight, and calculating a target domain comprehensive evaluation index according to each target domain basic evaluation index and the corresponding weight; the difference value of the source domain evaluation index is subtracted from the cross domain evaluation index to serve as the cross domain stability of the source domain model, and particularly the difference value of the source domain comprehensive evaluation index is subtracted from the cross domain comprehensive evaluation index to serve as the cross domain stability of the source domain model;
the difference value of the target domain evaluation index subtracted from the cross domain evaluation index is used as the cross domain fitness of the source domain model, and particularly the difference value of the target domain comprehensive evaluation index subtracted from the cross domain comprehensive evaluation index is used as the cross domain fitness of the source domain model.
5. The geographical-partition-based deep neural network model mobility metric method of claim 4, wherein the base evaluation metrics each comprise one or more of accuracy, precision, recall, interaction ratio, kappa coefficient, mean absolute error, mean square error, root mean square error, mean absolute percentage error, and decision coefficient.
6. The method for testing the mobility of the deep neural network model according to claim 5, wherein the Accuracy is calculated by:
the calculation mode of the Precision is as follows:
the Recall rate Recall is calculated in the following manner:
the interaction ratio IoU is calculated by:
the Kappa coefficient is calculated by:
the accuracy rate Po is specifically:
the occasional consistency error Pe is specifically:
the number of real cases of the test data corresponding to the TP-based evaluation index, the number of real negative cases of the test data corresponding to the TN-based evaluation index, the number of false positive cases of the test data corresponding to the FP-based evaluation index and the number of false negative cases of the test data corresponding to the FN-based evaluation index are all determined; po is the accuracy and Pe is the occasional consistency error;
the mean absolute error MAE is calculated by:
the mean square error MSE is calculated as:
the root mean square error RMSE is calculated as:
the mean absolute percentage error MAPE is calculated by:
determining the coefficient R 2 The calculation mode of (a) is as follows:
where N represents the number of samples, i represents the ith dependent variable, y i Represents the i-th dependent variable, and,the regression value of the dependent variable is represented,represents the regression value of the ith dependent variable, +.>Representing the mean of the dependent variable.
7. The method for testing the mobility of the deep neural network model according to claim 2, wherein the determining whether the source domain model is a neural network model that can be used in a cross-domain manner according to the cross-domain fitness and/or the cross-domain stability of the source domain model specifically comprises:
judging the excellent degree of the geographic migration of the source domain model according to the cross-domain fitness and/or the cross-domain stability;
judging whether the source domain model is a neural network model which can be used in a cross-domain mode according to the excellent degree of the geographic mobility of the source domain model.
8. The method for testing the mobility of the deep neural network model according to claim 7, wherein the determining the excellent degree of the geographic mobility of the source domain model according to the cross-domain fitness and/or the cross-domain stability is specifically:
if the target domain sample data are enough to train a target domain model, judging the excellent degree of the geographic migration of the source domain model according to the cross-domain fitness;
and if the target domain sample data is insufficient to train the target domain model, judging the excellent degree of the geographic migration of the source domain model according to the cross-domain stability.
9. The method for testing the mobility of the deep neural network model according to claim 7, wherein the judging the excellent degree of the geographic mobility of the source domain model according to the cross-domain fitness and/or the cross-domain stability is specifically as follows:
if the cross-domain fitness and/or the cross-domain stability are positive, judging that the excellent degree of the geographic mobility of the source domain model is excellent;
if the cross-domain fitness and/or the cross-domain stability are negative values with absolute values of 0-0.1, judging that the excellent degree of the geographic migration of the source domain model is good;
if the cross-domain fitness and/or the cross-domain stability are negative values with absolute values of 0.2-0.3, judging that the excellent degree of the geographic migration of the source domain model is medium;
and if the cross-domain fitness and/or the cross-domain stability are negative values with absolute values larger than 0.3, judging that the excellent degree of the geographic migration of the source domain model is poor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311304916.1A CN117216507A (en) | 2023-10-08 | 2023-10-08 | Deep neural network model mobility measurement method based on geographic partition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311304916.1A CN117216507A (en) | 2023-10-08 | 2023-10-08 | Deep neural network model mobility measurement method based on geographic partition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117216507A true CN117216507A (en) | 2023-12-12 |
Family
ID=89046142
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311304916.1A Pending CN117216507A (en) | 2023-10-08 | 2023-10-08 | Deep neural network model mobility measurement method based on geographic partition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117216507A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160253597A1 (en) * | 2015-02-27 | 2016-09-01 | Xerox Corporation | Content-aware domain adaptation for cross-domain classification |
| CN110378366A (en) * | 2019-06-04 | 2019-10-25 | 广东工业大学 | A kind of cross-domain image classification method based on coupling knowledge migration |
| CN113239801A (en) * | 2021-05-13 | 2021-08-10 | 北京阿叟阿巴科技有限公司 | Cross-domain action recognition method based on multi-scale feature learning and multi-level domain alignment |
| US20220382984A1 (en) * | 2021-05-28 | 2022-12-01 | Paypal, Inc. | Cross-domain named entity recognition using knowledge graph |
-
2023
- 2023-10-08 CN CN202311304916.1A patent/CN117216507A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160253597A1 (en) * | 2015-02-27 | 2016-09-01 | Xerox Corporation | Content-aware domain adaptation for cross-domain classification |
| CN110378366A (en) * | 2019-06-04 | 2019-10-25 | 广东工业大学 | A kind of cross-domain image classification method based on coupling knowledge migration |
| CN113239801A (en) * | 2021-05-13 | 2021-08-10 | 北京阿叟阿巴科技有限公司 | Cross-domain action recognition method based on multi-scale feature learning and multi-level domain alignment |
| US20220382984A1 (en) * | 2021-05-28 | 2022-12-01 | Paypal, Inc. | Cross-domain named entity recognition using knowledge graph |
Non-Patent Citations (1)
| Title |
|---|
| XIANA CHEN.ET.: "Short video-driven deep perception for city imagery", 《ENVIRONMENT AND PLANNING B URBAN ANALYTICS AND CITY SCIENCE 》, vol. 51, no. 3, 7 August 2023 (2023-08-07), pages 689 - 704 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ives | Predicting the response of populations to environmental change | |
| CN109543906B (en) | Atmospheric visibility prediction method and equipment | |
| Allouche et al. | Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS) | |
| Broennimann et al. | Measuring ecological niche overlap from occurrence and spatial environmental data | |
| CN115270965B (en) | Power distribution network line fault prediction method and device | |
| CN105740991A (en) | Climate change prediction method and system for fitting various climate modes based on modified BP neural network | |
| US20200351678A1 (en) | Method for implementing antenna azimuth correction based on user data | |
| CN112735535A (en) | Prediction model training method, prediction model training device, data prediction method, data prediction device and storage medium | |
| Elbagalati et al. | Development of an enhanced decision-making tool for pavement management using a neural network pattern-recognition algorithm | |
| CN109409757A (en) | A kind of city degree Stress appraisal method based on NB Algorithm and curve modeling | |
| Beuchat et al. | A robust framework for probabilistic precipitations downscaling from an ensemble of climate predictions applied to Switzerland | |
| Mustafa et al. | Measuring the effect of stochastic perturbation component in cellular automata urban growth model | |
| CN111143981A (en) | Virtual test model verification system and method | |
| Ortega‐Huerta et al. | Validating distribution models for twelve endemic bird species of tropical dry forest in western Mexico | |
| CN109543922B (en) | Time-period borrowing and returning amount prediction method for single-vehicle station group shared by piles | |
| Chiang et al. | Can student test scores provide useful measures of school principals’ performance | |
| Patel et al. | Identification of best CMIP6 global climate model for rainfall by ensemble implementation of MCDM methods and statistical inference | |
| CN115222081A (en) | Academic resource prediction method and device and computer equipment | |
| Bannayan et al. | Predicting realizations of daily weather data for climate forecasts using the non‐parametric nearest‐neighbour re‐sampling technique | |
| CN117216507A (en) | Deep neural network model mobility measurement method based on geographic partition | |
| Mccabe et al. | Raising the bar for the next generation of biological atlases: using existing data to inform the design and implementation of atlas monitoring | |
| CN112528417B (en) | Assessment method for semi-physical simulation of aircraft | |
| CN113744888B (en) | Regional epidemic trend prediction and early warning method and system | |
| CN114821322A (en) | Small sample remote sensing image classification method and system based on attention mechanism | |
| Baramiya et al. | Predicting coastal profiles evolution from a diffusion model based on real data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |