CN117807434A - Communication data set processing method and device - Google Patents
Communication data set processing method and device Download PDFInfo
- Publication number
- CN117807434A CN117807434A CN202311662402.3A CN202311662402A CN117807434A CN 117807434 A CN117807434 A CN 117807434A CN 202311662402 A CN202311662402 A CN 202311662402A CN 117807434 A CN117807434 A CN 117807434A
- Authority
- CN
- China
- Prior art keywords
- data set
- data
- samples
- sample
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004891 communication Methods 0.000 title claims abstract description 73
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 239000013074 reference sample Substances 0.000 claims abstract description 68
- 239000000523 sample Substances 0.000 claims abstract description 58
- 238000000034 method Methods 0.000 claims abstract description 42
- 238000012549 training Methods 0.000 claims abstract description 23
- 238000003062 neural network model Methods 0.000 claims abstract description 9
- 238000009826 distribution Methods 0.000 claims description 72
- 238000013473 artificial intelligence Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 15
- 238000005192 partition Methods 0.000 claims description 14
- 238000003860 storage Methods 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 6
- 230000011664 signaling Effects 0.000 claims description 4
- 230000007547 defect Effects 0.000 abstract description 2
- 238000013507 mapping Methods 0.000 description 40
- 238000012360 testing method Methods 0.000 description 17
- 238000004422 calculation algorithm Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 14
- 230000006870 function Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 10
- 238000010295 mobile communication Methods 0.000 description 10
- 230000011218 segmentation Effects 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 8
- 238000011156 evaluation Methods 0.000 description 7
- 238000011160 research Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 239000006185 dispersion Substances 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000010998 test method Methods 0.000 description 2
- 241001421757 Arcas Species 0.000 description 1
- 101150117498 arcA gene Proteins 0.000 description 1
- 201000003639 autosomal recessive cerebellar ataxia Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005562 fading Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000001303 quality assessment method Methods 0.000 description 1
- 238000012372 quality testing Methods 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Complex Calculations (AREA)
Abstract
The application discloses a communication data set processing method, wherein 2 data samples are arbitrarily taken in a communication data set and respectively used as a first reference sample and a second reference sample; determining the included angle between each data sample in the first data set and the reference sample in the vector space, and generating a first angle parameter set; determining the included angle between each data sample in the second data set and the reference sample in the vector space, and generating a second angle parameter set; generating a first position set and a second position set according to each angle parameter in the first angle parameter set and the second angle parameter set; and determining a data set corresponding to the position set with larger discrete degree in the first position set and the second position set, and entering a target data set for training the neural network model. The embodiment of the application also provides a device for realizing the method. The method and the device solve the problem that the communication data set for model training has quality defects.
Description
Technical Field
The present disclosure relates to the field of mobile communications and computer technologies, and in particular, to a method and an apparatus for processing a communication data set.
Background
With the great improvement of the computing capacity of equipment, the neural network model with huge parameter quantity can complete training in a short time and obtain performance exceeding that of the traditional method, and the surge of research on related algorithms in the industry is initiated. The data set is used as the basis of algorithm research, plays an increasingly prominent role in promoting deep exploration of machine/deep learning algorithms, and the development of the two forms a complementary relationship. On the one hand, researchers develop neural network models and algorithms based on existing datasets to achieve high-precision performance; on the other hand, new models and algorithms may promote the evolution of data sets towards more complex, larger scale routes when the performance on the original data set is saturated. Taking the development of the data set in the image classification field as an example, the early image net data set only comprises 20 types of images, a simple neural network model can obtain excellent performance even in a traditional algorithm, and as the image net data set is expanded to 1000 types to reach millions of data, the original model can not support high-precision performance experience any more, so that researchers are driven to design a higher-level model to improve the precision of the algorithm. In summary, the comprehensive research of the data set and the algorithm is inexhaustible, and the data set and the algorithm are continuously more mature, so that the vigorous development of the artificial intelligence technology is promoted.
In order to facilitate the tight coupling of the mobile communication field and artificial intelligence technology, it is essential to construct high quality wireless communication data sets. On the basis of the communication data set, the migration application of the classical neural network architecture in the fields of images, texts, voice and the like can be realized, and the problems of high complexity, limited precision and the like in the traditional communication algorithm are solved; furthermore, the method can help researchers to explore and mine the characteristics of communication data, promote to build a novel neural network structure (for example, a convolutional neural network is born in the image field, a cyclic neural network is born in the text field and the like) which is more attached to the characteristics of the communication data, and promote cross fusion of multiple fields in other fields.
The neural network model is a fitting model based on massive parameters, and the training effect of the neural network model is highly dependent on the richness of samples in the data set. When the data set scale is smaller, the trained model is usually in an overfitting state, and the robust performance of the model is poor. In addition, the presence of low quality samples (e.g., false marks, blurriness, outliers, etc.) in the data set can also significantly reduce the test performance of the model. Therefore, a high quality data set is an important premise and guarantee for realizing a high-precision algorithm. In the field of relatively sophisticated artificial intelligence, such as image, text and speech, there have been some related studies on dataset testing. For example, resolution, peak signal-to-noise ratio and structural similarity are widely adopted in the image field, segmentation signal-to-noise ratio in the voice field, and the like, and parameters such as information entropy for describing diversity by counting sample categories through labels.
However, the above-described test method is not adapted to quality testing of a communication data set for the following reasons: on one hand, the communication data is not human direct observation data, and a certain piece of communication data cannot be defined as a good sample in a comparison mode; on the other hand, the communication data is more changeable and lacks manual labeling, so that it is difficult to distinguish whether two pieces of communication data which are not labeled belong to the same category. Thus, testing the quality of a communication data set presents a challenge, and there is currently no set of quality assessment theory and method that matches the communication data set.
Disclosure of Invention
The application provides a communication data set processing method and device, which solve the problem that a communication data set for model training has quality defects.
In a first aspect, an embodiment of the present application proposes a method for processing a communication data set, where the communication data set includes one or more of wireless air interface data, signaling data, or network data, and the communication data set includes a first data set and a second data set, and the method includes the following steps:
taking 2 data samples in the communication data set as a first reference sample and a second reference sample respectively to form a target plane in a vector space;
Determining included angles between each data sample in the first data set and 2 reference samples in a vector space respectively, and generating a first angle parameter set;
determining included angles between each data sample in the second data set and 2 reference samples in a vector space respectively, and generating a second angle parameter set;
projection of each angle parameter in the first angle parameter set and the second angle parameter set on the target plane respectively generates a first position set and a second position set;
and determining a data set corresponding to the position set with larger discrete degree in the first position set and the second position set, and entering a target data set for training the neural network model.
In one embodiment of the present application, the first reference sample belongs to a first data set and the second reference sample belongs to a second data set.
In one embodiment of the present application, determining the angle between any one of the data samples and the reference sample includes: in a vector space, calculating cosine similarity between any one data sample and the reference sample; and determining the included angle of any data sample and the reference sample in a vector space according to the cosine similarity.
In one embodiment of the application, polar coordinates are established in the target plane, and a first reference position and a second reference position are determined according to a vector included angle between the first reference sample and the second reference sample; the first position set is generated by superposing the projection of the first angle parameter set on the target plane according to the first reference position; the second set of positions is generated by superimposing a projection of the second set of angular parameters onto the target plane according to the second reference position.
In one embodiment of the present application, determining the degree of discretization of the first set of locations includes: determining a distribution area of a first position set as a first distribution area on a target plane, and dividing the first distribution area by using a dividing angle step length to obtain a first distribution area division set; calculating the probability of the first position set in different partition areas in the first distribution area partition set;
in one embodiment of the present application, determining the degree of discretization of the second set of locations includes: determining a distribution area of the second position set as a second distribution area on the target plane, and dividing the second distribution area by using a dividing angle step length to obtain a second distribution area division set; the probability of the second set of locations being within different partitioned areas in the second set of partitioned areas is calculated.
In one embodiment of the present application, the second data set is obtained by expanding a first data set, including:
training a target-generated artificial intelligence model using N data samples in the first dataset;
combining N data samples in the first data set with M-N virtual samples constructed using a target-generated artificial intelligence model to generate the second data set (M > N) comprising M data samples.
In one embodiment of the present application, a first entropy value and a second entropy value are calculated according to statistical properties of the first set of locations and the second set of locations, respectively; and determining a data set corresponding to the position set with a larger entropy value.
In one embodiment of the present application, JS divergences of the first and second position sets are calculated according to statistical properties of the first and second position sets, respectively;
in one embodiment of the present application, in response to the JS divergence being greater than a set threshold, either data set is included in the target data set, and data samples are added in a set proportion, the added data samples being derived from the other data set.
In one embodiment of the application, the first data set and the second data set are mixed in the target data set in response to the degree of discretization of both the first set of locations and the second set of locations being less than a set threshold.
In a second aspect, an embodiment of the present application further proposes a communication data set processing device, configured to implement a method according to any embodiment of the first aspect of the present application, including:
the acquisition module is used for acquiring a communication data set and first and second reference samples;
the generating module is used for generating the first angle parameter set and the second angle parameter set; and generating a first set of locations and a second set of locations;
and the determining module is used for determining a target data set, wherein the target data set comprises data sets corresponding to position sets with larger discrete degrees in the first position set and the second position set.
In a third aspect, embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, implements a method according to any of the embodiments of the first aspect of the present application.
In a fourth aspect, embodiments of the present application further provide an electronic device, including a memory, a processor, and a computer program stored on the memory and executable by the processor, the processor implementing a method according to any one of the embodiments of the first aspect of the present application when the computer program is executed by the processor.
The above-mentioned at least one technical scheme that this application embodiment adopted can reach following beneficial effect:
the invention designs a processing method for a data set in a mobile communication network, which plays an important role in improving network performance by integrating artificial intelligence technology in the mobile communication network. The communication data set refers to a set formed by putting together a large number of samples of air interface data, signaling data or network data (e.g., channel state information data, scheduling information data, communication quality data, etc.) of a communication system. The communication data set is necessary to be used when the artificial intelligence technology is applied to a communication system, namely the artificial intelligence technology can acquire a corresponding usable artificial intelligence model through executing a training process by a large amount of data, the quality of the data set has an important influence on the training effect of the artificial intelligence model, and a testing method of the data set is also an important research content and a basic tool in the artificial intelligence field. At present, due to the characteristics of complex statistical characteristics, uncertain data set labels and the like of a mobile communication network data set, a relatively perfect test method for the mobile communication network data set is not available in the industry. The invention can be used for carrying out operations such as expanding, characteristic analysis, classification, labeling, statistical characteristic analysis and the like on the mobile communication network data set, can be used for distinguishing the usability of the mobile communication network data set, and can improve the characteristics of diversity and the like of the mobile communication network data set, thereby improving the performance gain brought by the application of the artificial intelligence technology to the mobile communication network.
On the one hand, if the validity of the communication data set can be tested before the artificial intelligence algorithm model training and testing is performed, the cost overhead caused by the artificial intelligence algorithm model training and testing is significantly reduced. On the other hand, quality analysis and comparison are carried out on different communication data sets, and an improvement scheme of the communication data sets is provided on the basis of the quality analysis and comparison, so that generalization and accuracy of the artificial intelligence algorithm model are remarkably improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
FIG. 1 is a data space diagram of a communication data set;
FIG. 2 is a schematic plan view of a reference sample forming a target;
FIG. 3 is a schematic view of an angular projection of a data sample of a dataset onto a target plane;
FIG. 4 is a schematic diagram of first and second sets of data generated from first and second data sets;
FIG. 5 shows angle-split set and sample distribution statistics;
FIG. 6 is a flow chart of an embodiment of a communication dataset processing method of the present application;
FIG. 7 is a flow chart of another embodiment of a communication dataset processing method of the present application;
Fig. 8 is a schematic diagram of an embodiment of a communication data set processing apparatus.
Detailed Description
For the purposes, technical solutions and advantages of the present application, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
Because a standard communication data set is not available for comparison in the research field of communication data sets, for a certain communication data set to be tested, the characteristics of the communication data set can be learned based on an artificial intelligence technology, then the original data of the communication data set is expanded to construct an expanded data set, and the original data set is tested by comparing the characteristics of the original data set and the expanded data set. In the scheme, aiming at an original data set to be tested, a generated artificial intelligent model is used for expanding the original data set to be tested, a virtual data set and an expanded data set are constructed, and then the original data set, the virtual data set and the expanded data set are analyzed by using evaluation indexes such as diversity, similarity and characteristic values of the data sets, and finally test results of the original data set are obtained.
The following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.
Fig. 1 is a data space diagram of a communication data set. The communication data sets may be divided according to types of wireless air interface data, signaling data, network data, etc., and some common data sets include a channel data set, a modulation and coding data set, network traffic data, a network resource management data set, a channel/network time sequence prediction data set, etc. Taking a channel data set as an example, the data contains information such as fading, multipath propagation and the like experienced by a wireless signal in the physical space transmission process, and is embodied in the forms of signal intensity, phase and the like. As shown in fig. 1, an example of a channel strength data set is given in which the horizontal coordinates represent the number of symbols and the vertical coordinates represent the number of subcarriers.
In an embodiment of the present application, the communication data set comprises a first data set (i.e. data set Ω below) and a second data set (i.e. data set E below).
Fig. 2 is a schematic plan view of a reference sample constituting a target. Taking 2 data samples in the communication data set as a first reference sample and a second reference sample x respectively R,1 And x R,2 To form a target plane within the vector space. For example, the first reference sample belongs to a first data set and the second reference sample belongs to a second data set.
Establishing polar coordinates in the target plane according to the vector included angle alpha of the first reference sample and the second reference sample 1,2 A first reference position and a second reference position are determined. And the reference position can be used as a coordinate axis, a first reference coordinate axis and a second reference coordinate axis are obtained through the first reference sample and the second reference sample, and a target coordinate system is established by using the first reference coordinate axis and the second reference coordinate axis.
After the first reference sample and the second reference sample are normalized according to respective modulus values, the first reference position vector is x R,1 /||x R,1 The second reference position vector is x R,2 /||x R,2 I, the included angle alpha between the two 1,2 Forming a circumferential angle. That is, the first reference coordinate axis is calculated from the first reference sample, i.e., the first reference sample is divided by the first reference sampleMould length; and calculating the second reference coordinate axis according to the second reference sample, namely dividing the second reference sample by the module length of the second reference sample.
In order to obtain the included angle between the two, one method is to calculate the cosine similarity between the first reference sample and the second reference sample, wherein the cosine similarity is calculated by taking dot product operation by using the first reference sample and the second reference sample, dividing the dot product operation by the module length of the first reference sample, and dividing the module length of the second reference sample.
Further, an included angle between the first reference sample and the second reference sample is obtained according to cosine similarity between the first reference sample and the second reference sample.
FIG. 3 is a schematic view of an angular projection of a data sample of a dataset onto a target plane. Determining included angles between each data sample in the first data set and 2 reference samples in a vector space respectively, and generating a first angle parameter set; and determining the included angles of each data sample in the second data set and 2 reference samples in a vector space respectively, and generating a second angle parameter set. Projection of each angle parameter in the first angle parameter set and the second angle parameter set on the target plane respectively generates a first position set and a second position set; in the planar polar coordinate space shown in fig. 2-3, the first position set is generated by superimposing the projection of the first angle parameter set on the target plane according to the first reference position; the second set of positions is generated by superimposing a projection of the second set of angular parameters onto the target plane according to the second reference position. When the sampled data are normalized according to their vector length values, both the first set of locations and the second set of locations are located on the circumference as shown in fig. 2-3.
The angle parameter of the present application may be an angle value or a trigonometric function value of the angle value. For example, a specific process may be: in a vector space, calculating cosine similarity between each data sample in the first data set and the first reference sample one by one to obtain a 'one mapping one' parameter set; the cosine similarity is calculated by taking dot product operation by the data sample and the reference sample, dividing the dot product operation by the modular length of the data sample and dividing the modular length of the reference sample;
similarly, calculating cosine similarity between each data sample in the first data set and the second reference sample one by one to obtain a 'one-mapping-two' parameter set;
similarly, calculating cosine similarity between each data sample in the second data set and the first reference sample one by one to obtain a parameter set of two mapping one;
similarly, calculating cosine similarity between each data sample in the second data set and the second reference sample one by one to obtain the 'two-mapping-two' parameter set;
obtaining the position set through the parameter set, including: converting the parameter set of the first mapping one parameter set, the parameter set of the second mapping two parameter set, the parameter set of the second mapping one parameter set and the parameter set of the second mapping two parameter set into an angle set of the first mapping one angle set, an angle set of the second mapping two angle set, an angle set of the second mapping one angle set and an angle set of the second mapping two angle set; according to the angle set of one mapping, the angle set of two mapping, and the angle set of two mapping, all data samples in the first data set and the second data set are mapped into the target coordinate system, and the position set of one mapping, the position set of two mapping, and the position set of two mapping are correspondingly obtained. The position set of the first mapping one and the position set of the second mapping two form a first position set, and the position set of the second mapping one and the position set of the second mapping two form a second position set.
As shown in fig. 3, sample x in dataset Ω Ω,i And sample x in dataset E E,j The angle values are mapped on the circular arcs taking the reference sample as the axis. In sample x Ω,1 For example, according to the reference sample x R,1 And x R,2 Included angle theta between 1,1 And theta 1,2 And two reference samplesIncluded angle alpha between the bodies 1,2 Relation of x Ω,1 The relative position in the coordinate system can be divided into the following three cases:
when satisfy theta 1,1 +α 1,2 ＝θ 1,2 When x is Ω,1 Is positioned in an arcOutside and near reference sample x R,1 As shown in fig. 3 (a);
when satisfy theta 1,1 +θ 1,2 ＝α 1,2 When x is Ω,1 Is positioned in an arcAs shown in fig. 3 (b);
when satisfy theta 1,2 +α 1,2 ＝θ 1,1 When x is Ω,1 Is positioned in an arcOutside and near reference sample x R,2 As shown in fig. 3 (c).
FIG. 4 is a schematic diagram of first and second sets of locations generated by first and second data sets. Determining a distribution area of the first position set as a first distribution area R on the target plane Ω . Determining the distribution area of the second position set as a second distribution area R on the target plane E 。
Specifically, the one mapping angle set and the one mapping two angle set are respectively obtained according to the one mapping one parameter set, namely the one mapping one angle cosine value set and the one mapping two parameter set, namely the one mapping two angle cosine value set. And mapping the data samples in the first data set into the target coordinate system by using the mapping angle set and the mapping two-angle set to obtain a corresponding mapping position set and a mapping position set, which are collectively called as a first position set. And similarly, mapping the data samples in the second data set into the target coordinate system to obtain the corresponding two-mapping one-position set and two-mapping two-position set, which are collectively called as a second position set.
Using the set of locations, obtaining statistical properties of a second data set of the first data set, comprising:
obtaining a first distribution area and a second distribution area of all data samples in the first data set and the second data set on the target coordinate plane according to the first position set and the second position set;
dividing the first distribution area and the second distribution area into dividing areas by using the dividing angle step length to obtain a first distribution area dividing set and a second distribution area dividing set;
dividing all data samples in the first data set and the second data set into a first distribution area segmentation set and a second distribution area segmentation set by using the first mapping first position set, the second mapping second position set, the second mapping first position set and the second mapping second position set;
and calculating the distribution of all the data samples in the first data set and the second data set in different partition areas in the first distribution area partition set and the second distribution area partition set to obtain the statistical characteristics of the first data set and the statistical characteristics of the second data set.
Fig. 5 shows the angle-split set and sample distribution statistics. Dividing the first distribution area by using the dividing angle step length to obtain a first distribution area dividing set; the probability of the first set of locations being within different partitioned areas in the first set of distributed area partitions is calculated. Dividing the second distribution area by using the dividing angle step length to obtain a second distribution area dividing set; the probability of the second set of locations being within different partitioned areas in the second set of partitioned areas is calculated.
As shown in fig. 5, the first distribution area and the second distribution area are divided into divided areas by using a dividing angle step, and the obtained first distribution area division set includesThe second distributed area division set comprises +.>The segmented regions.
Since the samples in the communication dataset and the elements in the location set are corresponding, with respect to probability computation, it may also be: and calculating the number of data samples in the first data set falling into each divided area in the first distribution area divided set, and dividing the number by the total number of data samples in the first data set to obtain the probability that the data samples in the first data set fall into each divided area in the first distribution area divided set. And similarly, obtaining the probability that the data sample in the second data set falls into each partition area in the second distribution area partition set.
Further, diversity and similarity of the data sets may be analyzed based on the probability of distribution of the data samples in the set of segmentation angles.
Regarding diversity analysis, for example, a first statistical characteristic of the first data set, e.g., information entropy of the first data set, is derived from the probability that the data sample in the first data set is located in each of the first set of distribution region partitions and the probability that the data sample in the second data set is located in each of the second set of distribution region partitions; and a second statistical property, e.g., information entropy of the second data set; and testing the first data set and the second data set by the first statistical characteristic and the second statistical characteristic.
Testing the diversity of the first data set and the second data set according to the first statistical characteristic and the second statistical characteristic, comprising:
when the first statistical characteristic is greater than the second statistical characteristic, the diversity of the first data set is better than the diversity of the second data set;
when the first statistical characteristic is less than the second statistical characteristic, the diversity of the first data set is less than the diversity of the second data set;
the diversity of the first data set and the diversity of the second data set are the same when the first statistical characteristic is equal to the second statistical characteristic.
Regarding similarity analysis, performing union operation on the first distribution area and the second distribution area to obtain a third distribution area, and simultaneously obtaining a third distribution area segmentation set according to the first distribution area segmentation set and the second distribution area segmentation set, wherein overlapping parts of the first distribution area and the second distribution area do not carry out repeated statistics;
and calculating the number of the data samples in each of the third distribution area division sets in the first data set, so as to obtain the probability that the data samples in the first data set are in each of the third distribution area division sets, namely dividing the number of the data samples in the first data set in each of the third distribution area division sets by the number of the data samples in the first data set.
And similarly, obtaining the probability that the data sample in the second data set is positioned in each partition area in the third distribution area partition set.
And testing the similarity of the first data set and the second data set by using the probability that the data sample in the first data set is positioned in each divided area in the third distribution area divided set and the probability that the data sample in the second data set is positioned in each divided area in the third distribution area divided set.
And obtaining a third statistical characteristic between the first data set and the second data set according to the probability that the data sample in the first data set is positioned in each divided area in the third distribution area divided set and the probability that the data sample in the second data set is positioned in each divided area in the third distribution area divided set, wherein the third statistical characteristic is JS divergence between the first data set and the second data set.
And testing the similarity of the first data set and the second data set according to the third statistical characteristic, wherein the value range of the third statistical characteristic is [0,1], and the closer to 0 is the higher the similarity of the first data set and the second data set, and the closer to 1 is the lower the similarity of the first data set and the second data set.
FIG. 6 is a flow chart of an embodiment of a communication data set process of the present application.
Step 61: the samples in the communication data set to be tested Φ are data processed using a predefined method.
For example, data cleansing is performed on samples in the communication data set Φ to be tested, and invalid data samples are screened out.
For another example, format specification operation (such as adjusting data dimension, clipping, normalizing, etc.) is performed on the data samples in the to-be-tested communication data set Φ after the invalid data samples are screened, so as to obtain an original data set Ω after data processing.
Step 62: determining a first data set and a second data set to be tested, and distinguishing two cases:
case 1: a comparison test was performed on 2 known data sets (data set Ω and data set E).
Case 2 (see example step 72 for details): testing 1 known data set (data set omega), training a generative artificial intelligent model by using N data samples of the data set omega, performing generative expansion on the data set omega by using the trained generative artificial intelligent model after model training is completed, and generating M-N new samples to form a data set E together with the original N samples in the data set omega.
Step 63: and analyzing the statistical properties of the data set omega and the data set E to be tested.
Step 631: and calculating the angular similarity among the samples in the data set, taking cosine similarity as an example. Arbitrarily selecting two data samples x from the data set omega and the data set E R,1 And x R,2 As a reference sample, and calculate sample x in dataset Ω Ω,i I=1,..n and reference sample x R,k Angular similarity cos θ between k=1, 2 i,k Sample x in dataset E was calculated with i=1,.. E,j J=1,..m and reference sample x R,k Angular similarity cos β between k=1, 2 j,k ,j＝1,...,M,k＝1,2。
Step 632: a new coordinate system is established. Calculating x R,1 And x R,2 Inter-angulation similarity cos alpha 1,2 And converted into an angle value alpha 1,2 Build up of reference sample x R,1 /||x R,1 I and x R,2 /||x R,2 And I is a new coordinate system of two-dimensional plane coordinate axes.
Step 633: the positions of all samples in dataset Ω and dataset E in the new coordinate system are determined. For ease of description, assume that all data samples lie on the target plane, the angular similarity cos θ calculated in step 631 is calculated i,k And cos beta j,k Converted into an angle value theta i,k And beta j,k And sample x in dataset Ω Ω,i And sample x in dataset E E,j According to the angle value theta i,k And beta j,k Corresponding to the reference sample x R,1 And x R,2 The first position set and the second position set are respectively formed on the circular arc of the shaft. After mapping, the samples in the data set omega and the data set E are respectively concentrated and distributed in different sector areas R Ω And R is E Is a kind of medium.
Step 634: the statistical properties of dataset Ω and dataset E are calculated. First at an angleDividing the sample distribution region of the data set Ω and the data set E into L sub-regions for the segmentation step, wherein the sample distribution region R of the data set Ω Ω Comprises L Ω A sub-region. Next, the number of samples in each sub-region is counted as +.>And calculate the probability of occurrence of the sample in each sub-region +.>Analogically statistical sample distribution region R of data set E E Comprising L E Sample number in individual subregion +.>Re-calculating to obtain sample occurrence probability +.>According toAnd->The statistical properties of dataset Ω and dataset E are calculated.
Step 64: optionally, a diversity test is performed on dataset Ω and dataset E. Using the statistical properties of data set Ω and data set E obtained in step 63, entropy values H (Ω) and H (E) are calculated:
the test dataset Ω and dataset E are analyzed for diversity by comparing the entropy values H (Ω) and H (E) of dataset Ω and dataset E based on predefined evaluation rules.
Entropy values represent the degree of dispersion of data. And determining a data set corresponding to the position set with larger discrete degree in the first position set and the second position set, and entering a target data set for training the neural network model.
Step 65: optionally, similarity tests are performed on dataset Ω and dataset E. Using the statistical properties of dataset Ω and dataset E obtained in step 63, a JS divergence JS (P Ω |P E ) The value range of JS divergence is [0,1 ]]The closer to 0 means the higher the similarity of the two data sets, calculated as follows:
the diversity of the test dataset Ω and dataset E is analyzed by comparing JS divergence for dataset Ω and dataset E based on predefined evaluation rules.
FIG. 7 is a flow chart of another embodiment of the communication data set process of the present application.
Step 71: the samples in the communication data set to be tested Φ are data processed using a predefined method.
Step 711: and (3) cleaning the data of the samples in the communication data set phi to be tested, and screening out invalid data samples.
Step 712: and performing format specification operation (such as data dimension adjustment, amplitude limiting, normalization and the like) on the data samples in the communication data set phi to be tested after the invalid data samples are screened out, so as to obtain an original data set omega after data processing.
Step 72: the original dataset Ω is expanded using a generative artificial intelligence model.
Step 721: the raw dataset Ω is used to train the generative artificial intelligence model.
Step 722: M-N virtual samples are generated using the trained generative artificial intelligence model, where M > N.
Step 723: and constructing a virtual data set psi with the sample number of M-N by using the generated M-N virtual samples, and constructing an extended data set E with the sample number of M by using the original data set omega with the sample number of N and the virtual data set psi with the sample number of M-N together.
Step 73: the original dataset Ω and the extended dataset E are subjected to a diversity analysis using predefined evaluation indicators.
Step 731: and quantitatively measuring the similarity among the samples in the data set. Selecting two data samples x from the original data set Ω and the extended data set E R,1 And x R,2 As a reference sample, and calculate sample x in dataset Ω Ω,i I=1,.. E,j J=1,..m and reference sample x R,k K=1, 2 similarity. Cosine similarity is taken as an example, and the calculation mode is adoptedThe following is shown:
step 732: a new coordinate system is established. Calculating x R,1 And x R,2 Cosine similarity cos alpha between 1,2 And converted into an angle value alpha 1,2 Build up of reference sample x R,1 /||x R,1 I and x R,2 /||x R,2 The i is the reference frame of the two-dimensional plane coordinate axis, as shown in fig. 2.
Step 733: the positions of all samples in dataset Ω and dataset E in the new coordinate system are determined. Cosine similarity cos θ calculated in step 731 i,k And cos beta j,k Converted into an angle value theta i,k And beta j,k And sample x in dataset Ω Ω,i And sample x in dataset E E,j The angle values are mapped to arcs around the reference sample as axes, as shown in fig. 3.
The mapped effects are shown in FIG. 4, in which the samples in dataset Ω and dataset E are each centrally distributed in different sector areas R Ω And R is E Is a kind of medium.
Step 734: the diversity of the dataset Ω is evaluated with the entropy value H (Ω) as an index. As shown in fig. 5, first at an angleDividing the sample distribution region of the data set Ω and the data set E into L sub-regions for the segmentation step, wherein the sample distribution region R of the data set Ω Ω Comprises L Ω A sub-region. Then, the number of samples in each sub-region is counted as n Ω,l ,l＝1,...,L Ω And calculate that the sample appears in the first sub-region R Ω,l The probability of p Ω,l ＝n Ω,l N. The entropy value is used as an index for measuring diversity, and the larger the entropy value isThe better the diversity of the data set, the following way of calculation is:
step 735: the diversity of the data set E was evaluated using the entropy value H (E) as an index. The process is the same as step 734, in which the sample distribution region R of the data set E is counted E Comprising L E Number of samples n in each sub-region E,l ,l＝1,...,L E Then calculate the probability p of sample occurrence in each sub-region E,l ＝n E,l And (n+m), and finally calculating the entropy value of the data set E according to the following formula:
it should be noted that, step 735 and step 734 are parallel alternative solutions.
Step 736: based on a predefined evaluation rule, the diversity between the original data set omega and the extended data set E is analyzed, and then the diversity difference between the original data set omega and the extended data set E is judged. Comparing the entropy values H (omega) and H (E) of the data set omega and the data set E calculated in the steps 734-735, and when H (omega) is more than H (E), the diversity of the data set omega is better; conversely, the diversity of the data set E is better.
Step 74: the original dataset Ω and the extended dataset E are subjected to a similarity analysis using predefined evaluation indicators.
Step 741: mapping all samples in dataset Ω and dataset E into the new coordinate system, and the specific process is the same as in steps 731-733.
Step 742: r is calculated Ω ∪R E The probability of sample distribution in the subareas of (a) is p Ω,l And p E,l L=1,.. the specific process of L is the same as steps 734-735.
Step 743: with JS scattering JS (P) Ω |P E ) As an index, the similarity of the data set Ω and the data set E was evaluated. The value range of JS divergence is [0,1 ]]The closer to 0 means the higher the similarity of the two data sets, calculated as follows:
Wherein P is Ω And P E Representing the distribution laws of dataset Ω and dataset E, respectively.
Step 744: based on predefined evaluation rules, the similarity between the original dataset Ω and the expanded dataset E is analyzed.
Fig. 8 is a schematic diagram of an embodiment of a communication data set processing apparatus.
The embodiment of the application also provides a communication data set processing device, which is used for implementing the method of any embodiment of the first aspect of the application, and comprises the following steps:
an acquisition module 81 is configured to acquire the communication data set and the first and second reference samples. The functions implemented by the acquisition module may specifically include the functions described in steps 61 to 62 and steps 71 to 72 of the embodiment.
A generating module 82, configured to generate the first angle parameter set and the second angle parameter set; and is further configured to generate a first set of locations and a second set of locations. The functions implemented by the generating module may specifically include the functions described in the embodiment steps 631 to 633 and 731 to 733.
A determining module 83, configured to determine a target data set, where the target data set includes a data set corresponding to a location set with a greater degree of dispersion in the first location set and the second location set. The functions implemented by the determining module may specifically also include the functions described in the embodiment steps 634, 64, 734 to 736.
In one embodiment, the determining module is further configured to determine a target data set, where the target data set includes a data set that has a high similarity between the first location set and the second location set, and optimally corresponds to a location set where the JS divergence of the first location set and/or the second location set is less than a set threshold. The functions of the determining module may in particular also comprise the functions as described in steps 65, 74.
It should be further noted that, based on the embodiments shown in fig. 1 to 8 of the present application, a target data set is generated, and in one embodiment of the present application, a first entropy value and a second entropy value are calculated according to statistical characteristics of the first location set and the second location set, respectively; and determining a data set corresponding to the position set with a larger entropy value. Alternatively, in one embodiment of the present application, the first data set and the second data set are blended in the target data set in response to the degree of discretization of both the first set of locations and the second set of locations being less than a set threshold. In the preferred embodiment of the present application, the degree of dispersion, i.e., diversity, is expressed in terms of entropy.
Generating a target data set, in another embodiment of the application, calculating JS divergences of the first and second location sets, respectively, based on statistical properties of the first and second location sets; any data set for which the JS divergence is less than the set threshold is determined, or, in one embodiment of the present application, in response to the JS divergence being greater than the set threshold, any data set is included in the target data set and data samples are added in a set proportion, the added data samples being derived from another data set. In the preferred embodiment of the present application, JS divergence is used to represent similarity. Any data set herein is either the first data set or the second data set.
Specifically, for example: when meeting the conditions H (omega) not less than H (E) and H (omega) not less than H set And JS (P) Ω |P E )≤JS set When H is set And JS (JavaScript) set The threshold values of the entropy value and the JS divergence are preset respectively, the fact that the data set omega is similar to the data set E is explained at the moment, the diversity of the data set omega is higher than that of the data set E, meanwhile, the threshold requirement can be met, and the data set omega is used as a target data set for model training;
as another example, when the condition H (E) is equal to or greater than H (Ω), H (E) is equal to or greater than H set And JS (P) Ω |P E )≤JS set When the data set E is used as a target data set, model training is carried out;
for another example, when meetingThe conditions H (omega) are more than or equal to H (E) and H (omega) is more than or equal to H set And JS (P) Ω |P E )＞JS set When the diversity of the data set omega is higher than that of the data set E, the threshold requirement can be met, but the similarity of the data set omega and the target data set is lower, at the moment, the target data set is mainly the data set omega, data are extracted from the data set E according to the proportion gamma and added into the target data set, and then model training is carried out;
as another example, when the condition H (E) is equal to or greater than H (Ω), H (E) is equal to or greater than H set And JS (P) Ω |P E )＞JS set When the model training process is performed, the target data set is mainly a data set E, and data is extracted from the data set omega according to a proportion gamma and added into the target data set process of the model training process;
as another example, when the condition H (E). Ltoreq.H is satisfied set And H (Ω) is less than or equal to H set When both are not meeting the requirement, the index of the mixture of the two data sets, i.e. H (Ω. U.E), is calculated again. When H (Ω ≡E) < H set Model training is performed without using two data sets; when H (Ω ≡E) is greater than or equal to H set And when the two data sets are mixed, performing model training on the target data set generated by mixing the two data sets.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Accordingly, the present application also proposes a computer readable storage medium, on which a computer program is stored, which program, when being executed by a processor, implements a method as described in any of the embodiments of the present application.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Further, the present application also proposes an electronic device (or computing device) comprising a memory, a processor and a computer program stored on the memory and executable by the processor, said processor implementing a method according to any of the embodiments of the present application when said computer program is executed.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media. Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.
Claims (12)
1. A method of processing a communication data set, the communication data set comprising one or more of wireless air interface data, signaling data, or network data, the communication data set comprising a first data set and a second data set, comprising:
Taking 2 data samples in the communication data set as a first reference sample and a second reference sample respectively to form a target plane in a vector space;
determining included angles between each data sample in the first data set and 2 reference samples in a vector space respectively, and generating a first angle parameter set;
determining included angles between each data sample in the second data set and 2 reference samples in a vector space respectively, and generating a second angle parameter set;
projection of each angle parameter in the first angle parameter set and the second angle parameter set on the target plane respectively generates a first position set and a second position set;
and determining a data set corresponding to the position set with larger discrete degree in the first position set and the second position set, and entering a target data set for training the neural network model.
2. The communication data set processing method of claim 1, wherein the first reference sample belongs to a first data set and the second reference sample belongs to a second data set.
3. The method of communication dataset processing as claimed in claim 1, wherein determining an angle between any of the data samples and the reference sample comprises: in a vector space, calculating cosine similarity between any one data sample and the reference sample; and determining the included angle of any data sample and the reference sample in a vector space according to the cosine similarity.
4. The communication data set processing method of claim 1, wherein polar coordinates are established in the target plane, and the first reference position and the second reference position are determined according to a vector included angle of the first reference sample and the second reference sample; the first position set is generated by superposing the projection of the first angle parameter set on the target plane according to the first reference position; the second set of positions is generated by superimposing a projection of the second set of angular parameters onto the target plane according to the second reference position.
5. The communication data set processing method of claim 1, wherein,
determining a degree of discretization of the first set of locations, comprising: determining a distribution area of a first position set as a first distribution area on a target plane, and dividing the first distribution area by using a dividing angle step length to obtain a first distribution area division set; calculating the probability of the first position set in different partition areas in the first distribution area partition set;
determining a degree of discretization of the second set of locations, comprising: determining a distribution area of the second position set as a second distribution area on the target plane, and dividing the second distribution area by using a dividing angle step length to obtain a second distribution area division set; the probability of the second set of locations being within different partitioned areas in the second set of partitioned areas is calculated.
6. The communication data set processing method according to claim 1, wherein the second data set is obtained by a first data set expansion, comprising:
training a target-generated artificial intelligence model using N data samples in the first dataset;
combining N data samples in the first data set with M-N virtual samples constructed using a target-generated artificial intelligence model to generate the second data set (M > N) comprising M data samples.
7. The communication data set processing method of claim 1, wherein,
respectively calculating a first entropy value and a second entropy value according to the statistical characteristics of the first position set and the second position set;
and determining a data set corresponding to the position set with a larger entropy value.
8. The communication data set processing method of claim 1, wherein,
according to the statistical characteristics of the first position set and the second position set, JS divergences of the first position set and the second position set are calculated respectively;
in response to the JS divergence being greater than a set threshold, any data set is included in the target data set, and data samples are added in a set proportion, the added data samples being derived from another data set.
9. The communication data set processing method of claim 1, wherein,
in response to the degree of discretization of both the first set of locations and the second set of locations being less than a set threshold, first data sets and second data sets are blended in the target data set.
10. A communication data set processing device for implementing the method of any one of claims 1 to 9, comprising
The acquisition module is used for acquiring a communication data set and first and second reference samples;
the generating module is used for generating the first angle parameter set and the second angle parameter set; and generating a first set of locations and a second set of locations;
and the determining module is used for determining a target data set, wherein the target data set comprises data sets corresponding to position sets with larger discrete degrees in the first position set and the second position set.
11. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1-9.
12. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 1-9 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311662402.3A CN117807434A (en) | 2023-12-06 | 2023-12-06 | Communication data set processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311662402.3A CN117807434A (en) | 2023-12-06 | 2023-12-06 | Communication data set processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117807434A true CN117807434A (en) | 2024-04-02 |
Family
ID=90424453
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311662402.3A Pending CN117807434A (en) | 2023-12-06 | 2023-12-06 | Communication data set processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117807434A (en) |
Citations (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107273926A (en) * | 2017-06-12 | 2017-10-20 | 大连海事大学 | A kind of linear discriminant analysis dimension reduction method weighted based on cosine similarity |
| CN109657615A (en) * | 2018-12-19 | 2019-04-19 | 腾讯科技（深圳）有限公司 | A kind of training method of target detection, device and terminal device |
| CN111582477A (en) * | 2020-05-09 | 2020-08-25 | 北京百度网讯科技有限公司 | Training method and device of neural network model |
| CN112529172A (en) * | 2019-09-18 | 2021-03-19 | 华为技术有限公司 | Data processing method and data processing apparatus |
| CN112529025A (en) * | 2019-09-17 | 2021-03-19 | 华为技术有限公司 | Data processing method and device |
| CN112884040A (en) * | 2021-02-19 | 2021-06-01 | 北京小米松果电子有限公司 | Training sample data optimization method and system, storage medium and electronic equipment |
| US20210174149A1 (en) * | 2018-11-20 | 2021-06-10 | Xidian University | Feature fusion and dense connection-based method for infrared plane object detection |
| CN112949780A (en) * | 2020-04-21 | 2021-06-11 | 佳都科技集团股份有限公司 | Feature model training method, device, equipment and storage medium |
| WO2021238281A1 (en) * | 2020-05-26 | 2021-12-02 | 华为技术有限公司 | Neural network training method, image classification system, and related device |
| CN113743426A (en) * | 2020-05-27 | 2021-12-03 | 华为技术有限公司 | Training method, device, equipment and computer readable storage medium |
| CN113902022A (en) * | 2021-10-14 | 2022-01-07 | 安徽江淮汽车集团股份有限公司 | Method for acquiring similarity of data samples of automatic driving scene |
| US20220051106A1 (en) * | 2020-08-12 | 2022-02-17 | Inventec (Pudong) Technology Corporation | Method for training virtual animal to move based on control parameters |
| CN114445562A (en) * | 2022-02-17 | 2022-05-06 | 北京大甜绵白糖科技有限公司 | Three-dimensional reconstruction method and device, electronic device and storage medium |
| CN115687924A (en) * | 2022-10-31 | 2023-02-03 | 中国农业银行股份有限公司 | Model training method and device, electronic equipment and storage medium |
| CN115797632A (en) * | 2022-12-01 | 2023-03-14 | 北京科技大学 | Image segmentation method based on multi-task learning |
| CN116086370A (en) * | 2021-11-08 | 2023-05-09 | 北京遥感设备研究所 | Fastening point positioning method and device based on three-dimensional space |
| CN116109799A (en) * | 2023-04-13 | 2023-05-12 | 深圳思谋信息科技有限公司 | Method, device, computer equipment and storage medium for training adjustment model |
| CN116124047A (en) * | 2023-01-19 | 2023-05-16 | 西北工业大学 | Roughness anisotropic parameter measurement method based on normal vector statistical characteristics |
| CN116304710A (en) * | 2023-03-22 | 2023-06-23 | 平安科技（深圳）有限公司 | Complementary sample generation method, device, equipment and storage medium |
| CN116485633A (en) * | 2023-04-10 | 2023-07-25 | 北京城市网邻信息技术有限公司 | Point cloud display diagram generation method and device, electronic equipment and storage medium |
| CN116596044A (en) * | 2023-07-18 | 2023-08-15 | 华能山东发电有限公司众泰电厂 | Power generation load prediction model training method and device based on multi-source data |
| WO2023165361A1 (en) * | 2022-03-02 | 2023-09-07 | 华为技术有限公司 | Data processing method and related device |
| CN116719934A (en) * | 2023-05-23 | 2023-09-08 | 中国人民解放军国防科技大学 | Method for extracting small sample relation under continuous learning based on prompt contrast learning |
| CN116775790A (en) * | 2023-06-21 | 2023-09-19 | 杭州电子科技大学信息工程学院 | Self-adaptive coordinate conversion algorithm based on accurate point positioning of model |
| CN116975651A (en) * | 2023-07-24 | 2023-10-31 | 腾讯科技（深圳）有限公司 | Similarity determination model processing method, target object searching method and device |
| CN117033956A (en) * | 2023-07-20 | 2023-11-10 | 深圳市大数据研究院 | Data processing method, system, electronic equipment and medium based on data driving |
| CN117033995A (en) * | 2022-08-26 | 2023-11-10 | 腾讯科技（深圳）有限公司 | Training method, device, equipment, medium and product of prediction model |
| CN117056771A (en) * | 2023-07-26 | 2023-11-14 | 杭州电子科技大学 | Migration learning method combining intra-class feature alignment and flexible super-parameter measurement learning |
| CN117077018A (en) * | 2023-10-12 | 2023-11-17 | 微网优联科技(成都)有限公司 | Data processing method, device and storage medium based on machine learning |
-
2023
- 2023-12-06 CN CN202311662402.3A patent/CN117807434A/en active Pending
Patent Citations (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107273926A (en) * | 2017-06-12 | 2017-10-20 | 大连海事大学 | A kind of linear discriminant analysis dimension reduction method weighted based on cosine similarity |
| US20210174149A1 (en) * | 2018-11-20 | 2021-06-10 | Xidian University | Feature fusion and dense connection-based method for infrared plane object detection |
| CN109657615A (en) * | 2018-12-19 | 2019-04-19 | 腾讯科技（深圳）有限公司 | A kind of training method of target detection, device and terminal device |
| CN112529025A (en) * | 2019-09-17 | 2021-03-19 | 华为技术有限公司 | Data processing method and device |
| CN112529172A (en) * | 2019-09-18 | 2021-03-19 | 华为技术有限公司 | Data processing method and data processing apparatus |
| CN112949780A (en) * | 2020-04-21 | 2021-06-11 | 佳都科技集团股份有限公司 | Feature model training method, device, equipment and storage medium |
| CN111582477A (en) * | 2020-05-09 | 2020-08-25 | 北京百度网讯科技有限公司 | Training method and device of neural network model |
| WO2021238281A1 (en) * | 2020-05-26 | 2021-12-02 | 华为技术有限公司 | Neural network training method, image classification system, and related device |
| CN113743426A (en) * | 2020-05-27 | 2021-12-03 | 华为技术有限公司 | Training method, device, equipment and computer readable storage medium |
| US20220051106A1 (en) * | 2020-08-12 | 2022-02-17 | Inventec (Pudong) Technology Corporation | Method for training virtual animal to move based on control parameters |
| CN112884040A (en) * | 2021-02-19 | 2021-06-01 | 北京小米松果电子有限公司 | Training sample data optimization method and system, storage medium and electronic equipment |
| CN113902022A (en) * | 2021-10-14 | 2022-01-07 | 安徽江淮汽车集团股份有限公司 | Method for acquiring similarity of data samples of automatic driving scene |
| CN116086370A (en) * | 2021-11-08 | 2023-05-09 | 北京遥感设备研究所 | Fastening point positioning method and device based on three-dimensional space |
| CN114445562A (en) * | 2022-02-17 | 2022-05-06 | 北京大甜绵白糖科技有限公司 | Three-dimensional reconstruction method and device, electronic device and storage medium |
| WO2023165361A1 (en) * | 2022-03-02 | 2023-09-07 | 华为技术有限公司 | Data processing method and related device |
| CN117033995A (en) * | 2022-08-26 | 2023-11-10 | 腾讯科技（深圳）有限公司 | Training method, device, equipment, medium and product of prediction model |
| CN115687924A (en) * | 2022-10-31 | 2023-02-03 | 中国农业银行股份有限公司 | Model training method and device, electronic equipment and storage medium |
| CN115797632A (en) * | 2022-12-01 | 2023-03-14 | 北京科技大学 | Image segmentation method based on multi-task learning |
| CN116124047A (en) * | 2023-01-19 | 2023-05-16 | 西北工业大学 | Roughness anisotropic parameter measurement method based on normal vector statistical characteristics |
| CN116304710A (en) * | 2023-03-22 | 2023-06-23 | 平安科技（深圳）有限公司 | Complementary sample generation method, device, equipment and storage medium |
| CN116485633A (en) * | 2023-04-10 | 2023-07-25 | 北京城市网邻信息技术有限公司 | Point cloud display diagram generation method and device, electronic equipment and storage medium |
| CN116109799A (en) * | 2023-04-13 | 2023-05-12 | 深圳思谋信息科技有限公司 | Method, device, computer equipment and storage medium for training adjustment model |
| CN116719934A (en) * | 2023-05-23 | 2023-09-08 | 中国人民解放军国防科技大学 | Method for extracting small sample relation under continuous learning based on prompt contrast learning |
| CN116775790A (en) * | 2023-06-21 | 2023-09-19 | 杭州电子科技大学信息工程学院 | Self-adaptive coordinate conversion algorithm based on accurate point positioning of model |
| CN116596044A (en) * | 2023-07-18 | 2023-08-15 | 华能山东发电有限公司众泰电厂 | Power generation load prediction model training method and device based on multi-source data |
| CN117033956A (en) * | 2023-07-20 | 2023-11-10 | 深圳市大数据研究院 | Data processing method, system, electronic equipment and medium based on data driving |
| CN116975651A (en) * | 2023-07-24 | 2023-10-31 | 腾讯科技（深圳）有限公司 | Similarity determination model processing method, target object searching method and device |
| CN117056771A (en) * | 2023-07-26 | 2023-11-14 | 杭州电子科技大学 | Migration learning method combining intra-class feature alignment and flexible super-parameter measurement learning |
| CN117077018A (en) * | 2023-10-12 | 2023-11-17 | 微网优联科技(成都)有限公司 | Data processing method, device and storage medium based on machine learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Kasparick et al. | Kernel-based adaptive online reconstruction of coverage maps with side information | |
| CN102938071B (en) | Fuzzy clustering analysis method for detecting synthetic aperture radar (SAR) image changes based on non-local means | |
| CN110555841B (en) | SAR image change detection method based on self-attention image fusion and DEC | |
| CN112508044A (en) | Artificial intelligence AI model evaluation method, system and equipment | |
| CN107545000A (en) | The information-pushing method and device of knowledge based collection of illustrative plates | |
| CN112218330B (en) | Positioning method and communication device | |
| RU2008105393A (en) | COMPUTER-BASED APPLICATION FORMATION AND VERIFICATION OF TRAINING IMAGES INTENDED FOR MULTI-POINT THEOSTATISTIC ANALYSIS | |
| CN108446616B (en) | Road extraction method based on full convolution neural network ensemble learning | |
| CN112216356A (en) | High-entropy alloy hardness prediction method based on machine learning | |
| Redondi | Radio map interpolation using graph signal processing | |
| CN109002792B (en) | SAR image change detection method based on layered multi-model metric learning | |
| Chin et al. | Intelligent indoor positioning based on artificial neural networks | |
| CN107908807B (en) | Small subsample reliability evaluation method based on Bayesian theory | |
| CN109874104A (en) | User location localization method, device, equipment and medium | |
| CN108846381A (en) | SAR image change detection based on maximal margin metric learning | |
| EP3902313A1 (en) | Field strength testing method | |
| Zhang et al. | Floor recognition based on SVM for WiFi indoor positioning | |
| CN112867021B (en) | Improved TrAdaBoost-based indoor positioning method for transfer learning | |
| Shodamola et al. | Towards addressing the spatial sparsity of MDT reports to enable zero touch network automation | |
| Si et al. | An efficient deep convolutional neural network with features fusion for radar signal recognition | |
| Qin et al. | A wireless sensor network location algorithm based on insufficient fingerprint information | |
| CN113947280A (en) | Combined evaluation method based on feedback adjustment weight | |
| CN116528282B (en) | Coverage scene recognition method, device, electronic equipment and readable storage medium | |
| CN117807434A (en) | Communication data set processing method and device | |
| CN115424000A (en) | Pointer instrument identification method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |