WO2024037393A1 - 拓扑识别模型的压缩方法、装置、电子设备及介质 - Google Patents

拓扑识别模型的压缩方法、装置、电子设备及介质 Download PDF

Info

Publication number
WO2024037393A1
WO2024037393A1 PCT/CN2023/111880 CN2023111880W WO2024037393A1 WO 2024037393 A1 WO2024037393 A1 WO 2024037393A1 CN 2023111880 W CN2023111880 W CN 2023111880W WO 2024037393 A1 WO2024037393 A1 WO 2024037393A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
pruning
training data
neural network
training
Prior art date
Application number
PCT/CN2023/111880
Other languages
English (en)
French (fr)
Inventor
苑佳楠
霍超
白晖峰
张港红
高建
郑利斌
于华东
尹志斌
罗安琴
谢凡
申一帆
杨双双
丁啸
Original Assignee
北京智芯微电子科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京智芯微电子科技有限公司 filed Critical 北京智芯微电子科技有限公司
Publication of WO2024037393A1 publication Critical patent/WO2024037393A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Energy or water supply

Definitions

  • the present disclosure relates to the field of computer technology, and specifically to a compression method, device, electronic equipment and medium for a topology recognition model.
  • the topology recognition model can be used to identify the topology structure of the distribution network.
  • the topology recognition model is obtained by training the deep neural network model. The recognition accuracy is high, but due to the complex structure it can only Runs on the server side.
  • power IoT terminal hardware resources are limited, and there are limitations in storage, computing power, power consumption, and bandwidth. Therefore, it is difficult to deploy deep learning models on power IoT terminal devices.
  • embodiments of the present disclosure provide a compression method, device, electronic device, and medium for a topology recognition model.
  • embodiments of the present disclosure provide a compression method for a topology recognition model, which includes: pruning a model to be compressed to obtain a pruning model, and training the pruning model to obtain trained pruning models.
  • model wherein, the model to be compressed is a topology recognition model, and the topology recognition model is a machine learning model running on the server for identifying the topology structure of the power grid;
  • the trained pruning model is The number of bits of the weight parameter is quantized from the first number of bits to the second number of bits to obtain a high-quantization pruning model, and the high-quantization pruning model is trained to obtain a trained high-quantization pruning model; wherein , the second number of bits is smaller than the first number of bits; use the trained high-quantization pruning model as a model to be compressed, and continue to prune and quantize the model to be compressed until a good compression is obtained
  • the topology recognition model is generated, and the compressed topology recognition model is deployed to the power Internet of Things terminal.
  • pruning the model to be compressed to obtain the pruned model includes: determining redundant channels in the model to be compressed through sparse regularization training, and removing them from the model to be compressed. The redundant channel is used to obtain the pruning model.
  • the method further includes: using the fully connected layer as a linear classifier, using the linear classifier to obtain the recognition accuracy of each neural network layer in the model to be compressed; if any of the neural network layers in the model to be compressed The recognition of two adjacent neural network layers If the difference between the recognition accuracy is less than the preset difference threshold, the neural network layer with the lowest recognition accuracy among any two adjacent neural network layers is removed.
  • the fully connected layer is used as a linear classifier
  • using the linear classifier to obtain the recognition accuracy of each neural network layer in the model to be compressed includes: selecting any two from the model to be compressed. adjacent neural network layers, remove one of the two adjacent neural network layers to obtain a neural network layer accuracy prediction model; use the annotated prediction data set to predict the accuracy of the neural network layer
  • the model is tested to obtain the output accuracy of the neural network layer accuracy prediction model, and the output accuracy is used as the identification accuracy of the neural network layer that has not been removed among any two adjacent neural network layers.
  • training the pruning model to obtain a trained pruning model includes: using the model to be compressed to annotate the training data in the unlabeled training data set to obtain a training data set. ; Use the training data set to train the pruning model to obtain a trained pruning model.
  • using the model to be compressed to annotate the training data in the unlabeled training data set to obtain the training data set includes: inputting the training data into the model to be compressed, and converting the training data to the model to be compressed.
  • the output result of the compression model is used as the label of the training data; the training data and the label of the training data are used as a set of training data in the training data set.
  • using the training data set to train the pruning model to obtain a trained pruning model includes: using a loss function, based on the output of the pruning model and the training The label of the training data in the data set is used to obtain the loss value; according to the loss value, it is judged whether the loss function has converged; if so, stop training, otherwise, adjust the parameters of the pruning model and perform the next training until The loss function converges or the number of training times reaches the preset training number threshold.
  • the method further includes: if the number of training times reaches the preset training number threshold and the loss function has not converged, referring to the model to be compressed, increasing channels or neural channels of the pruning model Network layer.
  • the number of bits of the weight parameters of each neural network layer in the trained pruning model is quantized from the first number of bits to the second number of bits to obtain a highly quantized pruning model
  • the method includes: quantizing the weight parameters of each neural network layer in the pruning model from a first number of bits to a second number of bits, and training the pruning model; fully connecting the pruning model The number of bits of the weight parameter of the layer is quantized from the first number of bits to the second number of bits, and the pruning model is trained; the activation function of the pruning model is quantized to obtain the high-quantized pruning branch model.
  • training the high-quantization pruning model to obtain a trained high-quantization pruning model includes: selecting at least one neural network layer from the high-quantization pruning model to construct a loss function; Use the pruning model to annotate the training data in the unlabeled training data set to obtain a training data set; train the high-quantization pruning model according to the loss function and the training data set to obtain a well-trained Highly quantized pruning model.
  • using the pruning model to annotate the training data in the unlabeled training data set to obtain the training data set includes: inputting the training data into the pruning model, and applying the pruning data to the pruning model.
  • the output of the reference network layer in the branch model is used as the label of the training data; wherein the reference network layer corresponds to the at least one neural network layer selected in the high-quantization pruning model; the training data and the labels of the training data as a set of training data in the training data set.
  • training the high-quantity pruning model according to the loss function and the training data set to obtain a trained high-quantity pruning model includes: using the loss function, according to The output of the at least one neural network layer in the high-quantization pruning model and the label of the training data in the training data set are used to obtain a loss value; based on the loss value, it is judged Whether the loss function converges; if so, stop training; otherwise, adjust the parameters of the high-quantization pruning model and perform the next training until the loss function converges or the number of training times reaches the preset training number threshold.
  • the method further includes: if the number of training times reaches the preset training number threshold and the loss function has not converged, referring to the weight parameters of the neural network layer in the pruning model, The number of bits of at least one weight parameter in the high-quantization pruning model is increased from the second number of bits to the first number of bits.
  • quantizing the weight parameters of each neural network layer in the pruning model from a first number of bits to a second number of bits includes: sequentially using a preset algorithm to quantize the weight parameters of each neural network layer in the pruning model from the first number of bits to the second number of bits. Select at least one neural network layer, and quantize the weight parameter of the at least one neural network layer from a first number of bits to a second number of bits.
  • embodiments of the present disclosure provide a compression device for a topology recognition model, including: a pruning module, used to prune the model to be compressed, obtain a pruning model, and train the pruning model, Obtain a trained pruning model; wherein the model to be compressed is a topology recognition model, and the topology recognition model is a machine learning model running on the server for identifying the power grid topology; a quantification module is used to convert the training The number of bits of the weight parameters of each neural network layer in a good pruning model is quantized from the first number of bits to the second number of bits to obtain a high-quantized pruning model, and the high-quantized pruning model is trained, Obtain a trained high-quantization pruning model; wherein the second number of bits is smaller than the first number of bits; use the trained high-quantization pruning model as a model to be compressed, and continue to compress the model to be compressed The model is pruned and quantified until a compressed topology identification model is obtained, and the compressed topology identification model is deployed to the power Internet of Things
  • pruning the model to be compressed to obtain the pruned model includes: determining redundant channels in the model to be compressed through sparse regularization training, and removing them from the model to be compressed. The redundant channel is used to obtain the pruning model.
  • the device further includes: a network layer recognition accuracy acquisition module, used to use the fully connected layer as a linear classifier, and use the linear classifier to obtain the recognition accuracy of each neural network layer in the model to be compressed. ; Removal module, used to remove the difference between the recognition accuracy of any two adjacent neural network layers in the model to be compressed by less than the preset difference threshold. The neural network layer with the lowest recognition accuracy is removed.
  • a network layer recognition accuracy acquisition module used to use the fully connected layer as a linear classifier, and use the linear classifier to obtain the recognition accuracy of each neural network layer in the model to be compressed.
  • Removal module used to remove the difference between the recognition accuracy of any two adjacent neural network layers in the model to be compressed by less than the preset difference threshold. The neural network layer with the lowest recognition accuracy is removed.
  • the fully connected layer is used as a linear classifier
  • using the linear classifier to obtain the recognition accuracy of each neural network layer in the model to be compressed includes: selecting any two from the model to be compressed. adjacent neural network layers, remove one of the two adjacent neural network layers to obtain a neural network layer accuracy prediction model; use the annotated prediction data set to predict the accuracy of the neural network layer
  • the model is tested to obtain the output accuracy of the neural network layer accuracy prediction model, and the output accuracy is used as the identification accuracy of the neural network layer that has not been removed among any two adjacent neural network layers.
  • training the pruning model to obtain a trained pruning model includes: using the model to be compressed to annotate the training data in the unlabeled training data set to obtain a training data set. ; Use the training data set to train the pruning model to obtain a trained pruning model.
  • using the model to be compressed to annotate the training data in the unlabeled training data set to obtain the training data set includes: inputting the training data into the model to be compressed, and converting the training data to the model to be compressed.
  • the output result of the compression model is used as the label of the training data; the training data and the label of the training data are used as a set of training data in the training data set.
  • using the training data set to train the pruning model to obtain a trained pruning model includes: using a loss function, based on the output of the pruning model and the training The labels of the training data in the data set are used to obtain the loss Loss value; according to the loss value, determine whether the loss function has converged; if so, stop training, otherwise, adjust the parameters of the pruning model and perform the next training until the loss function converges or the number of training times reaches Default training times threshold.
  • the device further includes: a channel or network layer adding module, configured to increase the number of times of training with reference to the model to be compressed if the number of training times reaches the preset number of training times threshold and the loss function has not converged yet.
  • a channel or network layer adding module configured to increase the number of times of training with reference to the model to be compressed if the number of training times reaches the preset number of training times threshold and the loss function has not converged yet.
  • the channel or neural network layer of the pruning model configured to increase the number of times of training with reference to the model to be compressed if the number of training times reaches the preset number of training times threshold and the loss function has not converged yet.
  • the number of bits of the weight parameters of each neural network layer in the trained pruning model is quantized from the first number of bits to the second number of bits to obtain a highly quantized pruning model
  • the method includes: quantizing the weight parameters of each neural network layer in the pruning model from a first number of bits to a second number of bits, and training the pruning model; fully connecting the pruning model The number of bits of the weight parameter of the layer is quantized from the first number of bits to the second number of bits, and the pruning model is trained; the activation function of the pruning model is quantized to obtain the high-quantized pruning branch model.
  • training the high-quantization pruning model to obtain a trained high-quantization pruning model includes: selecting at least one neural network layer from the high-quantization pruning model to construct a loss function; Use the pruning model to annotate the training data in the unlabeled training data set to obtain a training data set; train the high-quantization pruning model according to the loss function and the training data set to obtain a well-trained Highly quantized pruning model.
  • using the pruning model to annotate the training data in the unlabeled training data set to obtain the training data set includes: inputting the training data into the pruning model, and pruning the pruning data into the pruning model.
  • the output of the reference network layer in the branch model is used as the label of the training data; wherein the reference network layer corresponds to the at least one neural network layer selected in the high-quantization pruning model; the training data and the labels of the training data as a set of training data in the training data set.
  • training the high-quantity pruning model according to the loss function and the training data set to obtain a trained high-quantity pruning model includes: using the loss function, according to The output of the at least one neural network layer in the high-quantization pruning model and the label of the training data in the training data set are used to obtain a loss value; based on the loss value, it is judged whether the loss function has converged; if so, then Stop training, otherwise, adjust the parameters of the high-quantization pruning model, and perform the next training until the loss function converges or the number of training times reaches the preset training number threshold.
  • the device further includes: an inverse quantization module, configured to refer to the neural network layer in the pruning model if the number of training times reaches the preset training times threshold and the loss function has not converged.
  • the weight parameter increases the number of bits of at least one weight parameter in the high-quantization pruning model from the second number of bits to the first number of bits.
  • quantizing the weight parameters of each neural network layer in the pruning model from a first number of bits to a second number of bits includes: sequentially using a preset algorithm to quantize the weight parameters of each neural network layer in the pruning model from the first number of bits to the second number of bits. Select at least one neural network layer, and quantize the weight parameter of the at least one neural network layer from a first number of bits to a second number of bits.
  • an embodiment of the present disclosure provides a chip, which includes the device described in any embodiment of the second aspect.
  • embodiments of the present disclosure provide an electronic device including a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are processed by the implementer to implement the method described above.
  • embodiments of the present disclosure provide a computer-readable storage medium on which computer instructions are stored, and when the computer instructions are executed by a processor, the above method is implemented.
  • the model to be compressed is pruned to obtain a pruning model, and the pruning model is trained to obtain a trained pruning model; each neural network layer in the trained pruning model is The number of bits of the weight parameter is quantized from the first number of bits to the second number of bits to obtain a high-quantization pruning model, and the high-quantization pruning model is trained to obtain a trained high-quantization pruning model.
  • Figure 1 shows an application scenario diagram of a compression method for a topology recognition model according to an embodiment of the present disclosure.
  • FIG. 2 shows an exemplary flowchart of a compression method of a topology recognition model according to an embodiment of the present disclosure.
  • FIG. 3 shows an exemplary schematic diagram of a model to be compressed according to an embodiment of the present disclosure.
  • FIG. 4 shows an exemplary schematic diagram of training a high-quantization pruning model with reference to the pruning model according to an embodiment of the present disclosure.
  • FIG. 5 shows an exemplary schematic diagram of a compression device of a topology recognition model according to an embodiment of the present disclosure.
  • FIG. 6 shows a structural block diagram of an electronic device according to an embodiment of the present disclosure.
  • FIG. 7 shows a schematic structural diagram of a computer system suitable for implementing methods according to embodiments of the present disclosure.
  • the operations are all operations authorized, confirmed by the user, or actively selected by the user.
  • Figure 1 shows an application scenario diagram of a compression method for a topology recognition model according to an embodiment of the present disclosure.
  • the application scenario may include a server 110, a terminal 120 and a network 130.
  • data or information can be exchanged between the server 110 and the terminal 120 through the network 130 .
  • the server 110 can obtain the information and/or data in the terminal 120 through the network 130, or can send the information and/or data to the terminal 120 through the network 130.
  • the terminal 120 is a power Internet of Things terminal device that needs to deploy a topology identification model.
  • terminal 120 may obey The server 110 receives the compressed convolutional network model, and uses the compressed topology recognition model locally on the terminal 120 to perform the prediction task.
  • the terminal 120 may be one of a mobile device, a tablet computer, and other devices with input and/or output functions, or any combination thereof.
  • the server 110 can prune the model to be compressed to obtain a pruning model, and train the pruning model to obtain a trained pruning model.
  • the server 110 can quantize the number of bits of the weight parameters of each neural network layer in the trained pruning model from the first number of bits to the second number of bits to obtain a highly quantized pruning model. And train the high-quantity pruning model to obtain the trained high-quantity pruning model.
  • the server 110 may be a single server or a group of servers.
  • the server group may be centralized or distributed (for example, the server 110 may be a distributed system), may be dedicated, or may be provided by other devices or systems simultaneously.
  • server 110 may be local or remote.
  • the server 110 may be implemented on a cloud platform or provided in a virtual manner.
  • cloud platforms may include private clouds, public clouds, hybrid clouds, community clouds, distributed clouds, on-premise clouds, multi-tier clouds, etc., or any combination thereof.
  • network 130 may be any one or more of a wired network or a wireless network.
  • network 130 may include a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), etc., or any combination thereof.
  • LAN local area network
  • WAN wide area network
  • WLAN wireless local area network
  • MAN metropolitan area network
  • FIG. 2 shows an exemplary flowchart of a compression method of a topology recognition model according to an embodiment of the present disclosure.
  • the compression method of the topology recognition model includes the following steps S201–S202:
  • step S201 the model to be compressed is pruned to obtain a pruning model, and the pruning model is trained to obtain a trained pruning model.
  • the model to be compressed is the deep neural network model that needs to be compressed.
  • the model to be compressed can be a pre-trained topology recognition model.
  • the topology recognition model is a machine learning model running on the server for identifying the topology of the power grid.
  • the topology recognition model can be constructed based on a convolutional neural network model.
  • the input of the topology identification model can be the voltage and current data on the low-voltage station side of the power supply transformer, and the output can be the location of each meter in the power supply transformer station and the correlation between each meter.
  • the prediction accuracy of the topology recognition model can be set and trained using the training data set. The purpose of training is to determine the parameters of the topology recognition model.
  • the trained topology recognition model as the model to be compressed, ensure that the accuracy drop value of the model to be compressed is within the prediction range, and compress the model to be compressed.
  • the training data of the training data set can be the historical voltage and current data on the low-voltage station side of the power supply transformer.
  • the training data is annotated with the location of each meter in the power supply transformer zone corresponding to the historical voltage and current data, and the correlation between the meters. relation.
  • the model to be compressed has high recognition accuracy and a large number of redundant weight parameters. During the process of processing input data by the model to be compressed, not all weight parameters participate in effective calculations and have an impact on the prediction results.
  • the pruning method reduces the size of the model network to be compressed and reduces the computational complexity by removing redundant weight parameters, channels or network layers from the network structure of the model to be compressed.
  • sparse regularization training can be used to determine redundant channels in the model to be compressed and remove the redundant channels from the model to be compressed to obtain a pruned model.
  • Sparse refers to eliminating some features in the data to generalize the model and reduce the chance of overfitting.
  • Regularization refers to reducing the scope of solution by adding some rules (restrictions) to the objective function that needs to be trained.
  • the BN layer scaling factor can be introduced as a numerical representation of the activation degree of the corresponding channel.
  • the intermediate network layer of the convolutional neural network includes a lot of important information related to the prediction task. By analyzing the correlation between any intermediate layer of the convolutional neural network and its adjacent network layer, it can be achieved while maintaining the recognition accuracy. More accurate pruning.
  • the fully connected layer can be used as a linear classifier, and the linear classifier can be used to obtain the recognition accuracy of each neural network layer in the model to be compressed; if the recognition accuracy of any two adjacent neural network layers in the model to be compressed is If the difference between them is less than the preset difference threshold, the neural network layer with the lowest recognition accuracy among any two adjacent neural network layers will be removed.
  • any two adjacent neural network layers can be selected from the model to be compressed, and one of the two adjacent neural network layers can be removed to obtain the accuracy prediction model of the neural network layer; using The prediction data set is used to test the neural network layer accuracy prediction model, and the output accuracy of the neural network layer accuracy prediction model is obtained.
  • the output accuracy is used as the recognition accuracy of the neural network layer that has not been removed in any two adjacent neural network layers. .
  • the model to be compressed (i.e., the topology recognition model) is constructed according to the convolutional neural network model.
  • the model to be compressed includes multiple convolutional layers (the actual model to be compressed includes many convolutional network layers. For description For convenience, only convolutional layer 1 and convolutional layer 2) and multiple fully connected layers are drawn.
  • the fully connected layer (e.g., fully connected layer 1) of the topology recognition model can be used as a linear classifier to evaluate the effectiveness of each convolution layer (e.g., convolution layer 1 and convolution layer 2), using adaptive average pooling.
  • the algorithm unifies the embedding vector lengths of each convolutional layer, so that convolutional layers with different feature traits output feature maps of the same size.
  • the convolution layer 1 can be removed, and then the linear classifier can be fine-tuned layer by layer through the backpropagation algorithm. Each layer can obtain the best recognition accuracy.
  • the topology recognition model at this time can be used as the neural network layer accuracy prediction model.
  • Multiple test data in the annotated test data set are input into the neural network layer accuracy prediction model respectively, and multiple test results are obtained.
  • the accuracy prediction model of the neural network layer can be obtained.
  • the prediction accuracy is used as the recognition accuracy of convolutional layer 2. Following the same method as the above example, the recognition accuracy of layer 1 can be convolved.
  • Some embodiments of the present disclosure introduce knowledge distillation technology, by comparing the model to be compressed (trained topology recognition model) and the pruning model, obtain the distillation loss, establish a loss function, and realize the guidance of the pruning model by the model to be compressed, thereby Effectively improve the performance of pruning models.
  • the model to be compressed can be used to label the training data in the unlabeled training data set to obtain a training data set.
  • the training data can be input into the model to be compressed, the output result of the model to be compressed can be used as the label of the training data, and the training data and the label of the training data can be used as a set of training data in the training data set.
  • the training data set obtained by the above method can be used to train the pruning model to obtain a trained pruning model.
  • a loss function for example, square loss function
  • a loss function can be used, according to the output of the pruning model and the label of the training data in the training data set to obtain the loss value; based on the loss value, determine whether the loss function has converged (for example, determine whether the loss value is less than the preset threshold); if so, stop training, otherwise, adjust the parameters of the pruning model, And perform the next training until the loss function converges or the number of training times reaches the preset training number threshold.
  • the model to be compressed can indicate that the pruning process has affected the accuracy of the model to be compressed. Then, you can refer to the model to be compressed to increase the channels or neural channels of the pruning model.
  • Network layer During the specific implementation process, the channels or neural network layers pruned from the model to be compressed can be added to the pruning model one by one. After each channel or neural network layer is added, the model is trained as above until the loss function converges during the training process. , then the pruning model when the loss function converges is used as the trained pruning model.
  • step S202 the number of bits of the weight parameters of each neural network layer in the trained pruning model is quantized from the first number of bits to the second number of bits to obtain a high-quantized pruning model, and the high-quantized pruning model is obtained.
  • the branch model is trained to obtain a trained high-quantization pruning model; in which the second number of bits is smaller than the first number of bits.
  • Model quantization refers to approximating some similar weight parameters of the deep neural network model to the same value. By reducing the bit width representation of the weight parameters, the model is compressed and accelerated, so that the model accuracy and performance are basically unchanged, but the model can be reduced. A compression method that increases scale and speeds up operations.
  • the bit width of the model weight parameter can be represented by the number of bits.
  • the number of bits of the weight parameter can be: 1, 2, 4, 8, 16, 32 and other powers of 2.
  • the weight parameters of deep learning models are usually 32 bits. High bitrate means that the recognition accuracy that the model can achieve is very high. At the same time, the size of the model is large, the inference speed is slow, and the consumption of hardware resources is high.
  • the weight parameters of each neural network layer in the pruning model can be gradually quantified.
  • the weight parameters of each neural network layer in the pruning model are first quantized to a higher number of bits, and then the higher number of bits is used. The number of bits continues to be quantized to a lower number of bits, and so on until the compression task is completed.
  • the weight parameters of each neural network layer in the pruning model can be quantized from the first bit to the second bit, and the pruning model can be trained; the fully connected layer of the pruning model The number of bits of the weight parameter is quantized from the first number of bits to the second number of bits, and the pruning model is trained; the activation function of the pruning model is quantified to obtain a highly quantized pruning model.
  • the weight parameters of each neural network layer and fully connected layer in the pruning model can be quantized from 32 bits to 16 bits.
  • a preset algorithm may be used to select at least one neural network layer from the pruning model one after another, and the weight parameter of the at least one neural network layer may be quantized from the first number of bits to the second number of bits.
  • Preset algorithms may include but are not limited to: random strategies, prune-based strategies, etc.
  • the random strategy is: randomly divide the weight parameters into disjoint parts, that is, group them with equal probability.
  • the strategy based on prune is: determine the threshold of each layer layer by layer (usually given according to the split ratio), compare the absolute value of the weight parameter with the threshold, and divide it into non-overlapping groups.
  • the high-quantized pruning model can be trained to obtain the trained high-quantized pruning model.
  • Embodiments of the present disclosure introduce the idea of knowledge distillation to train high-bit quantization models.
  • the model backbones of the pruning model and the high-quantization pruning model are exactly the same, except for the differences in parameters and activation functions.
  • the high-quantization pruning model can be divided into different parts, and multiple parts are selected to construct the guidance loss function as a constraint function. Accuracy constraints are imposed on the above quantized results.
  • the weight parameters of the high-quantization pruning model are guided by the weight parameters of the pruning model to compensate for the decrease in model accuracy caused by the quantization process and improve the accuracy of the quantitative pruning model.
  • At least one neural network layer can be selected from the highly quantized pruning model to construct a loss function (for example, an absolute value loss function), and the pruning model is used to perform the unlabeled training data set Label the training data to obtain the training data set.
  • the training data can be input into the pruning model, and the output of the reference network layer in the pruning model is used as the label of the training data; among them, the reference network layer and the high-quantization pruning model
  • the selected at least one neural network layer corresponds to the training data and the label of the training data as a set of training data in the training data set.
  • the high-quantization pruning model can be trained based on the loss function and the training data set to obtain the trained high-quantization pruning model.
  • a loss function can be used to obtain a loss value based on the predicted value of the output of at least one neural network layer in the high-quantization pruning model and the label of the training data in the training data set. According to the loss value to determine whether the loss function has converged. If so, stop training. Otherwise, adjust the parameters of the high-quantization pruning model and perform the next training until the loss function converges or the number of training times reaches the preset training number threshold.
  • the weight parameter increases the number of bits of at least one weight parameter in the high-quantization pruning model from the second number of bits to the first number of bits.
  • step S203 the trained high-quantization pruning model is used as a model to be compressed, and the model to be compressed is continued to be pruned and quantized until a compressed topology identification model is obtained, and the compressed topology identification model is deployed to the power plant. connection terminal.
  • the trained high-quantized pruning model can be used as the model to be compressed, and the model to be compressed can be pruned and quantized according to steps S201 to S202. , until the compressed topology recognition model is obtained.
  • the compressed topology recognition model can be a topology recognition model that meets the preset model indicators, and the compressed topology recognition model is deployed to the power IoT terminal.
  • the topology recognition model identifies topological characteristic signals by sampling and monitoring the power lines. Mainly For voltage, current, power and other information. What the model obtains is topology identification feature information, which can realize physical topology identification of the distribution station area.
  • the preset model indicators include: compression ratio, calculation amount, thrust delay, accuracy loss, etc.
  • the application of the compressed topology identification model includes: the low-voltage distribution network line topology node sensing system mainly includes two on-site sensing devices, one is the main terminal arranged on the distribution transformer side of the station area, and the other is arranged on the cable
  • the slave terminals on the branch box and metering box sides are configured with current transformers.
  • the device uses power frequency communication technology, carrier communication technology, and node dynamic sensing algorithms to identify and verify the connection relationships of low-voltage distribution network topology nodes.
  • the basic principle is to take advantage of the fact that power frequency communication signals are difficult to cross-talk between stations.
  • the slave machine determines whether it is the same station area by receiving the station area signature code sent by the host. The same idea is used for node identification in the station area.
  • the main terminal broadcasts the characteristic code to the station area. After receiving the characteristic code from the terminal, it is judged whether it belongs to the station area. If it belongs to the station area, it will be registered in the network of the main terminal, thus Get a list of all nodes in the station area.
  • the communication between master and slave terminals uses power line carrier technology.
  • FIG. 5 shows a structural block diagram of a compression device of a topology recognition model according to an embodiment of the present disclosure.
  • the device can be implemented as part or all of the electronic device through software, hardware, or a combination of both.
  • the compression device 500 of the topology recognition model includes a pruning module 510 and a quantization module 520 .
  • the pruning module 510 is used to prune the model to be compressed to obtain a pruning model, and to train the pruning model to obtain to the trained pruning model; wherein the model to be compressed is a topology recognition model, and the topology recognition model is a machine learning model running on the server for identifying the topology of the power grid.
  • the quantization module 520 is used to quantize the number of bits of the weight parameters of each neural network layer in the trained pruning model from the first number of bits to the second number of bits to obtain a highly quantized pruning model, and
  • the high-quantization pruning model is trained to obtain a trained high-quantization pruning model; wherein the second number of bits is smaller than the first number of bits.
  • the deployment module 530 is used to use the trained high-quantization pruning model as a model to be compressed, continue to prune and quantify the model to be compressed, until a compressed topology recognition model is obtained, and compress the compressed model.
  • the topology recognition model is deployed to the power IoT terminal.
  • pruning the model to be compressed to obtain the pruned model includes: determining redundant channels in the model to be compressed through sparse regularization training, and removing them from the model to be compressed. The redundant channel is used to obtain the pruning model.
  • the device further includes: a network layer recognition accuracy acquisition module, used to use the fully connected layer as a linear classifier, and use the linear classifier to obtain the recognition accuracy of each neural network layer in the model to be compressed. ; Removal module, used to remove the difference between the recognition accuracy of any two adjacent neural network layers in the model to be compressed by less than the preset difference threshold. The neural network layer with the lowest recognition accuracy is removed.
  • a network layer recognition accuracy acquisition module used to use the fully connected layer as a linear classifier, and use the linear classifier to obtain the recognition accuracy of each neural network layer in the model to be compressed.
  • Removal module used to remove the difference between the recognition accuracy of any two adjacent neural network layers in the model to be compressed by less than the preset difference threshold. The neural network layer with the lowest recognition accuracy is removed.
  • the fully connected layer is used as a linear classifier
  • using the linear classifier to obtain the recognition accuracy of each neural network layer in the model to be compressed includes: selecting any two from the model to be compressed. adjacent neural network layers, remove one of the two adjacent neural network layers to obtain a neural network layer accuracy prediction model; use the annotated prediction data set to predict the accuracy of the neural network layer
  • the model is tested to obtain the output accuracy of the neural network layer accuracy prediction model, and the output accuracy is used as the identification accuracy of the neural network layer that has not been removed among any two adjacent neural network layers.
  • training the pruning model to obtain a trained pruning model includes: using the model to be compressed to annotate the training data in the unlabeled training data set to obtain a training data set. ; Use the training data set to train the pruning model to obtain a trained pruning model.
  • using the model to be compressed to annotate the training data in the unlabeled training data set to obtain the training data set includes: inputting the training data into the model to be compressed, and converting the training data to the model to be compressed.
  • the output result of the compression model is used as the label of the training data; the training data and the label of the training data are used as a set of training data in the training data set.
  • using the training data set to train the pruning model to obtain a trained pruning model includes: using a loss function, based on the output of the pruning model and the training The label of the training data in the data set is used to obtain the loss value; according to the loss value, it is judged whether the loss function has converged; if so, stop training, otherwise, adjust the parameters of the pruning model and perform the next training until The loss function converges or the number of training times reaches the preset training number threshold.
  • the device further includes: a channel or network layer adding module, configured to increase the number of times of training with reference to the model to be compressed if the number of training times reaches the preset number of training times threshold and the loss function has not converged yet.
  • a channel or network layer adding module configured to increase the number of times of training with reference to the model to be compressed if the number of training times reaches the preset number of training times threshold and the loss function has not converged yet.
  • the channel or neural network layer of the pruning model configured to increase the number of times of training with reference to the model to be compressed if the number of training times reaches the preset number of training times threshold and the loss function has not converged yet.
  • the number of bits of the weight parameters of each neural network layer in the trained pruning model is quantized from the first number of bits to the second number of bits to obtain a highly quantized pruning model
  • the method includes: quantizing the weight parameters of each neural network layer in the pruning model from a first number of bits to a second number of bits, and training the pruning model; The number of bits of the weight parameter of the fully connected layer is quantized from the first number of bits to the second number of bits, and the pruning model is trained; the activation function of the pruning model is quantized to obtain the high Quantitative pruning model.
  • training the high-quantization pruning model to obtain a trained high-quantization pruning model includes: selecting at least one neural network layer from the high-quantization pruning model to construct a loss function; Use the pruning model to annotate the training data in the unlabeled training data set to obtain a training data set; train the high-quantization pruning model according to the loss function and the training data set to obtain a well-trained Highly quantized pruning model.
  • using the pruning model to annotate the training data in the unlabeled training data set to obtain the training data set includes: inputting the training data into the pruning model, and applying the pruning data to the pruning model.
  • the output of the reference network layer in the branch model is used as the label of the training data; wherein the reference network layer corresponds to the at least one neural network layer selected in the high-quantization pruning model; the training data and the labels of the training data as a set of training data in the training data set.
  • training the high-quantity pruning model according to the loss function and the training data set to obtain a trained high-quantity pruning model includes: using the loss function, according to The output of the at least one neural network layer in the high-quantization pruning model and the label of the training data in the training data set are used to obtain a loss value; based on the loss value, it is judged whether the loss function has converged; if so, then Stop training, otherwise, adjust the parameters of the high-quantization pruning model, and perform the next training until the loss function converges or the number of training times reaches the preset training number threshold.
  • the device further includes: an inverse quantization module, configured to refer to the neural network layer in the pruning model if the number of training times reaches the preset training times threshold and the loss function has not converged.
  • the weight parameter increases the number of bits of at least one weight parameter in the high-quantization pruning model from the second number of bits to the first number of bits.
  • quantizing the weight parameters of each neural network layer in the pruning model from a first number of bits to a second number of bits includes: sequentially using a preset algorithm to quantize the weight parameters of each neural network layer in the pruning model from the first number of bits to the second number of bits. Select at least one neural network layer, and quantize the weight parameter of the at least one neural network layer from a first number of bits to a second number of bits.
  • the compression device of the topology recognition model includes: a processor, wherein the processor is used to execute the above program module stored in the memory, including: the pruning module 510, the quantization module 520, and the deployment module 530 , network layer recognition accuracy acquisition module, removal module, network layer addition module and inverse quantization module.
  • Embodiments of the present disclosure also provide a chip, which includes a compression device for the above topology recognition model.
  • the device can be implemented as part or all of the chip through software, hardware, or a combination of both.
  • FIG. 6 shows a structural block diagram of the electronic device according to an embodiment of the present disclosure.
  • the electronic device includes a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the present disclosure. method of the embodiment.
  • FIG. 7 shows a schematic structural diagram of a computer system suitable for implementing methods according to embodiments of the present disclosure.
  • the computer system includes a processing unit that can execute various functions in the above-described embodiments according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). method.
  • ROM read-only memory
  • RAM random access memory
  • various programs and data required for the operation of the computer system are also stored.
  • the processing unit, ROM and RAM communicate with each other via a bus connected. Input/output (I/O) interfaces are also connected to the bus.
  • the following components are connected to the I/O interface: an input section including a keyboard, mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section including a hard disk, etc.; and a storage section including a hard disk, etc.
  • the communication part of the network interface card such as LAN card and modem.
  • the communication section performs communication processes via a network such as the Internet.
  • Drives are also connected to I/O interfaces as needed.
  • Removable media such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed, so that the computer program read therefrom is installed into the storage section as needed.
  • the processing unit can be implemented as a processing unit such as CPU, GPU, TPU, FPGA, NPU, etc.
  • the method described above may be implemented as a computer software program.
  • embodiments of the present disclosure include a computer program product including a computer program tangibly embodied on a machine-readable medium, the computer program including program code for performing the above-described method.
  • the computer program may be downloaded and installed from the network via the communications component, and/or installed from removable media.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more components for implementing the specified logical function(s).
  • Executable instructions may also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.
  • the units or modules described in the embodiments of the present disclosure may be implemented in software or programmable hardware.
  • the described units or modules may also be provided in the processor, and the names of these units or modules do not constitute a limitation on the units or modules themselves under certain circumstances.
  • the present disclosure also provides a computer-readable storage medium.
  • the computer-readable storage medium may be the computer-readable storage medium included in the electronic device or computer system in the above embodiments; it may also exist independently. , a computer-readable storage medium that is not installed in the device.
  • the computer-readable storage medium stores one or more programs, which are used by one or more processors to perform the methods described in the present disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Business, Economics & Management (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Economics (AREA)
  • Public Health (AREA)
  • General Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Water Supply & Treatment (AREA)
  • Tourism & Hospitality (AREA)
  • Strategic Management (AREA)
  • Primary Health Care (AREA)
  • Marketing (AREA)
  • Image Analysis (AREA)

Abstract

本公开涉及计算机技术领域，具体涉及一种拓扑识别模型的压缩方法、装置、电子设备及介质，所述拓扑识别模型的压缩方法包括：对待压缩模型进行剪枝，得到剪枝模型，并对所述剪枝模型进行训练，得到训练好的剪枝模型；将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，并对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型；将所述训练好的高量化剪枝模型作为待压缩模型，继续对所述待压缩模型进行剪枝和量化，直至得到压缩好的拓扑识别模型，并将所述压缩好的拓扑识别模型部署到电力物联终端，从而在确保拓扑识别模型的精度满足需求的情况下，减小拓扑识别模型的尺寸。

Description

拓扑识别模型的压缩方法、装置、电子设备及介质 技术领域
本公开涉及计算机技术领域，具体涉及一种拓扑识别模型的压缩方法、装置、电子设备及介质。
背景技术
精确地获知配电网的拓扑结构是电网规划、运行和控制的先决条件。然而，随着大量新能源接入，配电网拓扑变化频繁，因此，配电网的拓扑结构往往是未知的。为了精确识别配电网的拓扑结构，可以使用拓扑识别模型对配电网的拓扑结构进行识别，拓扑识别模型是对深度神经网络模型进行训练得到的，识别精度较高，但由于结构复杂只能运行在服务端。随着配电网业务的发展，需要在电力物联终端上运行拓扑识别模型，以便捷的识别供电变压器台区测的拓扑结构。然而电力物联终端硬件资源有限，存在存储、计算能力、功耗和带宽等方面的限制，因此很难在电力物联终端设备上部署深度学习模型。
对拓扑识别模型进行压缩，将压缩好的拓扑识别模型部署在电力物联终端设备，是较好的解决方案。然而对拓扑识别模型进行压缩必然会带来拓扑识别模型测精度下降的问题。
因此，如何在确保拓扑识别模型的精度满足需求的情况下，有效减小拓扑识别模型的尺寸，从而可以在电力物联终端上运行拓扑识别模型，是亟待解决的技术问题。
发明内容
为了解决相关技术中的问题，本公开实施例提供一种拓扑识别模型的压缩方法、装置、电子设备及介质。
第一方面，本公开实施例中提供了一种拓扑识别模型的压缩方法，包括：对待压缩模型进行剪枝，得到剪枝模型，并对所述剪枝模型进行训练，得到训练好的剪枝模型；其中，所述待压缩模型为拓扑识别模型，所述拓扑识别模型为运行在服务端的用于识别电网拓扑结构的机器学习模型；将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，并对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型；其中，所述第二比特位数小于所述第一比特位数；将所述训练好的高量化剪枝模型作为待压缩模型，继续对所述待压缩模型进行剪枝和量化，直至得到压缩好的拓扑识别模型，并将所述压缩好的拓扑识别模型部署到电力物联终端。
在一些实施例中，所述对待压缩模型进行剪枝，得到剪枝模型，包括：通过稀疏正则化训练，确定所述待压缩模型中的冗余通道，并从所述待压缩模型中移除所述冗余通道，得到所述剪枝模型。
在一些实施例中，所述方法还包括：将全连接层作为线性分类器，使用所述线性分类器得到所述待压缩模型中各个神经网络层的识别精度；如果所述待压缩模型中任意两个相邻神经网络层的识 别精度之间的差值小于预设差值阈值，则将所述任意两个相邻神经网络层中识别精度最低的神经网络层移除。
在一些实施例中，所述将全连接层作为线性分类器，使用所述线性分类器得到所述待压缩模型中各个神经网络层的识别精度，包括：从所述待压缩模型中选择任意两个相邻神经网络层，移除所述任意两个相邻神经网络层中的一个神经网络层，得到神经网络层精度预测模型；使用带标注的预测数据集，对所述神经网络层精度预测模型进行测试，得到所述神经网络层精度预测模型的输出精度，将所述输出精度作为所述任意两个相邻神经网络层中未被移除的神经网络层的识别精度。
在一些实施例中，所述对所述剪枝模型进行训练，得到训练好的剪枝模型，包括：使用所述待压缩模型，对无标签训练数据集中的训练数据做标注，得到训练数据集；使用所述训练数据集，对所述剪枝模型进行训练，得到训练好的剪枝模型。
在一些实施例中，所述使用所述待压缩模型，对无标签训练数据集中的训练数据做标注，得到训练数据集，包括：将所述训练数据输入所述待压缩模型，将所述待压缩模型的输出结果作为所述训练数据的标签；将所述训练数据与所述训练数据的标签，作为所述训练数据集中的一组训练数据。
在一些实施例中，所述使用所述训练数据集，对所述剪枝模型进行训练，得到训练好的剪枝模型，包括：使用损失函数，根据所述剪枝模型的输出与所述训练数据集中训练数据的标签，得到损失值；根据所述损失值，判断所述损失函数是否收敛；若是，则停止训练，否则，调整所述剪枝模型的参数，并进行下一次训练，直至所述损失函数收敛或者训练次数达到预设训练次数阈值。
在一些实施例中，所述方法还包括：如果训练次数达到所述预设训练次数阈值，所述损失函数还未收敛，则参照所述待压缩模型，增加所述剪枝模型的通道或者神经网络层。
在一些实施例中，所述将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，包括；将所述剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，并对所述剪枝模型进行训练；将所述剪枝模型的全连接层的权重参数的比特位数由第一比特位数量化到第二比特位数，并对所述剪枝模型进行训练；对所述剪枝模型的激活函数进行量化，得到所述高量化剪枝模型。
在一些实施例中，所述对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型，包括：从所述高量化剪枝模型中选择至少一个神经网络层构建损失函数；使用所述剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集；根据所述损失函数和所述训练数据集，对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型。
在一些实施例中，所述使用所述剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集，包括：将所述训练数据输入所述剪枝模型，将所述剪枝模型中的参照网络层的输出作为所述训练数据的标签；其中，所述参照网络层与所述高量化剪枝模型中被选择的所述至少一个神经网络层对应；将所述训练数据与所述训练数据的标签，作为所述训练数据集中的一组训练数据。
在一些实施例中，所述根据所述损失函数和所述训练数据集，对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型，包括：使用所述损失函数，根据所述高量化剪枝模型中的所述至少一个神经网络层的输出与所述训练数据集中训练数据的标签，得到损失值；根据所述损失值，判断 所述损失函数是否收敛；若是，则停止训练，否则，调整所述高量化剪枝模型的参数，并进行下一次训练，直至所述损失函数收敛或者训练次数达到预设训练次数阈值。
在一些实施例中，所述方法还包括：如果训练次数达到所述预设训练次数阈值，所述损失函数还未收敛，则参照所述剪枝模型中神经网络层的权重参数，将所述高量化剪枝模型中的至少一个权重参数的比特位数由所述第二比特位数增加到所述第一比特位数。
在一些实施例中，所述将所述剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，包括：逐次使用预设算法从所述剪枝模型中选择至少一个神经网络层，将所述至少一个神经网络层的权重参数由第一比特位数量化到第二比特位数。
第二方面，本公开实施例中提供了一种拓扑识别模型的压缩装置，包括：剪枝模块，用于对待压缩模型进行剪枝，得到剪枝模型，并对所述剪枝模型进行训练，得到训练好的剪枝模型；其中，所述待压缩模型为拓扑识别模型，所述拓扑识别模型为运行在服务端的用于识别电网拓扑结构的机器学习模型；量化模块，用于将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，并对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型；其中，所述第二比特位数小于所述第一比特位数；将所述训练好的高量化剪枝模型作为待压缩模型，继续对所述待压缩模型进行剪枝和量化，直至得到压缩好的拓扑识别模型，并将所述压缩好的拓扑识别模型部署到电力物联终端。
在一些实施例中，所述对待压缩模型进行剪枝，得到剪枝模型，包括：通过稀疏正则化训练，确定所述待压缩模型中的冗余通道，并从所述待压缩模型中移除所述冗余通道，得到所述剪枝模型。
在一些实施例中，所述装置还包括：网络层识别精度获取模块，用于将全连接层作为线性分类器，使用所述线性分类器得到所述待压缩模型中各个神经网络层的识别精度；移除模块，用于如果所述待压缩模型中任意两个相邻神经网络层的识别精度之间的差值小于预设差值阈值，则将所述任意两个相邻神经网络层中识别精度最低的神经网络层移除。
在一些实施例中，所述将全连接层作为线性分类器，使用所述线性分类器得到所述待压缩模型中各个神经网络层的识别精度，包括：从所述待压缩模型中选择任意两个相邻神经网络层，移除所述任意两个相邻神经网络层中的一个神经网络层，得到神经网络层精度预测模型；使用带标注的预测数据集，对所述神经网络层精度预测模型进行测试，得到所述神经网络层精度预测模型的输出精度，将所述输出精度作为所述任意两个相邻神经网络层中未被移除的神经网络层的识别精度。
在一些实施例中，所述对所述剪枝模型进行训练，得到训练好的剪枝模型，包括：使用所述待压缩模型，对无标签训练数据集中的训练数据做标注，得到训练数据集；使用所述训练数据集，对所述剪枝模型进行训练，得到训练好的剪枝模型。
在一些实施例中，所述使用所述待压缩模型，对无标签训练数据集中的训练数据做标注，得到训练数据集，包括：将所述训练数据输入所述待压缩模型，将所述待压缩模型的输出结果作为所述训练数据的标签；将所述训练数据与所述训练数据的标签，作为所述训练数据集中的一组训练数据。
在一些实施例中，所述使用所述训练数据集，对所述剪枝模型进行训练，得到训练好的剪枝模型，包括：使用损失函数，根据所述剪枝模型的输出与所述训练数据集中训练数据的标签，得到损 失值；根据所述损失值，判断所述损失函数是否收敛；若是，则停止训练，否则，调整所述剪枝模型的参数，并进行下一次训练，直至所述损失函数收敛或者训练次数达到预设训练次数阈值。
在一些实施例中，所述装置还包括：通道或网络层增加模块，用于如果训练次数达到所述预设训练次数阈值，所述损失函数还未收敛，则参照所述待压缩模型，增加所述剪枝模型的通道或者神经网络层。
在一些实施例中，所述将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，包括；将所述剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，并对所述剪枝模型进行训练；将所述剪枝模型的全连接层的权重参数的比特位数由第一比特位数量化到第二比特位数，并对所述剪枝模型进行训练；对所述剪枝模型的激活函数进行量化，得到所述高量化剪枝模型。
在一些实施例中，所述对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型，包括：从所述高量化剪枝模型中选择至少一个神经网络层构建损失函数；使用所述剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集；根据所述损失函数和所述训练数据集，对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型。
在一些实施例中，所述使用所述剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集，包括：将所述训练数据输入所述剪枝模型，将所述剪枝模型中的参照网络层的输出作为所述训练数据的标签；其中，所述参照网络层与所述高量化剪枝模型中被选择的所述至少一个神经网络层对应；将所述训练数据与所述训练数据的标签，作为所述训练数据集中的一组训练数据。
在一些实施例中，所述根据所述损失函数和所述训练数据集，对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型，包括：使用所述损失函数，根据所述高量化剪枝模型中的所述至少一个神经网络层的输出与所述训练数据集中训练数据的标签，得到损失值；根据所述损失值，判断所述损失函数是否收敛；若是，则停止训练，否则，调整所述高量化剪枝模型的参数，并进行下一次训练，直至所述损失函数收敛或者训练次数达到预设训练次数阈值。
在一些实施例中，所述装置还包括：反量化模块，用于如果训练次数达到所述预设训练次数阈值，所述损失函数还未收敛，则参照所述剪枝模型中神经网络层的权重参数，将所述高量化剪枝模型中的至少一个权重参数的比特位数由所述第二比特位数增加到所述第一比特位数。
在一些实施例中，所述将所述剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，包括：逐次使用预设算法从所述剪枝模型中选择至少一个神经网络层，将所述至少一个神经网络层的权重参数由第一比特位数量化到第二比特位数。
第三方面，本公开实施例提供了一种芯片，所述芯片包括如第二方面中任一实施例所述的装置。
第四方面，本公开实施例提供了一种电子设备，包括存储器和处理器，其中，所述存储器用于存储一条或多条计算机指令，其中，所述一条或多条计算机指令被所述处理器执行以实现如上所述的方法。
第五方面，本公开实施例中提供了一种计算机可读存储介质，其上存储有计算机指令，该计算机指令被处理器执行时实现如上所述的方法。
根据本公开实施例提供的技术方案，对待压缩模型进行剪枝，得到剪枝模型，并对剪枝模型进行训练，得到训练好的剪枝模型；将训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，并对高量化剪枝模型进行训练，得到训练好的高量化剪枝模型。通过剪枝操作和量化操作交替进行，可以在确保深度学习模型的精度满足需求的情况下，有效减小深度学习模型的尺寸。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
结合附图，通过以下非限制性实施方式的详细描述，本公开的其它特征、目的和优点将变得更加明显。在附图中。
图1示出根据本公开实施例的拓扑识别模型的压缩方法的应用场景图。
图2示出根据本公开实施例的拓扑识别模型的压缩方法的示例性流程图。
图3示出根据本公开实施例的待压缩模型的示例性示意图。
图4示出根据本公开实施例的参照剪枝模型对高量化剪枝模型进行训练的示例性示意图。
图5示出根据本公开实施例的拓扑识别模型的压缩装置的示例性示意图。
图6示出根据本公开的实施例的电子设备的结构框图。
图7示出适于用来实现根据本公开实施例的方法的计算机系统的结构示意图。
具体实施方式
下文中，将参考附图详细描述本公开的示例性实施例，以使本领域技术人员可容易地实现它们。此外，为了清楚起见，在附图中省略了与描述示例性实施例无关的部分。
在本公开中，应理解，诸如“包括”或“具有”等的术语旨在指示本说明书中所公开的特征、数字、步骤、行为、部件、部分或其组合的存在，并且不欲排除一个或多个其他特征、数字、步骤、行为、部件、部分或其组合存在或被添加的可能性。
另外还需要说明的是，在不冲突的情况下，本公开中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本公开。
在本公开中，如涉及对用户信息或用户数据的获取操作或向他人展示用户信息或用户数据的操作，则所述操作均为经用户授权、确认，或由用户主动选择的操作。
图1示出根据本公开实施例的拓扑识别模型的压缩方法的应用场景图。
如图1所示，在应用场景中可以包括服务端110、终端120和网络130。
在一些实施例中，服务端110、终端120之间可以通过网络130进行数据或者信息的交互。例如，服务端110可以通过网络130获取终端120中的信息和/或数据，或者可以通过网络130将信息和/或数据发送到终端120。
终端120为需要部署拓扑识别模型的电力物联终端设备。在一些实施例中，终端120可以从服 务端110接收压缩好的卷积网络模型，并在终端120本地使用压缩好的拓扑识别模型执行预测任务。终端120可以是移动设备、平板计算机等具有输入和/或输出功能的设备中的一种或其任意组合。
在一些实施例中，服务端110可以对待压缩模型进行剪枝，得到剪枝模型，并对剪枝模型进行训练，得到训练好的剪枝模型。在一些实施例中，服务端110可以将训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，并对高量化剪枝模型进行训练，得到训练好的高量化剪枝模型。服务端110可以是单一服务器或服务器组。该服务器组可以是集中式或分布式的(例如，服务端110可以是分布式系统)，可以是专用的也可以由其他设备或系统同时提供服务。在一些实施例中，服务端110可以是区域的或者远程的。在一些实施例中，服务端110可以在云平台上实施，或者以虚拟方式提供。仅作为示例，云平台可以包括私有云、公共云、混合云、社区云、分布云、内部云、多层云等或其任意组合。
在一些实施例中，网络130可以是有线网络或无线网络中的任意一种或多种。例如，网络130可以包括局域网络(LAN)、广域网络(WAN)、无线局域网络(WLAN)、城域网(MAN)等或其任意组合。
图2示出根据本公开实施例的拓扑识别模型的压缩方法的示例性流程图。如图2所示，拓扑识别模型的压缩方法包括以下步骤S201–S202：
在步骤S201中，对待压缩模型进行剪枝，得到剪枝模型，并对剪枝模型进行训练，得到训练好的剪枝模型。
待压缩模型是需要对其进行压缩的深度神经网络模型。在一些实施例中，待压缩模型可以为预训练的拓扑识别模型，该拓扑识别模型为运行在服务端的用于识别电网拓扑结构的机器学习模型，拓扑识别模型可以基于卷积神经网络模型构造。拓扑识别模型的输入可以为供电变压器低压台区侧的电压电流数据，输出可以为供电变压器台区里各个电表的位置，以及各个电表之间的关联关系。在具体实施过程中，可以设定好拓扑识别模型的预测精度，使用训练数据集对其进行训练，训练的目的是要确定拓扑识别模型的参数。将训练好的拓扑识别模型作为待压缩模型，确保待压缩模型的精度下降值在预测范围内，对待压缩模型进行压缩。训练数据集的训练数据可以为供电变压器低压台区侧的历史电压电流数据，训练数据的标注为历史上该电压电流数据对应的供电变压器台区里各个电表的位置，以及各个电表之间的关联关系。
待压缩模型有较高的识别精度，同时拥有大量的冗余权重参数。在待压缩模型对输入数据进行处理的过程中，并非全部的权重参数参与有效的计算，并对预测结果产生影响。剪枝方法通过把待压缩模型的网络结构中冗余的权值参数、通道或网络层去掉，减少待压缩模型网络的规模，降低计算复杂度。
在一些实施例中，可以通过稀疏正则化训练，确定待压缩模型中的冗余通道，并从待压缩模型中移除冗余通道，得到剪枝模型。稀疏指的是消除数据中一些特征，用来使模型泛化，减小过拟合的几率。正则化(regularizer)指的是通过给需要训练的目标函数加上一些规则(限制)，缩小求解的范围。在具体实施过程中，可以引入BN层缩放因子作为对应通道的激活程度的数值化表示，通过稀疏正则化训练，减小冗余通道的比例系数，通过设定阈值移除冗余通道，具体可以包括以下步 骤：
1、对模型的卷积层和BN层进行L1正则化训练获得权重参数值稀疏的卷积神经网络。
2、依据滤波器稀疏性和BN层缩放因子，判断卷积神经网络中各网络层的重要性。
3、对稀疏滤波器及对应连接进行结构化剪枝。
卷积神经网络的中间网络层包括很多与预测任务相关的重要信息，通过分析卷积神经网络的任意中间层与和其相邻的网络层的关联性，可以在保持识别精度的条件下，实现更为准确的剪枝。
在一些实施例中，可以将全连接层作为线性分类器，使用线性分类器得到待压缩模型中各个神经网络层的识别精度；如果待压缩模型中任意两个相邻神经网络层的识别精度之间的差值小于预设差值阈值，则将任意两个相邻神经网络层中识别精度最低的神经网络层移除。在具体实施过程中，可以从待压缩模型中选择任意两个相邻神经网络层，移除任意两个相邻神经网络层中的一个神经网络层，得到神经网络层精度预测模型；使用带标注的预测数据集，对神经网络层精度预测模型进行测试，得到神经网络层精度预测模型的输出精度，将输出精度作为任意两个相邻神经网络层中未被移除的神经网络层的识别精度。
仅作为示例，如图3所示，待压缩模型(即拓扑识别模型)根据卷积神经网络模型构建，待压缩模型包括多个卷积层(实际待压缩模型包括很多卷积网络层，为了描述方便，仅画出了卷积层1和卷积层2)和多个全连接层。可以将拓扑识别模型的全连接层(例如，全连接层1)作为评估各卷积层(例如，卷积层1和卷积层2)的有效性的线性分类器，采用自适应平均池化算法统一各卷积层的嵌入向量长度，使得不同特征性状的卷积层输出尺寸相同的特征图。例如，可以去除卷积层1，然后通过反向传播算法对线性分类器进行逐层的微调，每一层得到最佳识别精度，将此时的拓扑识别模型作为神经网络层精度预测模型，将带标注的测试数据集中的多个测试数据分别输入神经网络层精度预测模型，得到多个测试结果，根据多个测试结果与测试数据集中对应测试数据的标注，可以得到神经网络层精度预测模型的预测精度，将该预测精度作为卷积层2的识别精度。按照上述示例相同的方法，可以卷积层1的识别精度。然后将卷积层1的识别精度和卷积层2的识别精度进行减法运算，并判断运算结果的绝对值是否小于预设差值阈值，如果小于预设差值阈值，则说明卷积层1和卷积层2中的一个卷积层对拓扑识别模型的最终预测结果的影响可以忽略，则将卷积层1和卷积层2中识别精度最低的一个卷积层移除。
在上述对待压缩模型的剪枝过程，可能会造成部分有效信息的损失，导致剪枝模型性能有所下降。本公开的一些实施例引入知识蒸馏技术，通过对比待压缩模型(训练好的拓扑识别模型)和剪枝模型，获得蒸馏损失，建立损失函数，实现由待压缩模型对剪枝模型的指导，从而有效提升剪枝模型性能。
在一些实施例中，可以使用待压缩模型，对无标签训练数据集中的训练数据做标注，得到训练数据集。在具体实施过程中，可以将训练数据输入待压缩模型，将待压缩模型的输出结果作为训练数据的标签，将训练数据与训练数据的标签，作为训练数据集中的一组训练数据。
在一些实施例中，可以使用上述方法得到的训练数据集，对剪枝模型进行训练，得到训练好的剪枝模型。在具体实施过程中，可以使用损失函数(例如，平方损失函数)，根据剪枝模型的输出 与训练数据集中训练数据的标签，得到损失值；根据损失值，判断损失函数是否收敛(例如，判断损失值是否小于预设阈值)；若是，则停止训练，否则，调整剪枝模型的参数，并进行下一次训练，直至损失函数收敛或者训练次数达到预设训练次数阈值。
在一些实施例中，如果训练次数达到预设训练次数阈值，损失函数还未收敛，可以说明剪枝处理影响了待压缩模型的精度，则可以参照待压缩模型，增加剪枝模型的通道或者神经网络层。在具体实施过程中，可以将从待压缩模型剪除的通道或者神经网络层逐次增加到剪枝模型中，每一次增加通道或者神经网络层之后，对模型进行如上训练，直至训练过程中损失函数收敛，则将损失函数收敛时的剪枝模型作为训练好的剪枝模型。
在步骤S202中，将训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，并对高量化剪枝模型进行训练，得到训练好的高量化剪枝模型；其中，第二比特位数小于第一比特位数。
模型量化是指将深度神经网络模型某些相近的权重参数近似成为相同的数值，通过降低权重参数的位宽表示来进行模型压缩与加速，使模型准确率性能基本不变，但可以减小模型规模、加快运算速度的一种压缩方法。模型权重参数的位宽可以使用比特位数来表示，权重参数的比特位数可以为：1、2、4、8、16、32等2的幂次倍。深度学习模型的权重参数通常为32比特。高比特意味着模型可以达到的识别精度很高，同时模型的体积很大，推理速度较慢，硬件资源的消耗较多。
在一些实施例中，可以将剪枝模型中各个神经网络层的权重参数进行逐步量化，先将剪枝模型中各个神经网络层的权重参数量化到一个较高的比特位数，然后由较高的比特位数继续量化到较低的比特位数，以此类推，直至压缩任务完成。
在具体实施过程中，可以将剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，并对剪枝模型进行训练；将剪枝模型的全连接层的权重参数的比特位数由第一比特位数量化到第二比特位数，并对剪枝模型进行训练；对剪枝模型的激活函数进行量化，得到高量化剪枝模型。例如，可以将剪枝模型中的各个神经网络层、全连接层的权重参数由32比特量化到16比特。在具体实施过程中，可以逐次使用预设算法从剪枝模型中选择至少一个神经网络层，将至少一个神经网络层的权重参数由第一比特位数量化到第二比特位数。预设算法可以包括但不限于：随机策略、基于prune(剪枝)的策略等。随机策略为：随机把权重参数分成不相交的部分，即以相等概率分组。基于prune的策略为：逐层确定每层的阈值(一般是按照拆分比率给定)，取权重参数的绝对值和阈值比较，分成不重叠的组。
在通过上述方法得到高量化剪枝模型之后，可以对高量化剪枝模型进行训练，得到训练好的高量化剪枝模型。
本公开的实施例引入了知识蒸馏的思想对高比特量化模型进行训练。剪枝模型和高量化剪枝模型的模型骨干完全相同，只是参数和激活函数具有差异，可以将高量化剪枝模型分为不同的部分，选取其中多个部分构建引导损失函数，作为约束函数，对上述量化结果进行精度约束。对于不满足精度的网络层，通过剪枝模型的权重参数对高量化剪枝模型的权重参数进指导，对量化过程带来的模型精度下降进行弥补，提高量化剪枝模型的精度。
在一些实施例中，如图4所示，可以从高量化剪枝模型中选择至少一个神经网络层构建损失函数(例如，绝对值损失函数)，并使用剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集。在具体实施过程中，如图4所示，可以将训练数据输入剪枝模型，将剪枝模型中的参照网络层的输出作为训练数据的标签；其中，参照网络层与高量化剪枝模型中被选择的至少一个神经网络层对应，将训练数据与训练数据的标签，作为训练数据集中的一组训练数据。
在得到训练数据集之后，可以根据损失函数和训练数据集，对高量化剪枝模型进行训练，得到训练好的高量化剪枝模型。
在具体实施过程中，如图4所示，可以使用损失函数，根据高量化剪枝模型中的至少一个神经网络层的输出的预测值与训练数据集中训练数据的标签，得到损失值，根据损失值，判断损失函数是否收敛，若是，则停止训练，否则，调整高量化剪枝模型的参数，并进行下一次训练，直至损失函数收敛或者训练次数达到预设训练次数阈值。
在一些实施例中，如果训练次数达到预设训练次数阈值，损失函数还未收敛，这说明上述量化过程对剪枝模型的精度产生了较大影响，则可以参照剪枝模型中神经网络层的权重参数，将高量化剪枝模型中的至少一个权重参数的比特位数由第二比特位数增加到第一比特位数。在参照剪枝模型，对高量化剪枝模型中的权重参数进行调整后，需要继续对高量化剪枝模型进行训练，直至训练过程中损失函数收敛，则将函数收敛时的高量化剪枝模型作为训练好的高量化剪枝模型。
在步骤S203中，将训练好的高量化剪枝模型作为待压缩模型，继续对待压缩模型进行剪枝和量化，直至得到压缩好的拓扑识别模型，并将压缩好的拓扑识别模型部署到电力物联终端。
在一些实施例中，在经过步骤S202得到训练好的高量化剪枝模型之后，可以将训练好的高量化剪枝模型作为待压缩模型，继续根据步骤S201～S202对待压缩模型进行剪枝和量化，直至得到压缩好的拓扑识别模型。压缩好的拓扑识别模型可以为满足预设模型指标的拓扑识别模型，并将压缩好的拓扑识别模型部署到电力物联终端，其中拓扑识别模型通过对电力线进行采样监测来识别拓扑特征信号，主要为电压、电流、电量等信息。所述模型得到的是拓扑识别特征信息，可实现配电台区物理拓扑识别。其中预设模型指标包括：压缩比、计算量、推力时延、精度损失等。
在本申请的实施例中，应用压缩好的拓扑识别模型包括：低压配网线拓扑节点感知系统主要包括两种现场感知装置，一是布置在台区配变侧的主终端，二是布置在电缆分支箱和计量箱侧的从终端，从终端配置电流互感器。该装置通过工频通信技术、载波通信技术、节点动态感知算法实现低压配网拓扑节点连接关系的识别和校验。其基本原理是利用工频通信信号在台区间难以串扰的性质，从机通过接收主机发送的台区特征码，判定是否为同一台区。台区节点识别亦采用相同的思路，通过主终端向台区广播特征码，从终端接收到特征码后经判断是否属于本台区，若属于本台区则会注册到主终端的网络里，从而得到台区所有节点的列表。其中主从终端的通信使用电力线载波技术。
图5示出根据本公开的实施例的拓扑识别模型的压缩装置的结构框图。其中，该装置可以通过软件、硬件或者两者的结合实现成为电子设备的部分或者全部。
如图5所示，所述拓扑识别模型的压缩装置500包括剪枝模块510、量化模块520。
剪枝模块510，用于对待压缩模型进行剪枝，得到剪枝模型，并对所述剪枝模型进行训练，得 到训练好的剪枝模型；其中，所述待压缩模型为拓扑识别模型，所述拓扑识别模型为运行在服务端的用于识别电网拓扑结构的机器学习模型。
量化模块520，用于将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，并对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型；其中，所述第二比特位数小于所述第一比特位数。
部署模块530，用于将所述训练好的高量化剪枝模型作为待压缩模型，继续对所述待压缩模型进行剪枝和量化，直至得到压缩好的拓扑识别模型，并将所述压缩好的拓扑识别模型部署到电力物联终端。
在一些实施例中，所述对待压缩模型进行剪枝，得到剪枝模型，包括：通过稀疏正则化训练，确定所述待压缩模型中的冗余通道，并从所述待压缩模型中移除所述冗余通道，得到所述剪枝模型。
在一些实施例中，所述装置还包括：网络层识别精度获取模块，用于将全连接层作为线性分类器，使用所述线性分类器得到所述待压缩模型中各个神经网络层的识别精度；移除模块，用于如果所述待压缩模型中任意两个相邻神经网络层的识别精度之间的差值小于预设差值阈值，则将所述任意两个相邻神经网络层中识别精度最低的神经网络层移除。
在一些实施例中，所述将全连接层作为线性分类器，使用所述线性分类器得到所述待压缩模型中各个神经网络层的识别精度，包括：从所述待压缩模型中选择任意两个相邻神经网络层，移除所述任意两个相邻神经网络层中的一个神经网络层，得到神经网络层精度预测模型；使用带标注的预测数据集，对所述神经网络层精度预测模型进行测试，得到所述神经网络层精度预测模型的输出精度，将所述输出精度作为所述任意两个相邻神经网络层中未被移除的神经网络层的识别精度。
在一些实施例中，所述对所述剪枝模型进行训练，得到训练好的剪枝模型，包括：使用所述待压缩模型，对无标签训练数据集中的训练数据做标注，得到训练数据集；使用所述训练数据集，对所述剪枝模型进行训练，得到训练好的剪枝模型。
在一些实施例中，所述使用所述待压缩模型，对无标签训练数据集中的训练数据做标注，得到训练数据集，包括：将所述训练数据输入所述待压缩模型，将所述待压缩模型的输出结果作为所述训练数据的标签；将所述训练数据与所述训练数据的标签，作为所述训练数据集中的一组训练数据。
在一些实施例中，所述使用所述训练数据集，对所述剪枝模型进行训练，得到训练好的剪枝模型，包括：使用损失函数，根据所述剪枝模型的输出与所述训练数据集中训练数据的标签，得到损失值；根据所述损失值，判断所述损失函数是否收敛；若是，则停止训练，否则，调整所述剪枝模型的参数，并进行下一次训练，直至所述损失函数收敛或者训练次数达到预设训练次数阈值。
在一些实施例中，所述装置还包括：通道或网络层增加模块，用于如果训练次数达到所述预设训练次数阈值，所述损失函数还未收敛，则参照所述待压缩模型，增加所述剪枝模型的通道或者神经网络层。
在一些实施例中，所述将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，包括；将所述剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，并对所述剪枝模型进行训练；将所述剪枝模型的 全连接层的权重参数的比特位数由第一比特位数量化到第二比特位数，并对所述剪枝模型进行训练；对所述剪枝模型的激活函数进行量化，得到所述高量化剪枝模型。
在一些实施例中，所述对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型，包括：从所述高量化剪枝模型中选择至少一个神经网络层构建损失函数；使用所述剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集；根据所述损失函数和所述训练数据集，对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型。
在一些实施例中，所述使用所述剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集，包括：将所述训练数据输入所述剪枝模型，将所述剪枝模型中的参照网络层的输出作为所述训练数据的标签；其中，所述参照网络层与所述高量化剪枝模型中被选择的所述至少一个神经网络层对应；将所述训练数据与所述训练数据的标签，作为所述训练数据集中的一组训练数据。
在一些实施例中，所述根据所述损失函数和所述训练数据集，对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型，包括：使用所述损失函数，根据所述高量化剪枝模型中的所述至少一个神经网络层的输出与所述训练数据集中训练数据的标签，得到损失值；根据所述损失值，判断所述损失函数是否收敛；若是，则停止训练，否则，调整所述高量化剪枝模型的参数，并进行下一次训练，直至所述损失函数收敛或者训练次数达到预设训练次数阈值。
在一些实施例中，所述装置还包括：反量化模块，用于如果训练次数达到所述预设训练次数阈值，所述损失函数还未收敛，则参照所述剪枝模型中神经网络层的权重参数，将所述高量化剪枝模型中的至少一个权重参数的比特位数由所述第二比特位数增加到所述第一比特位数。
在一些实施例中，所述将所述剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，包括：逐次使用预设算法从所述剪枝模型中选择至少一个神经网络层，将所述至少一个神经网络层的权重参数由第一比特位数量化到第二比特位数。
在本申请的实施例中，上述拓扑识别模型的压缩装置包括：处理器，其中所述处理器用于执行存在存储器的上述程序模块，包括：所述剪枝模块510，量化模块520，部署模块530、网络层识别精度获取模块、移除模块、网络层增加模块和反量化模块。
上述拓扑识别模型的压缩装置的实施例中，各模块的具体处理及其带来的技术效果可分别参考对应方法实施例中的相关说明，在此不再赘述。
本公开实施例还提供一种芯片，所述芯片包括上述拓扑识别模型的压缩装置，所述装置可以通过软件、硬件或者两者的结合实现成为芯片的部分或者全部。
本公开还公开了一种电子设备，图6示出根据本公开的实施例的电子设备的结构框图。
如图6所示，所述电子设备包括存储器和处理器，其中，存储器用于存储一条或多条计算机指令，其中，所述一条或多条计算机指令被所述处理器执行以实现根据本公开的实施例的方法。
图7示出适于用来实现根据本公开实施例的方法的计算机系统的结构示意图。
如图7所示，计算机系统包括处理单元，其可以根据存储在只读存储器(ROM)中的程序或者从存储部分加载到随机访问存储器(RAM)中的程序而执行上述实施例中的各种方法。在RAM中，还存储有计算机系统操作所需的各种程序和数据。处理单元、ROM以及RAM通过总线彼此 相连。输入/输出(I/O)接口也连接至总线。
以下部件连接至I/O接口：包括键盘、鼠标等的输入部分；包括诸如阴极射线管(CRT)、液晶显示器(LCD)等以及扬声器等的输出部分；包括硬盘等的存储部分；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分。通信部分经由诸如因特网的网络执行通信过程。驱动器也根据需要连接至I/O接口。可拆卸介质，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器上，以便于从其上读出的计算机程序根据需要被安装入存储部分。其中，所述处理单元可实现为CPU、GPU、TPU、FPGA、NPU等处理单元。
特别地，根据本公开的实施例，上文描述的方法可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括有形地包含在机器可读介质上的计算机程序，所述计算机程序包含用于执行上述方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分从网络上被下载和安装，和/或从可拆卸介质被安装。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分，所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元或模块可以通过软件的方式实现，也可以通过可编程硬件的方式来实现。所描述的单元或模块也可以设置在处理器中，这些单元或模块的名称在某种情况下并不构成对该单元或模块本身的限定。
作为另一方面，本公开还提供了一种计算机可读存储介质，该计算机可读存储介质可以是上述实施例中电子设备或计算机系统中所包含的计算机可读存储介质；也可以是单独存在，未装配入设备中的计算机可读存储介质。计算机可读存储介质存储有一个或者一个以上程序，所述程序被一个或者一个以上的处理器用来执行描述于本公开的方法。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的发明范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离所述发明构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。

Claims (27)

  1. 一种拓扑识别模型的压缩方法，其特征在于，所述方法包括：
    对待压缩模型进行剪枝，得到剪枝模型，并对所述剪枝模型进行训练，得到训练好的剪枝模型，如果训练次数达到预设训练次数阈值，损失函数还未收敛，则参照所述待压缩模型，增加所述剪枝模型的通道或者神经网络层；其中，所述待压缩模型为拓扑识别模型，所述拓扑识别模型为运行在服务端的用于识别电网拓扑结构的机器学习模型；
    将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，并对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型；其中，所述第二比特位数小于所述第一比特位数；对于不满足精度的神经网络层，通过剪枝模型的权重参数对高量化剪枝模型的权重参数进指导训练；
    将所述训练好的高量化剪枝模型作为待压缩模型，继续对所述待压缩模型进行剪枝和量化，直至得到压缩好的拓扑识别模型，并将所述压缩好的拓扑识别模型部署到电力物联终端；
    其中，所述对所述剪枝模型进行训练，得到训练好的剪枝模型，包括：
    使用所述待压缩模型，对无标签训练数据集中的训练数据做标注，得到训练数据集；所述训练数据集的训练数据为供电变压器低压台区侧的历史电压电流数据，所述训练数据的标注为历史上该电压电流数据对应的供电变压器台区里各个电表的位置，以及各个电表之间的关联关系；
    使用所述训练数据集，对所述剪枝模型进行训练，得到训练好的剪枝模型。
  2. 根据权利要求1所述的方法，其特征在于，所述对待压缩模型进行剪枝，得到剪枝模型，包括：
    通过稀疏正则化训练，确定所述待压缩模型中的冗余通道，并从所述待压缩模型中移除所述冗余通道，得到所述剪枝模型。
  3. 根据权利要求2所述的方法，其特征在于，所述方法还包括：
    将全连接层作为线性分类器，使用所述线性分类器得到所述待压缩模型中各个神经网络层的识别精度；
    如果所述待压缩模型中任意两个相邻神经网络层的识别精度之间的差值小于预设差值阈值，则将所述任意两个相邻神经网络层中识别精度最低的神经网络层移除。
  4. 根据权利要求3所述的方法，其特征在于，所述将全连接层作为线性分类器，使用所述线性分类器得到所述待压缩模型中各个神经网络层的识别精度，包括：
    从所述待压缩模型中选择任意两个相邻神经网络层，移除所述任意两个相邻神经网络层中的一个神经网络层，得到神经网络层精度预测模型；
    使用带标注的预测数据集，对所述神经网络层精度预测模型进行测试，得到所述神经网络层精度预测模型的输出精度，将所述输出精度作为所述任意两个相邻神经网络层中未被移除的神经网络层的识别精度。
  5. 根据权利要求1所述的方法，其特征在于，所述使用所述待压缩模型，对无标签训练数据 集中的训练数据做标注，得到训练数据集，包括：
    将所述训练数据输入所述待压缩模型，将所述待压缩模型的输出结果作为所述训练数据的标签；
    将所述训练数据与所述训练数据的标签，作为所述训练数据集中的一组训练数据。
  6. 根据权利要求1所述的方法，其特征在于，所述使用所述训练数据集，对所述剪枝模型进行训练，得到训练好的剪枝模型，包括：
    使用损失函数，根据所述剪枝模型的输出与所述训练数据集中训练数据的标签，得到损失值；
    根据所述损失值，判断所述损失函数是否收敛；
    若是，则停止训练，否则，调整所述剪枝模型的参数，并进行下一次训练，直至所述损失函数收敛或者训练次数达到预设训练次数阈值。
  7. 根据权利要求1所述的方法，其特征在于，所述将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，包括；
    将所述剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，并对所述剪枝模型进行训练；
    将所述剪枝模型的全连接层的权重参数的比特位数由第一比特位数量化到第二比特位数，并对所述剪枝模型进行训练；
    对所述剪枝模型的激活函数进行量化，得到所述高量化剪枝模型。
  8. 根据权利要求1所述的方法，其特征在于，所述对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型，包括：
    从所述高量化剪枝模型中选择至少一个神经网络层构建损失函数；
    使用所述剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集；
    根据所述损失函数和所述训练数据集，对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型。
  9. 根据权利要求8所述的方法，其特征在于，所述使用所述剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集，包括：
    将所述训练数据输入所述剪枝模型，将所述剪枝模型中的参照网络层的输出作为所述训练数据的标签；其中，所述参照网络层与所述高量化剪枝模型中被选择的所述至少一个神经网络层对应；
    将所述训练数据与所述训练数据的标签，作为所述训练数据集中的一组训练数据。
  10. 根据权利要求8所述的方法，其特征在于，所述根据所述损失函数和所述训练数据集，对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型，包括：
    使用所述损失函数，根据所述高量化剪枝模型中的所述至少一个神经网络层的输出与所述训练数据集中训练数据的标签，得到损失值；
    根据所述损失值，判断所述损失函数是否收敛；
    若是，则停止训练，否则，调整所述高量化剪枝模型的参数，并进行下一次训练，直至所述损失函数收敛或者训练次数达到预设训练次数阈值。
  11. 根据权利要求10所述的方法，其特征在于，所述方法还包括：
    如果训练次数达到所述预设训练次数阈值，所述损失函数还未收敛，则参照所述剪枝模型中神经网络层的权重参数，将所述高量化剪枝模型中的至少一个权重参数的比特位数由所述第二比特位数增加到所述第一比特位数。
  12. 根据权利要求7所述的方法，其特征在于，所述将所述剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，包括：
    逐次使用预设算法从所述剪枝模型中选择至少一个神经网络层，将所述至少一个神经网络层的权重参数由第一比特位数量化到第二比特位数。
  13. 一种拓扑识别模型的压缩装置，其特征在于，包括：
    剪枝模块，用于对待压缩模型进行剪枝，得到剪枝模型，并对所述剪枝模型进行训练，得到训练好的剪枝模型，如果训练次数达到预设训练次数阈值，损失函数还未收敛，则参照所述待压缩模型，增加所述剪枝模型的通道或者神经网络层；其中，所述待压缩模型为拓扑识别模型，所述拓扑识别模型为运行在服务端的用于识别电网拓扑结构的机器学习模型；
    量化模块，用于将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，并对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型；其中，所述第二比特位数小于所述第一比特位数；对于不满足精度的网络层，通过剪枝模型的权重参数对高量化剪枝模型的权重参数进指导训练；
    部署模块，用于将所述训练好的高量化剪枝模型作为待压缩模型，继续对所述待压缩模型进行剪枝和量化，直至得到压缩好的拓扑识别模型，并将所述压缩好的拓扑识别模型部署到电力物联终端；
    其中，所述对所述剪枝模型进行训练，得到训练好的剪枝模型，包括：
    使用所述待压缩模型，对无标签训练数据集中的训练数据做标注，得到训练数据集；所述训练数据集的训练数据为供电变压器低压台区侧的历史电压电流数据，所述训练数据的标注为历史上该电压电流数据对应的供电变压器台区里各个电表的位置，以及各个电表之间的关联关系；
    使用所述训练数据集，对所述剪枝模型进行训练，得到训练好的剪枝模型。
  14. 根据权利要求13所述的装置，其特征在于，所述对待压缩模型进行剪枝，得到剪枝模型，包括：
    通过稀疏正则化训练，确定所述待压缩模型中的冗余通道，并从所述待压缩模型中移除所述冗余通道，得到所述剪枝模型。
  15. 根据权利要求14所述的装置，其特征在于，所述装置还包括：
    网络层识别精度获取模块，用于将全连接层作为线性分类器，使用所述线性分类器得到所述待压缩模型中各个神经网络层的识别精度；
    移除模块，用于如果所述待压缩模型中任意两个相邻神经网络层的识别精度之间的差值小于预设差值阈值，则将所述任意两个相邻神经网络层中识别精度最低的神经网络层移除。
  16. 根据权利要求15所述的装置，其特征在于，所述将全连接层作为线性分类器，使用所述线性分类器得到所述待压缩模型中各个神经网络层的识别精度，包括：
    从所述待压缩模型中选择任意两个相邻神经网络层，移除所述任意两个相邻神经网络层中的一个神经网络层，得到神经网络层精度预测模型；
    使用带标注的预测数据集，对所述神经网络层精度预测模型进行测试，得到所述神经网络层精度预测模型的输出精度，将所述输出精度作为所述任意两个相邻神经网络层中未被移除的神经网络层的识别精度。
  17. 根据权利要求13所述的装置，其特征在于，所述使用所述待压缩模型，对无标签训练数据集中的训练数据做标注，得到训练数据集，包括：
    将所述训练数据输入所述待压缩模型，将所述待压缩模型的输出结果作为所述训练数据的标签；
    将所述训练数据与所述训练数据的标签，作为所述训练数据集中的一组训练数据。
  18. 根据权利要求13所述的装置，其特征在于，所述使用所述训练数据集，对所述剪枝模型进行训练，得到训练好的剪枝模型，包括：
    使用损失函数，根据所述剪枝模型的输出与所述训练数据集中训练数据的标签，得到损失值；
    根据所述损失值，判断所述损失函数是否收敛；
    若是，则停止训练，否则，调整所述剪枝模型的参数，并进行下一次训练，直至所述损失函数收敛或者训练次数达到预设训练次数阈值。
  19. 根据权利要求13所述的装置，其特征在于，所述将所述训练好的剪枝模型中各神经网络层的权重参数的比特位数由第一比特位数量化到第二比特位数，得到高量化剪枝模型，包括；
    将所述剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，并对所述剪枝模型进行训练；
    将所述剪枝模型的全连接层的权重参数的比特位数由第一比特位数量化到第二比特位数，并对所述剪枝模型进行训练；
    对所述剪枝模型的激活函数进行量化，得到所述高量化剪枝模型。
  20. 根据权利要求13所述的装置，其特征在于，所述对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型，包括：
    从所述高量化剪枝模型中选择至少一个神经网络层构建损失函数；
    使用所述剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集；
    根据所述损失函数和所述训练数据集，对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型。
  21. 根据权利要求20所述的装置，其特征在于，所述使用所述剪枝模型，对无标签训练数据集中的训练数据做标注，得到训练数据集，包括：
    将所述训练数据输入所述剪枝模型，将所述剪枝模型中的参照网络层的输出作为所述训练数据的标签；其中，所述参照网络层与所述高量化剪枝模型中被选择的所述至少一个神经网络层对应；
    将所述训练数据与所述训练数据的标签，作为所述训练数据集中的一组训练数据。
  22. 根据权利要求20所述的装置，其特征在于，所述根据所述损失函数和所述训练数据集，对所述高量化剪枝模型进行训练，得到训练好的高量化剪枝模型，包括：
    使用所述损失函数，根据所述高量化剪枝模型中的所述至少一个神经网络层的输出与所述训练数据集中训练数据的标签，得到损失值；
    根据所述损失值，判断所述损失函数是否收敛；
    若是，则停止训练，否则，调整所述高量化剪枝模型的参数，并进行下一次训练，直至所述损失函数收敛或者训练次数达到预设训练次数阈值。
  23. 根据权利要求22所述的装置，其特征在于，所述装置还包括：
    反量化模块，用于如果训练次数达到所述预设训练次数阈值，所述损失函数还未收敛，则参照所述剪枝模型中神经网络层的权重参数，将所述高量化剪枝模型中的至少一个权重参数的比特位数由所述第二比特位数增加到所述第一比特位数。
  24. 根据权利要求19所述的装置，其特征在于，所述将所述剪枝模型中的各个神经网络层的权重参数由第一比特位数量化到第二比特位数，包括：
    逐次使用预设算法从所述剪枝模型中选择至少一个神经网络层，将所述至少一个神经网络层的权重参数由第一比特位数量化到第二比特位数。
  25. 一种芯片，其特征在于，
    所述芯片包括如权利要求13-24中任一项所述的装置。
  26. 一种电子设备，其特征在于，包括存储器和处理器；其中，所述存储器用于存储一条或多条计算机指令，其中，所述一条或多条计算机指令被所述处理器执行以实现权利要求1-12任一项所述的方法步骤。
  27. 一种计算机可读存储介质，其上存储有计算机指令，其特征在于，该计算机指令被处理器执行时实现权利要求1-12任一项所述的方法步骤。
PCT/CN2023/111880 2022-08-17 2023-08-09 拓扑识别模型的压缩方法、装置、电子设备及介质 WO2024037393A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210983746.3 2022-08-17
CN202210983746.3A CN115049058B (zh) 2022-08-17 2022-08-17 拓扑识别模型的压缩方法、装置、电子设备及介质

Publications (1)

Publication Number Publication Date
WO2024037393A1 true WO2024037393A1 (zh) 2024-02-22

Family

ID=83166498

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/111880 WO2024037393A1 (zh) 2022-08-17 2023-08-09 拓扑识别模型的压缩方法、装置、电子设备及介质

Country Status (2)

Country Link
CN (1) CN115049058B (zh)
WO (1) WO2024037393A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118014040A (zh) * 2024-03-22 2024-05-10 上海大学 一种基于动态稀疏正则化的神经网络剪枝方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115049058B (zh) * 2022-08-17 2023-01-20 北京智芯微电子科技有限公司 拓扑识别模型的压缩方法、装置、电子设备及介质
CN116894189B (zh) * 2023-09-11 2024-01-05 中移(苏州)软件技术有限公司 一种模型训练方法、装置、设备及可读存储介质
CN117910536B (zh) * 2024-03-19 2024-06-07 浪潮电子信息产业股份有限公司 文本生成方法及其模型梯度剪枝方法、装置、设备、介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108615049A (zh) * 2018-04-09 2018-10-02 华中科技大学 一种车辆部件检测模型压缩方法及系统
CN112698123A (zh) * 2020-12-01 2021-04-23 国网河南省电力公司电力科学研究院 一种基于决策树的低压台区用户拓扑关系识别方法
CN113033897A (zh) * 2021-03-26 2021-06-25 国网上海市电力公司 基于用户分支的电量相关性识别台区户变关系的方法
WO2021189906A1 (zh) * 2020-10-20 2021-09-30 平安科技（深圳）有限公司 基于联邦学习的目标检测方法、装置、设备及存储介质
WO2022160856A1 (zh) * 2021-01-27 2022-08-04 歌尔股份有限公司 一种分类网络及其实现方法和装置
CN115049058A (zh) * 2022-08-17 2022-09-13 北京智芯微电子科技有限公司 拓扑识别模型的压缩方法、装置、电子设备及介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635936A (zh) * 2018-12-29 2019-04-16 杭州国芯科技股份有限公司 一种基于重训练的神经网络剪枝量化方法
CN111160524A (zh) * 2019-12-16 2020-05-15 北京时代民芯科技有限公司 一种两阶段的卷积神经网络模型压缩方法
CN111275187A (zh) * 2020-01-16 2020-06-12 北京智芯微电子科技有限公司 深度神经网络模型的压缩方法及装置
CN112329922A (zh) * 2020-11-24 2021-02-05 北京大学 一种基于质谱数据集的神经网络模型压缩方法及系统
CN114819143A (zh) * 2022-04-15 2022-07-29 北京邮电大学 一种适用于通信网现场维护的模型压缩方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108615049A (zh) * 2018-04-09 2018-10-02 华中科技大学 一种车辆部件检测模型压缩方法及系统
WO2021189906A1 (zh) * 2020-10-20 2021-09-30 平安科技（深圳）有限公司 基于联邦学习的目标检测方法、装置、设备及存储介质
CN112698123A (zh) * 2020-12-01 2021-04-23 国网河南省电力公司电力科学研究院 一种基于决策树的低压台区用户拓扑关系识别方法
WO2022160856A1 (zh) * 2021-01-27 2022-08-04 歌尔股份有限公司 一种分类网络及其实现方法和装置
CN113033897A (zh) * 2021-03-26 2021-06-25 国网上海市电力公司 基于用户分支的电量相关性识别台区户变关系的方法
CN115049058A (zh) * 2022-08-17 2022-09-13 北京智芯微电子科技有限公司 拓扑识别模型的压缩方法、装置、电子设备及介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118014040A (zh) * 2024-03-22 2024-05-10 上海大学 一种基于动态稀疏正则化的神经网络剪枝方法

Also Published As

Publication number Publication date
CN115049058A (zh) 2022-09-13
CN115049058B (zh) 2023-01-20

Similar Documents

Publication Publication Date Title
WO2024037393A1 (zh) 拓扑识别模型的压缩方法、装置、电子设备及介质
CN110533183B (zh) 流水线分布式深度学习中异构网络感知的任务放置方法
WO2022083624A1 (zh) 一种模型的获取方法及设备
CN113361680B (zh) 一种神经网络架构搜索方法、装置、设备及介质
CN110417011B (zh) 一种基于互信息与迭代随机森林的在线动态安全评估方法
US20210064978A1 (en) Information processing device, information processing method, and storage medium
CN111553510A (zh) 一种短期风速预测方法
CN115225516B (zh) 基于改进abc-vmd的lssvm网络流量预测方法
CN115131560A (zh) 基于全局特征学习和局部特征判别聚合的点云分割方法
CN111313403A (zh) 一种基于马尔可夫随机场的低压配电系统网络拓扑识别方法
CN110008914A (zh) 一种基于神经网络的模式识别系统及识别方法
CN110610140B (zh) 人脸识别模型的训练方法、装置、设备及可读存储介质
CN113535515A (zh) 一种基于Zabbix监控系统的高效动态采集方法
CN112308298A (zh) 一种面向半导体生产线的多场景性能指标预测方法及系统
CN108808657B (zh) 一种电力负荷短期预测方法
CN111935762B (zh) 一种5g承载网下基于ewt和cnn的配网故障诊断方法和系统
CN113886454A (zh) 一种基于lstm-rbf的云资源预测方法
CN109065176A (zh) 一种血糖预测方法、装置、终端和存储介质
CN117201308A (zh) 网络资源分配方法、系统、存储介质及电子设备
CN116820762A (zh) 一种基于电力边缘芯片的边云协同计算方法
CN115907000A (zh) 一种用于电力系统最优潮流预测的小样本学习方法
CN114912854B (zh) 地铁列车运行调整方法、装置、电子设备及存储介质
CN113627655B (zh) 一种配电网灾前故障场景模拟预测方法及装置
CN113158134B (zh) 非侵入式负荷辨识模型的构建方法、装置和存储介质
CN115081609A (zh) 一种智能决策中的加速方法、终端设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23854296

Country of ref document: EP

Kind code of ref document: A1