CN107590534B - Method and device for training deep convolutional neural network model and storage medium - Google Patents
Method and device for training deep convolutional neural network model and storage medium Download PDFInfo
- Publication number
- CN107590534B CN107590534B CN201710977989.5A CN201710977989A CN107590534B CN 107590534 B CN107590534 B CN 107590534B CN 201710977989 A CN201710977989 A CN 201710977989A CN 107590534 B CN107590534 B CN 107590534B
- Authority
- CN
- China
- Prior art keywords
- nodes
- target node
- neural network
- training
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The present disclosure relates to a method, an apparatus, and a storage medium for training a deep convolutional neural network model, wherein the method comprises: in the process of training the deep convolutional neural network model through a gradient back propagation algorithm, when the output value of a target node included in a target layer in the deep convolutional neural network model is used as the input value of N nodes included in the next layer, N auxiliary nodes are added between the target node and the N nodes; the target layer is any layer included in the deep convolutional neural network, the target node is any node included in the target layer, and N is greater than 1 and less than or equal to the total number of nodes included in the next layer; and training the weight between the target node and the N nodes through the N auxiliary nodes. According to the method, the auxiliary node is added, a special dynamic application of a memory space for the intermediate calculation result is not needed, the gradient back propagation algorithm is enabled to be concise and efficient, and the training process of the deep convolutional neural network model is greatly accelerated.
Description
Technical Field
The present disclosure relates to the field of network technologies, and in particular, to a method and an apparatus for training a deep convolutional neural network model, and a storage medium.
Background
The deep convolutional neural network model is a network model capable of efficiently performing image recognition. That is, the processing can be performed through the convolution layer, the activation layer, the pooling layer and the full-link layer included in the deep convolutional neural network model, and finally, the final image recognition result is directly output through the category probability layer included in the deep convolutional neural network model, so that the complex pre-processing of the image is avoided. However, before the image is identified using the deep convolutional neural network model, the deep convolutional neural network model also needs to be trained.
Because each layer in the deep convolutional neural network model comprises a plurality of nodes, the training of the deep convolutional neural network model refers to the training of weights between two adjacent layers of nodes in the deep convolutional neural network model, so that a group of weights which enable the image recognition rate to be high is determined. Therefore, a deep convolutional neural network model training method capable of improving the image recognition rate is needed.
Disclosure of Invention
To overcome the problems in the related art, the present disclosure provides a method, an apparatus, and a storage medium for training a deep convolutional neural network model.
According to a first aspect of embodiments of the present disclosure, there is provided a method of training a deep convolutional neural network model, the method comprising:
in the process of training a deep convolutional neural network model through a gradient back propagation algorithm, when an output value of a target node included in a target layer in the deep convolutional neural network model is used as an input value of N nodes included in a next layer, adding N auxiliary nodes between the target node and the N nodes;
the target layer is any layer included in the deep convolutional neural network, the target node is any node included in the target layer, and the N is greater than 1 and less than or equal to the total number of nodes included in the next layer;
and training the weight between the target node and the N nodes through the N auxiliary nodes.
Optionally, the training, by the N auxiliary nodes, weights between the target node and the N nodes includes:
calculating gradients of the N nodes relative to the target node;
the gradients of the N nodes relative to the target node are correspondingly multiplied by the gradients of a loss function relative to the N nodes respectively, wherein the loss function is a composite function taking the weight between the nodes of two adjacent layers in the deep convolutional neural network model as an argument;
when an intermediate calculation result is obtained by multiplying each time, storing the intermediate calculation result obtained by calculation into the corresponding auxiliary node;
when the N auxiliary nodes have intermediate calculation results stored, transmitting the intermediate calculation results stored in the N auxiliary nodes to the target node;
and training the weights between the target node and the N nodes based on the intermediate calculation results stored by the N auxiliary nodes.
Optionally, the training the weights between the target node and the N nodes based on the intermediate calculation results stored by the N auxiliary nodes includes:
adding the intermediate calculation results stored in the N auxiliary nodes in the target node to obtain the gradient of the loss function relative to the target node;
training weights between the target node and the N nodes based on a gradient of the loss function relative to the target node.
Optionally, before the multiplying the gradients of the N nodes with respect to the target node by the gradient correspondences of the loss function with respect to the N nodes, respectively, the method further includes:
and acquiring gradients of the loss function relative to the N nodes from the N nodes respectively.
Optionally, after adding the intermediate calculation results stored in the N auxiliary nodes in the target node to obtain a gradient of the loss function with respect to the target node, the method further includes:
storing the gradient of the loss function relative to the target node into the target node.
According to a second aspect of embodiments of the present disclosure, there is provided an apparatus for training a deep convolutional neural network model, the apparatus comprising:
the adding module is used for adding N auxiliary nodes between a target node and N nodes when the output value of the target node included in a target layer in the deep convolutional neural network model is used as the input value of the N nodes included in the next layer in the process of training the deep convolutional neural network model through a gradient back propagation algorithm;
the target layer is any layer included in the deep convolutional neural network, the target node is any node included in the target layer, and the N is greater than 1 and less than or equal to the total number of nodes included in the next layer;
and the training module is used for training the weight between the target node and the N nodes through the N auxiliary nodes.
Optionally, the training module comprises:
a computation submodule for computing gradients of the N nodes relative to the target node;
the multiplication submodule is used for correspondingly multiplying the gradients of the N nodes relative to the target node with the gradients of a loss function relative to the N nodes, wherein the loss function is a composite function taking the weight between the nodes of two adjacent layers in the deep convolutional neural network model as an independent variable;
the storage submodule is used for storing the intermediate calculation result obtained by calculation into the corresponding auxiliary node when each time an intermediate calculation result is obtained by multiplication;
the transmission submodule is used for transmitting the intermediate calculation results stored in the N auxiliary nodes to the target node when the intermediate calculation results are stored in the N auxiliary nodes;
and the training submodule is used for training the weights between the target node and the N nodes based on the intermediate calculation results stored by the N auxiliary nodes.
Optionally, the training submodule is mainly configured to:
adding the intermediate calculation results stored in the N auxiliary nodes in the target node to obtain the gradient of the loss function relative to the target node;
training weights between the target node and the N nodes based on a gradient of the loss function relative to the target node.
Optionally, the training module further comprises:
and the acquisition submodule is used for respectively acquiring the gradients of the loss function relative to the N nodes from the N nodes.
Optionally, the training sub-module is further configured to:
storing the gradient of the loss function relative to the target node into the target node.
According to a third aspect of embodiments of the present disclosure, there is provided an apparatus for training a deep convolutional neural network model, the apparatus comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform the steps of the method of the first aspect.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon instructions which, when executed by a processor, implement the steps of the method of the first aspect described above.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
in the embodiment of the disclosure, when the deep convolutional neural network model is trained through the gradient back propagation algorithm, if an output value of a target node included in a target layer in the deep convolutional neural network model is used as an input value of N nodes included in a next layer, gradients of the N nodes need to be transmitted to the target node during the gradient back propagation, so that N auxiliary nodes can be added between the target node and the N nodes, and weights between the target node and the N nodes are trained through the N auxiliary nodes. Therefore, by adding the auxiliary nodes, a memory space does not need to be dynamically applied for the intermediate calculation result of the gradient of the N nodes, so that the gradient back propagation algorithm becomes simple and efficient, and the training process of the deep convolutional neural network model is greatly accelerated.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
FIG. 1 is a flow diagram illustrating a method of training a deep convolutional neural network model in accordance with an exemplary embodiment.
FIG. 2A is a flow diagram illustrating another method of training a deep convolutional neural network model in accordance with an exemplary embodiment.
FIG. 2B is a block diagram illustrating a deep convolutional network model according to an exemplary embodiment.
Fig. 2C is a schematic structural diagram illustrating the addition of N auxiliary nodes between a target node and N nodes included in a next layer according to an example embodiment.
FIG. 3A is a block diagram illustrating an apparatus for training a deep convolutional neural network model in accordance with an exemplary embodiment.
FIG. 3B is a block diagram illustrating a training module 302 according to an example embodiment.
FIG. 3C is a block diagram illustrating another training module 302 according to an example embodiment.
FIG. 4 is a block diagram illustrating an apparatus 400 for training a deep convolutional neural network model in accordance with an exemplary embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
For convenience of understanding, before explaining the embodiments of the present disclosure in detail, an application scenario related to the embodiments of the present disclosure will be described.
In practical application, when an image needs to be identified, the image can be input into a deep convolutional neural network model, and an image identification result is output through the deep convolutional neural network model. That is, the processing can be performed through the convolutional layer, the activation layer, the pooling layer and the full-link layer included in the deep convolutional neural network model, and finally, the final image recognition result is directly output through the class probability layer included in the deep convolutional neural network model. For example, an animal image is input into a deep convolutional neural network model, and is processed by a convolutional layer, an activation layer, a pooling layer and a full link layer, and the class probability layer outputs that the image is a dog with a probability of 0.9 and the image is a cat with a probability of 0.1.
In order to improve the accuracy of the deep convolutional neural network model for image recognition, the deep convolutional neural network model needs to be trained before image recognition. In the related art, a gradient back propagation algorithm is adopted to train the deep convolutional neural network model, and because the intermediate calculation results of the gradients of the N nodes included in the next layer need to be transmitted to the target node in order to obtain the gradient of the target node in the gradient back propagation algorithm, a memory space needs to be dynamically applied for the intermediate calculation results of the gradients of the N nodes, so that the operation process is complicated, and the deep convolutional neural network model is not beneficial to being trained. Accordingly, to address the above-mentioned problems, the present disclosure provides a method of training a deep convolutional neural network model.
The method for training the deep convolutional neural network model provided by the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.
FIG. 1 is a flowchart illustrating a method of training a deep convolutional neural network model, as shown in FIG. 1, including the following steps, in accordance with an exemplary embodiment.
In step 101, in the process of training the deep convolutional neural network model by the gradient back propagation algorithm, when an output value of a target node included in a target layer in the deep convolutional neural network model is used as an input value of N nodes included in a next layer, N auxiliary nodes are added between the target node and the N nodes.
The target layer is any layer included in the deep convolutional neural network, the target node is any node included in the target layer, and N is greater than 1 and less than or equal to the total number of nodes included in the next layer.
In step 102, weights between the target node and the N nodes are trained by the N auxiliary nodes.
In the embodiment of the disclosure, when the deep convolutional neural network model is trained through the gradient back propagation algorithm, if an output value of a target node included in a target layer in the deep convolutional neural network model is used as an input value of N nodes included in a next layer, gradients of the N nodes need to be transmitted to the target node during the gradient back propagation, so that N auxiliary nodes can be added between the target node and the N nodes, and weights between the target node and the N nodes are trained through the N auxiliary nodes. Therefore, by adding the auxiliary nodes, a memory space does not need to be dynamically applied for the intermediate calculation result of the gradient of the N nodes, so that the gradient back propagation algorithm becomes simple and efficient, and the training process of the deep convolutional neural network model is greatly accelerated.
Optionally, training the weight between the target node and the N nodes through N auxiliary nodes includes:
calculating gradients of the N nodes relative to a target node;
the gradients of the N nodes relative to the target node are respectively multiplied by the gradients of a loss function relative to the N nodes correspondingly, wherein the loss function is a composite function taking the weight between the nodes of two adjacent layers in the deep convolutional neural network model as an independent variable;
when an intermediate calculation result is obtained by multiplying each time, storing the intermediate calculation result obtained by calculation into the corresponding auxiliary node;
when the intermediate calculation results are stored in the N auxiliary nodes, the intermediate calculation results stored in the N auxiliary nodes are transmitted to the target node;
and training the weight between the target node and the N nodes based on the intermediate calculation results stored by the N auxiliary nodes.
Optionally, training the weights between the target node and the N nodes based on the intermediate calculation results stored by the N auxiliary nodes includes:
adding the intermediate calculation results stored in the N auxiliary nodes in the target node to obtain the gradient of the loss function relative to the target node;
and training the weights between the target node and the N nodes based on the gradient of the loss function relative to the target node.
Optionally, before multiplying the gradients of the N nodes with respect to the target node by the gradients of the loss functions with respect to the N nodes, respectively, the method further includes:
and acquiring gradients of the loss function relative to the N nodes from the N nodes respectively.
Optionally, after adding the intermediate calculation results stored in the N auxiliary nodes in the target node to obtain a gradient of the loss function with respect to the target node, the method further includes:
storing the gradient of the loss function relative to the target node into the target node.
All the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present disclosure, and the embodiments of the present disclosure are not described in detail again.
Fig. 2A is a flowchart illustrating a method of training a deep convolutional neural network model according to an exemplary embodiment, and the embodiment illustrated in fig. 1 will be expanded by the embodiments of the present disclosure. As shown in fig. 2A, the method includes the following steps.
In step 201, in the process of training the deep convolutional neural network model by the gradient back propagation algorithm, when an output value of a target node included in a target layer in the deep convolutional neural network model is used as an input value of N nodes included in a next layer, N auxiliary nodes are added between the target node and the N nodes.
The target layer is any layer included in the deep convolutional neural network, the target node is any node included in the target layer, and N is greater than 1 and less than or equal to the total number of nodes included in the next layer.
Before the image is identified by using the deep convolutional neural network model, weights between two adjacent layers of nodes in the deep convolutional neural network model need to be trained, the gradient back propagation algorithm can transmit gradients of the N nodes included in each layer back to a target node of the previous layer, and the target node trains the weights between the target node and the N nodes based on the acquired gradients. In the process of training the deep convolutional neural network model by the gradient back propagation algorithm, since the output value of the target node may be used as the input value of the N nodes included in the next layer, in the gradient back propagation, the intermediate calculation results of the gradients of the N nodes need to be transmitted to the target node. Since the intermediate calculation results of the gradients of each of the N nodes are obtained by calculation one by one, but cannot be obtained at the same time, it is necessary to store the intermediate calculation results of the gradients of the N nodes first, and in order to store the intermediate calculation results of the gradients of the N nodes, an auxiliary node may be added between the target node and the N nodes included in the next layer.
For example, fig. 2B is a schematic structural diagram of a deep convolutional network model, as shown in fig. 2B, a target layer includes 3 nodes x1, x2, and x3, and a next layer includes 3 nodes a1, a2, and a3, assuming that a node x1 in the target layer is a target node, an output value of the target node x1 is used as an input value of nodes a1, a2, and a3 included in the next layer, and therefore, intermediate calculation results of gradients of the nodes a1, a2, and a3 all need to be transferred to the target node x 1. Fig. 2C is a schematic structural diagram of adding N auxiliary nodes between a target node and N nodes included in a next layer, as shown in fig. 2C, the target node is x1, the nodes included in the next layer are a1, a2 and a3, and auxiliary nodes x11, x12 and x13 may be respectively added between the target node x1 and the nodes a1, a2 and a 3.
It should be noted that after N auxiliary nodes are added between the target node and N nodes included in the next layer, in order to train the deep convolutional neural network model, a loss function needs to be artificially defined, and when the function value of the loss function is minimum, a corresponding set of weights are weights that make the image recognition rate higher. The loss function is a composite function taking the weight between the nodes of two adjacent layers in the deep convolutional neural network model as an argument. In order to determine a set of weights that minimizes the function value of the loss function, the weights between the target node and the N nodes may be trained by the N auxiliary nodes according to the following step 202 and 207.
In step 202, gradients of the N nodes with respect to the target node are calculated.
In a possible implementation manner, since the deep convolutional neural network model includes multiple layers, and when performing image recognition, data needs to be transferred among the multiple layers for processing, so that for a target node in a target layer, a certain functional relationship exists between the target node and the N nodes in the next layer. Therefore, when calculating the gradients of the N nodes with respect to the target node, the partial derivatives can be respectively calculated for the variables in the function between the output value of the target node and the output values of the N nodes, so that the gradients of the N nodes with respect to the target node can be obtained.
For example, as shown in fig. 2B, the target node is x1, and the next level includes N nodes a1, a2 and a3, respectively, assuming that x is x1＝k1a1+k2a2+k3a3Wherein x is1Is the output value of node x1, a1、a2、a3Is the output value, k, of the nodes a1, a2, a31、k2、k3The gradient of node a1 relative to node x1 is a pair function x, which is a weight between node x1 and nodes a1, a2, a31＝k1a1+k2a2+k3a3A in (a)1Derivation, i.e., grad (a1, x1) ═ k1The gradient of the node a2 with respect to the node x1 is a pair function x1＝k1a1+k2a2+k3a3A in (a)2Derivation, i.e., grad (a2, x1) ═ k2The gradient of the node a3 with respect to the node x1 is a pair function x1＝k1a1+k2a2+k3a3A in (a)3Derivation, i.e., grad (a3, x1) ═ k3。
Further, after the gradients of the N nodes with respect to the target node are calculated, the gradients of the loss function with respect to the N nodes may be obtained from the N nodes, respectively. Wherein, the gradient of the loss function relative to the N nodes may be determined in the process of determining weight training between the nodes included in the next layer of the target layer and the nodes included in the next layer of the target layer, and stored in the N nodes. In this way, when the weights between the node included in the target layer and the node included in the next layer of the target layer need to be trained, the weights can be directly obtained from the N nodes respectively. However, when the N nodes are nodes included in the class probability layer, the gradient of the loss function with respect to the N nodes may be directly calculated and stored in the N nodes. That is, the partial derivatives are calculated for the variables corresponding to the N nodes in the loss function.
In step 203, the gradients of the N nodes relative to the target node are multiplied by the gradient correspondences of the penalty function relative to the N nodes, respectively.
For example, as shown in fig. 2B, the gradients of the nodes a1, a2, a3 with respect to the target node x1 are grad (a1, x1), grad (a2, x1), grad (a3, x1), the gradients of the loss functions with respect to the nodes a1, a2, a3 are d _ a1, d _ a2, d _ a3, the grad (a1, x1) is multiplied by d _ a1, the grad (a2, x1) is multiplied by d _ a2, the grad (a3, x1) is multiplied by d _ a1, d _ a2, d _ a3, respectively.
In step 204, when each multiplication results in an intermediate calculation result, the calculated intermediate calculation result is stored in the corresponding auxiliary node.
And multiplying the gradient of one of the N nodes included in the next layer relative to the target node by the gradient of the loss function relative to the node correspondingly to obtain an intermediate calculation result, and storing the obtained intermediate calculation result into the corresponding auxiliary node.
For example, as shown in fig. 2C, the gradient grad (a1, x1) of the node a1 with respect to the x1 in the next layer is multiplied by the gradient d _ a1 of the loss function with respect to the node a1 to obtain an intermediate calculation result grad (a1, x1) × d _ a1, and the intermediate calculation result grad (a1, x1) × d _ a1 is stored in the node x 11.
In step 205, when the intermediate calculation results have been stored in all of the N auxiliary nodes, the intermediate calculation results stored in the N auxiliary nodes are transmitted to the target node.
For example, as shown in fig. 2C, when the auxiliary node x11 stores the intermediate calculation result grad (a1, x1) × d _ a1, the node x12 stores the intermediate calculation result grad (a2, x1) × d _ a2, and the node x13 stores the intermediate calculation result grad (a3, x1) × d _ a3, the 3 intermediate calculation results stored in the auxiliary nodes x11, x12, and x13 are transmitted to the target node in parallel.
It should be noted that, after the intermediate calculation results stored in the N auxiliary nodes are transmitted to the target node, the weights between the target node and the N nodes may be trained according to the following steps 206-. Of course, there may be other implementation manners to train the weight between the target node and the N nodes in practical application, and this is not limited in this disclosure.
In step 206, the intermediate calculation results stored in the N auxiliary nodes are added in the target node to obtain the gradient of the loss function with respect to the target node.
When the target node receives the N intermediate calculation results transmitted by the N auxiliary nodes, the N intermediate calculation results are added in the target node, and then the gradient of the loss function with respect to the target node is obtained. Further, in order to facilitate training of weights between the nodes included in the target layer and the nodes included in the previous layer of the target layer, after the gradient of the loss function relative to the target node is calculated, the gradient of the loss function relative to the target node may be stored in the target node.
For example, as shown in fig. 2C, the 3 intermediate calculation results that the auxiliary nodes x11, x12, and x13 transfer to the target node x1 are grad (a1, x1) × d _ a1, grad (a2, x1) × d _ a2, and grad (a3, x1) × d _ a3, and in the target node x1, the 3 intermediate calculation results are added to obtain the gradient of the loss function with respect to the target node x1, which is d _ x1 ═ grad (a1, x1) × d _ a1+ grad (a2, x1) × d _ a2+ grad (a3, x1) × d _ a3, and the gradient of the loss function with respect to the target node x1, which is d _ x1, is stored in the target node x 1.
In step 207, weights between the target node and the N nodes are trained based on the gradient of the loss function with respect to the target node.
Because the default initial value of the weight between the target node and the N nodes is set before the weight between the target node and the N nodes in the target layer in the deep convolutional neural network is trained through the gradient back propagation algorithm, when the weight between the target node and the N nodes is trained based on the gradient of the loss function relative to the target node, the default initial value of the set weight and the gradient of the loss function relative to the target node can be added in the gradient direction, and thus the adjustment of the weight between the target node and the N nodes is realized.
In the embodiment of the disclosure, when a deep convolutional neural network model is trained through a gradient back propagation algorithm, if an output value of a target node included in a target layer in the deep convolutional neural network model is used as an input value of N nodes included in a next layer, when the deep convolutional neural network model is propagated backward in a gradient, intermediate computation results of gradients of the N nodes need to be transmitted to the target node, so N auxiliary nodes may be added between the target node and the N nodes, a weight between the target node and the N nodes is trained through the N auxiliary nodes, that is, the intermediate computation results of the gradients of the N nodes may be stored in the corresponding auxiliary nodes, when each auxiliary node stores an intermediate computation result, the intermediate computation result stored in each auxiliary node is transmitted to the target node in parallel, and then the weight between the target node and the N nodes is trained based on the intermediate computation results transmitted by the N auxiliary nodes And (5) refining. Therefore, by adopting the auxiliary node mode, a memory space does not need to be dynamically applied for the intermediate calculation result of the gradient of the N nodes, so that the gradient back propagation algorithm becomes simple and efficient, and the training process of the deep convolutional neural network model is greatly accelerated.
After the method provided by the embodiment of the present disclosure is explained in detail by the embodiment shown in fig. 1 and fig. 2A, the apparatus for training the convolutional neural network model provided by the implementation of the present disclosure will be described next.
FIG. 3A is a block diagram illustrating an apparatus for training a deep convolutional neural network model, according to an example embodiment. Referring to fig. 3A, the apparatus includes an adding module 301 and a training module 302.
An adding module 301, configured to, in a process of training a deep convolutional neural network model through a gradient back propagation algorithm, add N auxiliary nodes between a target node and N nodes when an output value of the target node included in a target layer in the deep convolutional neural network model is used as an input value of the N nodes included in a next layer;
the target layer is any layer included in the deep convolutional neural network, the target node is any node included in the target layer, and N is greater than 1 and less than or equal to the total number of nodes included in the next layer;
a training module 302, configured to train a weight between the target node and the N nodes through the N auxiliary nodes.
Optionally, referring to fig. 3B, the training module 302 includes:
a calculation submodule 3021 configured to calculate gradients of the N nodes with respect to the target node;
a multiplication submodule 3022, configured to multiply the gradients of the N nodes with respect to the target node, respectively, by the gradients of the loss function with respect to the N nodes, where the loss function is a complex function in which a weight between two adjacent layers of nodes in the deep convolutional neural network model is used as an argument;
a storage submodule 3023, configured to, when an intermediate calculation result is obtained by multiplying, store the calculated intermediate calculation result in the corresponding auxiliary node;
the transfer submodule 3024 is configured to, when intermediate calculation results have been stored in each of the N auxiliary nodes, transfer the intermediate calculation results stored in the N auxiliary nodes to the target node;
the training submodule 3025 is configured to train weights between the target node and the N nodes based on the intermediate calculation results stored in the N auxiliary nodes.
Optionally, the training submodule 3025 is specifically configured to:
adding the intermediate calculation results stored in the N auxiliary nodes in the target node to obtain the gradient of the loss function relative to the target node;
and training the weights between the target node and the N nodes based on the gradient of the loss function relative to the target node.
Optionally, referring to fig. 3C, the training module 302 further includes:
an obtaining submodule 3026 is configured to obtain gradients of the loss function with respect to the N nodes from the N nodes, respectively.
Optionally, the training submodule 3025 is further specifically configured to:
storing the gradient of the loss function relative to the target node into the target node.
In the embodiment of the disclosure, when a deep convolutional neural network model is trained through a gradient back propagation algorithm, if an output value of a target node included in a target layer in the deep convolutional neural network model is used as an input value of N nodes included in a next layer, when the deep convolutional neural network model is propagated backward in a gradient, intermediate computation results of gradients of the N nodes need to be transmitted to the target node, so N auxiliary nodes may be added between the target node and the N nodes, a weight between the target node and the N nodes is trained through the N auxiliary nodes, that is, the intermediate computation results of the gradients of the N nodes may be stored in the corresponding auxiliary nodes, when each auxiliary node stores an intermediate computation result, the intermediate computation result stored in each auxiliary node is transmitted to the target node in parallel, and then the weight between the target node and the N nodes is trained based on the intermediate computation results transmitted by the N auxiliary nodes And (5) refining. Therefore, by adopting the auxiliary node mode, a memory space does not need to be dynamically applied for the intermediate calculation result of the gradient of the N nodes, so that the gradient back propagation algorithm becomes simple and efficient, and the training process of the deep convolutional neural network model is greatly accelerated.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
FIG. 4 is a block diagram illustrating an apparatus 400 for training a deep convolutional neural network model in accordance with an exemplary embodiment. For example, the apparatus 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 4, the apparatus 400 may include one or more of the following components: processing components 402, memory 404, power components 406, multimedia components 408, audio components 410, input/output (I/O) interfaces 412, sensor components 414, and communication components 416.
The processing component 402 generally controls overall operation of the apparatus 400, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 402 can include one or more modules that facilitate interaction between the processing component 402 and other components. For example, the processing component 402 can include a multimedia module to facilitate interaction between the multimedia component 408 and the processing component 402.
The memory 404 is configured to store various types of data to support operations at the apparatus 400. Examples of such data include instructions for any application or method operating on the device 400, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 404 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The multimedia component 408 includes a screen that provides an output interface between the device 400 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the apparatus 400 is in an operation mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 410 is configured to output and/or input audio signals. For example, audio component 410 includes a Microphone (MIC) configured to receive external audio signals when apparatus 400 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 404 or transmitted via the communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.
The I/O interface 412 provides an interface between the processing component 402 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor component 414 includes one or more sensors for providing various aspects of status assessment for the apparatus 400. For example, the sensor assembly 414 may detect an open/closed state of the apparatus 400, the relative positioning of the components, such as a display and keypad of the apparatus 400, the sensor assembly 414 may also detect a change in the position of the apparatus 400 or a component of the apparatus 400, the presence or absence of user contact with the apparatus 400, orientation or acceleration/deceleration of the apparatus 400, and a change in the temperature of the apparatus 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 416 is configured to facilitate wired or wireless communication between the apparatus 400 and other devices. The apparatus 400 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 400 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the methods provided by the embodiments illustrated in fig. 1 and 2A and described above.
In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 404 comprising instructions, executable by the processor 420 of the apparatus 400 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
A non-transitory computer readable storage medium having instructions therein which, when executed by a processor of a mobile terminal, enable the mobile terminal to perform a method of training a deep convolutional neural network model, the method comprising:
in the process of training the deep convolutional neural network model through a gradient back propagation algorithm, when the output value of a target node included in a target layer in the deep convolutional neural network model is used as the input value of N nodes included in the next layer, N auxiliary nodes are added between the target node and the N nodes;
the target layer is any layer included in the deep convolutional neural network, the target node is any node included in the target layer, and N is greater than 1 and less than or equal to the total number of nodes included in the next layer;
and training the weight between the target node and the N nodes through the N auxiliary nodes.
Optionally, training the weight between the target node and the N nodes through N auxiliary nodes includes:
calculating gradients of the N nodes relative to a target node;
the gradients of the N nodes relative to the target node are respectively multiplied by the gradients of a loss function relative to the N nodes correspondingly, wherein the loss function is a composite function taking the weight between the nodes of two adjacent layers in the deep convolutional neural network model as an independent variable;
when an intermediate calculation result is obtained by multiplying each time, storing the intermediate calculation result obtained by calculation into the corresponding auxiliary node;
when the intermediate calculation results are stored in the N auxiliary nodes, the intermediate calculation results stored in the N auxiliary nodes are transmitted to the target node;
and training the weights between the target node and the N nodes based on the intermediate calculation results stored by the N auxiliary nodes.
Optionally, training the weights between the target node and the N nodes based on the intermediate calculation results stored by the N auxiliary nodes includes:
adding the intermediate calculation results stored in the N auxiliary nodes in the target node to obtain the gradient of the loss function relative to the target node;
and training the weights between the target node and the N nodes based on the gradient of the loss function relative to the target node.
Optionally, before multiplying the gradients of the N nodes with respect to the target node by the gradient correspondences of the loss function with respect to the N nodes, the method further includes:
and respectively acquiring gradients of the loss function relative to the N nodes from the N nodes.
Optionally, after adding the intermediate calculation results stored in the N auxiliary nodes in the target node to obtain a gradient of the loss function with respect to the target node, the method further includes:
storing the gradient of the loss function relative to the target node into the target node.
In the embodiment of the disclosure, when a deep convolutional neural network model is trained through a gradient back propagation algorithm, if an output value of a target node included in a target layer in the deep convolutional neural network model is used as an input value of N nodes included in a next layer, when the deep convolutional neural network model is propagated backward in a gradient, intermediate computation results of gradients of the N nodes need to be transmitted to the target node, so N auxiliary nodes may be added between the target node and the N nodes, a weight between the target node and the N nodes is trained through the N auxiliary nodes, that is, the intermediate computation results of the gradients of the N nodes may be stored in the corresponding auxiliary nodes, when each auxiliary node stores an intermediate computation result, the intermediate computation result stored in each auxiliary node is transmitted to the target node in parallel, and then the weight between the target node and the N nodes is trained based on the intermediate computation results transmitted by the N auxiliary nodes And (5) refining. Therefore, by adopting the auxiliary node mode, a memory space does not need to be dynamically applied for the intermediate calculation result of the gradient of the N nodes, so that the gradient back propagation algorithm becomes simple and efficient, and the training process of the deep convolutional neural network model is greatly accelerated.
In the above embodiments, the implementation may be wholly or partly realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the disclosure to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., Digital Versatile Disk (DVD)), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (10)
1. An image identification method based on a deep convolutional neural network model is characterized by comprising the following steps:
in the process of training a deep convolutional neural network model through a gradient back propagation algorithm, when an output value of a target node included in a target layer in the deep convolutional neural network model is used as an input value of N nodes included in a next layer, adding N auxiliary nodes between the target node and the N nodes;
the target layer is any layer included in the deep convolutional neural network, the target node is any node included in the target layer, and the N is greater than 1 and less than or equal to the total number of nodes included in the next layer;
training the weights between the target node and the N nodes through the N auxiliary nodes, so as to obtain a group of weights which enable the image recognition rate to be high;
carrying out image recognition by using the trained deep convolution neural network model;
wherein the training of the weights between the target node and the N nodes by the N auxiliary nodes to obtain a set of weights that result in a higher image recognition rate includes:
calculating gradients of the N nodes relative to the target node;
the gradients of the N nodes relative to the target node are correspondingly multiplied by the gradients of a loss function relative to the N nodes respectively, wherein the loss function is a composite function taking the weight between the nodes of two adjacent layers in the deep convolutional neural network model as an argument;
when an intermediate calculation result is obtained by multiplying each time, storing the intermediate calculation result obtained by calculation into the corresponding auxiliary node;
when the N auxiliary nodes have intermediate calculation results stored, transmitting the intermediate calculation results stored in the N auxiliary nodes to the target node;
and training the weights between the target node and the N nodes based on the intermediate calculation results stored by the N auxiliary nodes, so as to obtain a group of weights which enable the image recognition rate to be higher.
2. The method according to claim 1, wherein the training of the weights between the target node and the N nodes based on the intermediate calculation results stored by the N auxiliary nodes to obtain a set of weights that result in a high image recognition rate comprises:
adding the intermediate calculation results stored in the N auxiliary nodes in the target node to obtain the gradient of the loss function relative to the target node;
and training the weights between the target node and the N nodes based on the gradient of the loss function relative to the target node, thereby obtaining a group of weights which enable the image recognition rate to be higher.
3. The method of claim 1 or 2, wherein prior to multiplying the gradients of the N nodes relative to the target node by the respective gradient correspondences of the loss function relative to the N nodes, further comprising:
and acquiring gradients of the loss function relative to the N nodes from the N nodes respectively.
4. The method of claim 2, wherein after adding the intermediate computation results stored in the N auxiliary nodes in the target node to obtain a gradient of the loss function with respect to the target node, further comprising:
storing the gradient of the loss function relative to the target node into the target node.
5. An apparatus for image recognition based on a deep convolutional neural network model, the apparatus comprising:
the adding module is used for adding N auxiliary nodes between a target node and N nodes when the output value of the target node included in a target layer in the deep convolutional neural network model is used as the input value of the N nodes included in the next layer in the process of training the deep convolutional neural network model through a gradient back propagation algorithm;
the target layer is any layer included in the deep convolutional neural network, the target node is any node included in the target layer, and the N is greater than 1 and less than or equal to the total number of nodes included in the next layer;
the training module is used for training the weights between the target node and the N nodes through the N auxiliary nodes so as to obtain a group of weights with higher image recognition rate; carrying out image recognition by using the trained deep convolution neural network model;
the training module comprises:
a computation submodule for computing gradients of the N nodes relative to the target node;
the multiplication submodule is used for correspondingly multiplying the gradients of the N nodes relative to the target node with the gradients of a loss function relative to the N nodes, wherein the loss function is a composite function taking the weight between the nodes of two adjacent layers in the deep convolutional neural network model as an independent variable;
the storage submodule is used for storing the intermediate calculation result obtained by calculation into the corresponding auxiliary node when each time an intermediate calculation result is obtained by multiplication;
the transmission submodule is used for transmitting the intermediate calculation results stored in the N auxiliary nodes to the target node when the intermediate calculation results are stored in the N auxiliary nodes;
and the training submodule is used for training the weights between the target node and the N nodes based on the intermediate calculation results stored by the N auxiliary nodes, so that a group of weights with higher image recognition rate is obtained.
6. The apparatus of claim 5, wherein the training submodule is to:
adding the intermediate calculation results stored in the N auxiliary nodes in the target node to obtain the gradient of the loss function relative to the target node;
and training the weights between the target node and the N nodes based on the gradient of the loss function relative to the target node, thereby obtaining a group of weights which enable the image recognition rate to be higher.
7. The apparatus of claim 5 or 6, wherein the training module further comprises:
and the acquisition submodule is used for respectively acquiring the gradients of the loss function relative to the N nodes from the N nodes.
8. The apparatus of claim 6, wherein the training sub-module is further configured to:
storing the gradient of the loss function relative to the target node into the target node.
9. An apparatus for image recognition based on a deep convolutional neural network model, the apparatus comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform the steps of any of the methods of claims 1-4.
10. A computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the steps of any of the methods of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710977989.5A CN107590534B (en) | 2017-10-17 | 2017-10-17 | Method and device for training deep convolutional neural network model and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710977989.5A CN107590534B (en) | 2017-10-17 | 2017-10-17 | Method and device for training deep convolutional neural network model and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107590534A CN107590534A (en) | 2018-01-16 |
| CN107590534B true CN107590534B (en) | 2021-02-09 |
Family
ID=61052939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710977989.5A Active CN107590534B (en) | 2017-10-17 | 2017-10-17 | Method and device for training deep convolutional neural network model and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107590534B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113807498B (en) * | 2018-07-09 | 2022-10-04 | 北京市商汤科技开发有限公司 | Model expansion method and device, electronic equipment and storage medium |
| EP3871155A4 (en) | 2018-10-23 | 2022-09-28 | HRL Laboratories, LLC | Artificial neural network and method of training an artificial neural network with epigenetic neurogenesis |
| CN110162337B (en) * | 2019-05-31 | 2020-07-03 | 中科寒武纪科技股份有限公司 | Operation method, device and related product |
| CN110598852A (en) * | 2019-08-29 | 2019-12-20 | 北京小米移动软件有限公司 | Sub-network sampling method, and method and device for constructing super-network topology structure |
| EP4334850A1 (en) | 2021-05-07 | 2024-03-13 | HRL Laboratories, LLC | Neuromorphic memory circuit and method of neurogenesis for an artificial neural network |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102523117A (en) * | 2011-12-19 | 2012-06-27 | 中山爱科数字科技股份有限公司 | Network management method applied in cloud environment |
| CN104463324A (en) * | 2014-11-21 | 2015-03-25 | 长沙马沙电子科技有限公司 | Convolution neural network parallel processing method based on large-scale high-performance cluster |
| CN105512725A (en) * | 2015-12-14 | 2016-04-20 | 杭州朗和科技有限公司 | Neural network training method and equipment |
| CN106446946A (en) * | 2016-09-22 | 2017-02-22 | 北京小米移动软件有限公司 | Image recognition method and device |
| CN106682734A (en) * | 2016-12-30 | 2017-05-17 | 中国科学院深圳先进技术研究院 | Method and apparatus for increasing generalization capability of convolutional neural network |
| CN106910497A (en) * | 2015-12-22 | 2017-06-30 | 阿里巴巴集团控股有限公司 | A kind of Chinese word pronunciation Forecasting Methodology and device |
| CN107145904A (en) * | 2017-04-28 | 2017-09-08 | 北京小米移动软件有限公司 | Determination method, device and the storage medium of image category |
| CN107229968A (en) * | 2017-05-24 | 2017-10-03 | 北京小米移动软件有限公司 | Gradient parameter determines method, device and computer-readable recording medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5376920B2 (en) * | 2008-12-04 | 2013-12-25 | キヤノン株式会社 | Convolution operation circuit, hierarchical convolution operation circuit, and object recognition device |
| CN113255885A (en) * | 2014-04-11 | 2021-08-13 | 谷歌有限责任公司 | Parallelizing training of convolutional neural networks |
-
2017
- 2017-10-17 CN CN201710977989.5A patent/CN107590534B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102523117A (en) * | 2011-12-19 | 2012-06-27 | 中山爱科数字科技股份有限公司 | Network management method applied in cloud environment |
| CN104463324A (en) * | 2014-11-21 | 2015-03-25 | 长沙马沙电子科技有限公司 | Convolution neural network parallel processing method based on large-scale high-performance cluster |
| CN105512725A (en) * | 2015-12-14 | 2016-04-20 | 杭州朗和科技有限公司 | Neural network training method and equipment |
| CN106910497A (en) * | 2015-12-22 | 2017-06-30 | 阿里巴巴集团控股有限公司 | A kind of Chinese word pronunciation Forecasting Methodology and device |
| CN106446946A (en) * | 2016-09-22 | 2017-02-22 | 北京小米移动软件有限公司 | Image recognition method and device |
| CN106682734A (en) * | 2016-12-30 | 2017-05-17 | 中国科学院深圳先进技术研究院 | Method and apparatus for increasing generalization capability of convolutional neural network |
| CN107145904A (en) * | 2017-04-28 | 2017-09-08 | 北京小米移动软件有限公司 | Determination method, device and the storage medium of image category |
| CN107229968A (en) * | 2017-05-24 | 2017-10-03 | 北京小米移动软件有限公司 | Gradient parameter determines method, device and computer-readable recording medium |
Non-Patent Citations (2)
| Title |
|---|
| Near-Duplicate Video Retrieval by Aggregating Intermediate CNN Layers;Giorgos Kordopatis-Zilos 等;《International Conference on Multimedia Modeling》;20170131;1-12 * |
| 基于二叉树的反向Hash链遍历;傅建庆 等;《计算机研究与发展》;20121231;第49卷(第2期);294-303 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107590534A (en) | 2018-01-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107590534B (en) | Method and device for training deep convolutional neural network model and storage medium | |
| CN110390394B (en) | Batch normalization data processing method and device, electronic equipment and storage medium | |
| CN107967459B (en) | Convolution processing method, convolution processing device and storage medium | |
| CN107341509B (en) | Convolutional neural network training method and device and readable storage medium | |
| US11457479B2 (en) | Method and apparatus for configuring random access occasion, method and apparatus for random access | |
| CN110543849B (en) | Detector configuration method and device, electronic equipment and storage medium | |
| CN107194464B (en) | Training method and device of convolutional neural network model | |
| CN112070235A (en) | Abnormity positioning method and device of deep learning framework and storage medium | |
| CN111898018A (en) | Virtual resource sending method and device, electronic equipment and storage medium | |
| CN111985635A (en) | Method, device and medium for accelerating neural network inference processing | |
| CN109447258B (en) | Neural network model optimization method and device, electronic device and storage medium | |
| CN107480773B (en) | Method and device for training convolutional neural network model and storage medium | |
| CN113807498B (en) | Model expansion method and device, electronic equipment and storage medium | |
| CN107992894B (en) | Image recognition method, image recognition device and computer-readable storage medium | |
| CN107229968B (en) | Gradient parameter determination method, gradient parameter determination device and computer-readable storage medium | |
| CN106447747B (en) | Image processing method and device | |
| CN107885464B (en) | Data storage method, device and computer readable storage medium | |
| CN111859097B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN109245992B (en) | Request processing method and device, electronic equipment and storage medium | |
| CN109635926B (en) | Attention feature acquisition method and device for neural network and storage medium | |
| CN112861145B (en) | Data processing method and device for data processing | |
| CN115512116A (en) | Image segmentation model optimization method and device, electronic equipment and readable storage medium | |
| CN111259675B (en) | Neural network calculation-based method and device | |
| CN110457084B (en) | Loading method and device | |
| CN109981729B (en) | File processing method and device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |