CN113762496A - Method for reducing inference operation complexity of low-bit convolutional neural network - Google Patents

Method for reducing inference operation complexity of low-bit convolutional neural network Download PDF

Info

Publication number
CN113762496A
CN113762496A CN202010497777.9A CN202010497777A CN113762496A CN 113762496 A CN113762496 A CN 113762496A CN 202010497777 A CN202010497777 A CN 202010497777A CN 113762496 A CN113762496 A CN 113762496A
Authority
CN
China
Prior art keywords
quantization
feature map
formula
neural network
bit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010497777.9A
Other languages
Chinese (zh)
Other versions
CN113762496B (en
Inventor
张东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei Ingenic Technology Co ltd
Original Assignee
Hefei Ingenic Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hefei Ingenic Technology Co ltd filed Critical Hefei Ingenic Technology Co ltd
Priority to CN202010497777.9A priority Critical patent/CN113762496B/en
Publication of CN113762496A publication Critical patent/CN113762496A/en
Application granted granted Critical
Publication of CN113762496B publication Critical patent/CN113762496B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

The invention provides a method for reducing the inference operation complexity of a low-bit convolutional neural network, which comprises the following steps of quantizing by using stored data after training of an S1 neural network is finished, and assuming that the quantization of an ith layer is as follows:
Figure DDA0002523509910000011
Figure DDA0002523509910000012
wherein deltaiTo activate a function, QAFor the quantization formula of feature map, QwA quantization formula for the weight; s2 quantifying when the parameters of the formula in S1 meet the conditions
Figure DDA0002523509910000013
Obtaining through the operation of fixed point number:
Figure DDA0002523509910000014
s3 determines a threshold from the quantization of feature map: quantization of feature map:
Figure DDA0002523509910000015
the direct derivation of the threshold value from the quantization formula of feature map is (0.5, 1.5 … (2)k-0.5)), where k is the quantized bit-width; since the distance between the thresholds is all 1.0, only the hold is needed at the final quantization
Figure DDA0002523509910000016
Wherein
Figure DDA0002523509910000017
Then the threshold value
Figure DDA0002523509910000018
n∈{0,1…(2k‑1) Where k is the quantized bit-width; s4 since quantization is low, the value of the feature map after quantization is determined, and QAFor uniform quantization, so in S2
Figure DDA0002523509910000019
Pass and a series of thresholds (T)1，T2…Tn) And comparing to obtain a final quantification result. The method and the device solve the problems of high computational complexity and high computational resource requirements in the low-bit model reasoning process.

Description

Method for reducing inference operation complexity of low-bit convolutional neural network
Technical Field
The invention relates to the technical field of neural network acceleration, in particular to a method for reducing the reasoning and operation complexity of a low-bit convolutional neural network.
Background
In recent years, with the rapid development of science and technology, a big data age has come. Deep learning takes a Deep Neural Network (DNN) as a model, and achieves remarkable results in key fields of many human intelligence, such as image recognition, reinforcement learning, semantic analysis and the like. The Convolutional Neural Network (CNN) is a typical DNN structure, can effectively extract hidden layer features of an image and accurately classify the image, and is widely applied to the field of image identification and detection in recent years.
In particular, the multiply-shift implements a 32-bit quantization to a low bit: the result of the quantization convolution operation is stored as 32-bit shaping, and then multiplication and shift operation are carried out according to the pre-calculated parameters to realize the conversion from 32-bit to low bit.
However, when the 32bit quantization is low in the prior art, since the precision after quantization needs to be ensured, a series of addition and comparison operations need to be performed in the quantization process, which greatly increases the computational complexity and the computational resource, and especially when the quantization is 2bit, the cost is often too large.
Furthermore, the common terminology in the prior art is as follows:
convolutional Neural Networks (CNN): is a type of feedforward neural network that contains convolution calculations and has a depth structure.
And (3) quantification: quantization refers to the process of approximating a continuous value (or a large number of possible discrete values) of a signal to a finite number (or fewer) of discrete values.
Low bit rate: and quantizing the data into data with bit width of 8bit, 4bit or 2 bit.
Reasoning: and after the neural network training is finished, the stored data is used for carrying out the operation process.
Disclosure of Invention
The application provides a method for reducing the inference operation complexity of a low-bit convolutional neural network, aims to overcome the defects in the prior art and solves the problems of high computation complexity and high computation resource requirements in the inference process of the existing low-bit model.
Specifically, the invention provides a method for reducing the inference operation complexity of a low bit convolution neural network, which comprises the following steps:
s1, after the training of the neural network is finished, the stored data is used for quantization,
assume that quantization of the ith layer is as follows:
Figure BDA0002523509890000021
Figure BDA0002523509890000022
wherein deltaiTo activate a function, QAFor the quantization formula of feature map, QwA quantization formula for the weight;
s2, when the parameters of the formula in S1 meet the following conditions:
1)、
Figure BDA0002523509890000023
expressed in fixed point numbers scaled by floating point scalars
Figure BDA0002523509890000024
wintIs a fixed point number expressed in an integer;
2)、
Figure BDA0002523509890000025
expressed in fixed point numbers scaled by floating point scalars
Figure BDA0002523509890000026
xintIs a fixed point number expressed in an integer;
3)、δiis a monotonic function;
then, quantize
Figure BDA0002523509890000027
Obtained by the operation of fixed point number, namely:
Figure BDA0002523509890000028
Figure BDA0002523509890000031
s3, determining a threshold from the quantization of feature map:
the quantization formula of feature map is:
Figure BDA0002523509890000032
the quantization formula of the above feature map can directly deduce that the threshold value is (0.5, 1.5 … (2)k-0.5)), where k is the quantized bit-width;
since the distance between the thresholds is all 1.0, only the hold is needed at the final quantization
Figure BDA0002523509890000033
Wherein
Figure BDA0002523509890000034
Then the threshold value
Figure BDA0002523509890000035
n∈{0,1…(2k-1) Where k is the quantized bit-width;
s4, since the quantization is low bit, the value of the feature map after quantization is determined, and QATo quantize uniformly, δ in S2i(swsxsBN(wint·xint+bi/(swsxsBN) ) pass and the series of said threshold values (T) in step S31，T2…Tn) And comparing to obtain a final quantification result.
In step S2, when the quantization is a low-bit 2bit, the value of the feature map after quantization is 0,1,2, and 3.
The step S2 is due to deltaiIs a monotonic function, swsx> 0, so can also pass (w)int·xint+bi/(swsxsBN) ) and
Figure BDA0002523509890000036
to obtain a quantized result.
In the step S4, S isBNEach channel is not the same, so saving the threshold requires that each channel needs to be saved one.
Thus, the present application has the advantages that:
1. the quantization of 32 bits into low bits is realized directly through threshold comparison, so that the complexity of operation is reduced;
2. the overall running time of the quantization model is reduced;
3. the demand of computing resources is reduced;
the 64bit by 64bit operation is avoided.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention.
FIG. 1 is a schematic flow diagram of the process of the present invention.
Detailed Description
In order that the technical contents and advantages of the present invention can be more clearly understood, the present invention will now be described in further detail with reference to the accompanying drawings.
As shown in fig. 1, a method for reducing the complexity of inference operation of low bit convolution neural network of the present invention includes the following steps:
s1, after the training of the neural network is finished, the stored data is used for quantization,
assume that quantization of the ith layer is as follows:
Figure BDA0002523509890000041
Figure BDA0002523509890000042
wherein deltaiTo activate a function, QAFor the quantization formula of feature map, QwA quantization formula for the weight;
s2, when the parameters of the formula in S1 meet the following conditions:
1)、
Figure BDA0002523509890000043
expressed in fixed point numbers scaled by floating point scalars
Figure BDA0002523509890000044
wintIs a fixed point number expressed in an integer;
2)、
Figure BDA0002523509890000051
expressed in fixed point numbers scaled by floating point scalars
Figure BDA0002523509890000052
xintIs a fixed point number expressed in an integer;
3)、δiis a monotonic function;
then, quantize
Figure BDA0002523509890000053
Obtained by the operation of fixed point number, namely:
Figure BDA0002523509890000054
s3, determining a threshold from the quantization of feature map:
the quantization formula of feature map is:
Figure BDA0002523509890000055
the quantization formula of the above feature map can directly deduce that the threshold value is (0.5, 1.5 … (2)k-0.5)), where k is the quantized bit-width;
since the distance between the thresholds is all 1.0, only the hold is needed at the final quantization
Figure BDA0002523509890000056
Wherein
Figure BDA0002523509890000057
Then the threshold value
Figure BDA0002523509890000058
n∈{1，2…(2k-1) Where k is the quantized bit-width;
s4, since the quantization is low bit, the value of the feature map after quantization is determined, and QATo uniformly quantizeSo δ in S2i(swsxsBN(wint·xint+bi/(swsxsBN) ) pass and the series of said threshold values (T) in step S31，T2…Tn) And comparing to obtain a final quantification result.
In particular, the method of the present application can also be expressed as follows:
assume that the quantization calculation for the ith layer is as follows:
Figure BDA0002523509890000059
Figure BDA0002523509890000061
wherein deltaiTo activate a function, QAFor the quantization formula of feature map, QwQuantization formula for weight
The parameters in the above formula meet the following conditions:
1、
Figure BDA0002523509890000062
can be represented by fixed-point numbers scaled by floating-point scalars
Figure BDA0002523509890000063
wintIs a fixed point number expressed by an integer
2、
Figure BDA0002523509890000064
Can be represented by fixed-point numbers scaled by floating-point scalars
Figure BDA0002523509890000065
xintIs a fixed point number expressed by an integer
3、δiIs a monotonic function
So that the final is calculated
Figure BDA0002523509890000066
The following can be obtained by the fixed point number operation:
Figure BDA0002523509890000067
since the quantization is low, the value of the quantized feature map is actually determined (taking 2 bits as an example, the value of the feature map is 0,1,2,3), and Q isATo uniform quantization, soi(swsxsBN(wint·xint+bi/(swsxsBN) A passable and a series of threshold values (T)1，T2…Tn) Compared to obtain a quantized result due to deltaiIs a monotonic function, swsx> 0, so can also pass (w)int·xint+bi/(swsxsBN) ) and
Figure BDA0002523509890000068
to obtain a quantized result.
The determination of the threshold needs to be started from the quantization formula of feature map.
The quantization formula of feature map is:
Figure BDA0002523509890000069
the threshold value of (0.5, 1.5 … (2k-0.5)) can be directly derived from the above equation, where k is the quantized bit width. Since the distance between the thresholds is 1.0, we only need to save it in the final quantization
Figure BDA0002523509890000071
Wherein
Figure BDA0002523509890000072
Then the threshold value is set
Figure BDA0002523509890000073
n∈{0,1…(2k-1) Where k is the quantized bit-width; due to SBNEach channel is not the same, so saving the threshold requires that each channel needs to be saved one.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes may be made to the embodiment of the present invention by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (4)

1. A method for reducing the complexity of low bit convolutional neural network inferential computation, comprising the steps of:
s1, after the training of the neural network is finished, the stored data is used for quantization,
assume that quantization of the ith layer is as follows:
Figure FDA0002523509880000011
Figure FDA0002523509880000012
wherein deltaiTo activate a function, QAFor the quantization formula of feature map, QwA quantization formula for the weight;
s2, when the parameters of the formula in S1 meet the following conditions:
1)、
Figure FDA0002523509880000013
expressed in fixed point numbers scaled by floating point scalars
Figure FDA0002523509880000014
wintIs a fixed point number expressed in an integer;
2)、
Figure FDA0002523509880000015
expressed in fixed point numbers scaled by floating point scalars
Figure FDA0002523509880000016
xintIs a fixed point number expressed in an integer;
3)、δiis a monotonic function;
then, quantize
Figure FDA0002523509880000017
Obtained by the operation of fixed point number, namely:
Figure FDA0002523509880000018
s3, determining a threshold from the quantization of feature map:
the quantization formula of feature map is:
Figure FDA0002523509880000019
the threshold value is directly deduced from the quantization formula of the above feature map as (0.5, 1.5 … (2)k-0.5)), where k is the quantized bit-width;
since the distance between the thresholds is all 1.0, only the hold is needed at the final quantization
Figure FDA0002523509880000021
Wherein
Figure FDA0002523509880000022
Then the threshold value
Figure FDA0002523509880000023
n∈{0,1…(2k-1) Where k is the quantized bit-width;
s4, since the quantization is low bit, the value of the feature map after quantization is determined, and QATo quantize uniformly, δ in S2i(swsxsBN(wint·xint+bi/(swsxsBN) ) pass and the series of said threshold values (T) in step S31，T2…Tn) And comparing to obtain a final quantification result.
2. The method according to claim 1, wherein when the quantization is a low bit 2bit in step S2, the value of the feature map after quantization is 0,1,2, 3.
3. The method for reducing the complexity of inference operations of low bit convolution neural network as claimed in claim 1, wherein said step S2 is performed due to deltaiIs a monotonic function, swsx> 0, so can also pass (w)int·xint+bi/(swsxsBN) ) and
Figure FDA0002523509880000024
to obtain a quantized result.
4. The method for reducing the complexity of inference operations of low bit convolution neural network as claimed in claim 1, wherein S is the factor of S in step S4BNEach channel is not the same, so saving the threshold requires that each channel needs to be saved one.
CN202010497777.9A 2020-06-04 2020-06-04 Method for reducing low-bit convolutional neural network reasoning operation complexity Active CN113762496B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010497777.9A CN113762496B (en) 2020-06-04 2020-06-04 Method for reducing low-bit convolutional neural network reasoning operation complexity

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010497777.9A CN113762496B (en) 2020-06-04 2020-06-04 Method for reducing low-bit convolutional neural network reasoning operation complexity

Publications (2)

Publication Number Publication Date
CN113762496A true CN113762496A (en) 2021-12-07
CN113762496B CN113762496B (en) 2024-05-03

Family

ID=78783418

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010497777.9A Active CN113762496B (en) 2020-06-04 2020-06-04 Method for reducing low-bit convolutional neural network reasoning operation complexity

Country Status (1)

Country Link
CN (1) CN113762496B (en)

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107944458A (en) * 2017-12-08 2018-04-20 北京维大成科技有限公司 A kind of image-recognizing method and device based on convolutional neural networks
GB201821150D0 (en) * 2018-12-21 2019-02-06 Imagination Tech Ltd Methods and systems for selecting quantisation parameters for deep neural neitworks using back-propagation
CN109389212A (en) * 2018-12-30 2019-02-26 南京大学 A kind of restructural activation quantization pond system towards low-bit width convolutional neural networks
US20190138882A1 (en) * 2017-11-07 2019-05-09 Samusung Electronics Co., Ltd. Method and apparatus for learning low-precision neural network that combines weight quantization and activation quantization
CN110188877A (en) * 2019-05-30 2019-08-30 苏州浪潮智能科技有限公司 A kind of neural network compression method and device
US20190279072A1 (en) * 2018-03-09 2019-09-12 Canon Kabushiki Kaisha Method and apparatus for optimizing and applying multilayer neural network model, and storage medium
JP2019160319A (en) * 2018-03-09 2019-09-19 キヤノン株式会社 Method and device for optimizing and applying multi-layer neural network model, and storage medium
CN110363281A (en) * 2019-06-06 2019-10-22 上海交通大学 A kind of convolutional neural networks quantization method, device, computer and storage medium
US20190340492A1 (en) * 2018-05-04 2019-11-07 Microsoft Technology Licensing, Llc Design flow for quantized neural networks
US10592799B1 (en) * 2019-01-23 2020-03-17 StradVision, Inc. Determining FL value by using weighted quantization loss values to thereby quantize CNN parameters and feature values to be used for optimizing hardware applicable to mobile devices or compact networks with high precision
CN111105007A (en) * 2018-10-26 2020-05-05 中国科学院半导体研究所 Compression acceleration method of deep convolutional neural network for target detection

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190138882A1 (en) * 2017-11-07 2019-05-09 Samusung Electronics Co., Ltd. Method and apparatus for learning low-precision neural network that combines weight quantization and activation quantization
CN107944458A (en) * 2017-12-08 2018-04-20 北京维大成科技有限公司 A kind of image-recognizing method and device based on convolutional neural networks
US20190279072A1 (en) * 2018-03-09 2019-09-12 Canon Kabushiki Kaisha Method and apparatus for optimizing and applying multilayer neural network model, and storage medium
JP2019160319A (en) * 2018-03-09 2019-09-19 キヤノン株式会社 Method and device for optimizing and applying multi-layer neural network model, and storage medium
US20190340492A1 (en) * 2018-05-04 2019-11-07 Microsoft Technology Licensing, Llc Design flow for quantized neural networks
CN111105007A (en) * 2018-10-26 2020-05-05 中国科学院半导体研究所 Compression acceleration method of deep convolutional neural network for target detection
GB201821150D0 (en) * 2018-12-21 2019-02-06 Imagination Tech Ltd Methods and systems for selecting quantisation parameters for deep neural neitworks using back-propagation
CN109389212A (en) * 2018-12-30 2019-02-26 南京大学 A kind of restructural activation quantization pond system towards low-bit width convolutional neural networks
US10592799B1 (en) * 2019-01-23 2020-03-17 StradVision, Inc. Determining FL value by using weighted quantization loss values to thereby quantize CNN parameters and feature values to be used for optimizing hardware applicable to mobile devices or compact networks with high precision
CN110188877A (en) * 2019-05-30 2019-08-30 苏州浪潮智能科技有限公司 A kind of neural network compression method and device
CN110363281A (en) * 2019-06-06 2019-10-22 上海交通大学 A kind of convolutional neural networks quantization method, device, computer and storage medium

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
KRISHNAMOORTHI R: "Quantizing deep convolutional networks for efficient inference: A whitepaper", ARXIV PREPRINT ARXIV:1806.08342, 31 December 2018 (2018-12-31) *
ZHUANG B等: "Towards effective low-bitwidth convolutional neural networks", PROCEEDINGS OF THE IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, 31 December 2018 (2018-12-31) *
付强等: "卷积神经网络低位宽量化推理研究", 计算机与数字工程, 31 December 2019 (2019-12-31) *
牟帅: "基于位量化的深度神经网络加速与压缩研究", 中国硕士学位论文全文库 信息科技辑, 15 June 2018 (2018-06-15) *
蔡瑞初等: "面向"边缘"应用的卷积神经网络量化与压缩方法", 计算机应用, no. 09, 23 April 2018 (2018-04-23) *

Also Published As

Publication number Publication date
CN113762496B (en) 2024-05-03

Similar Documents

Publication Publication Date Title
CN110874625B (en) Data processing method and device
US10491239B1 (en) Large-scale computations using an adaptive numerical format
CN111612147A (en) Quantization method of deep convolutional network
CN112381205A (en) Neural network low bit quantization method
CN111179944B (en) Voice awakening and age detection method and device and computer readable storage medium
CN114708855B (en) Voice awakening method and system based on binary residual error neural network
CN110647990A (en) Cutting method of deep convolutional neural network model based on grey correlation analysis
CN115905855A (en) Improved meta-learning algorithm MG-copy
CN114943335A (en) Layer-by-layer optimization method of ternary neural network
CN116884398B (en) Speech recognition method, device, equipment and medium
CN113762496A (en) Method for reducing inference operation complexity of low-bit convolutional neural network
CN110837885B (en) Sigmoid function fitting method based on probability distribution
CN114169513B (en) Neural network quantization method and device, storage medium and electronic equipment
CN113408696A (en) Fixed point quantization method and device of deep learning model
WO2022247368A1 (en) Methods, systems, and mediafor low-bit neural networks using bit shift operations
CN112885367B (en) Fundamental frequency acquisition method, fundamental frequency acquisition device, computer equipment and storage medium
CN111614358B (en) Feature extraction method, system, equipment and storage medium based on multichannel quantization
CN113762452B (en) Method for quantizing PRELU activation function
CN112561050B (en) Neural network model training method and device
CN113761834A (en) Method, device and storage medium for acquiring word vector of natural language processing model
CN116468963A (en) Method for processing weight abnormal value during post-quantization of model
CN113762500B (en) Training method for improving model precision during quantization of convolutional neural network
CN113762495A (en) Method for improving precision of low bit quantization model of convolutional neural network model
CN115238873B (en) Neural network model deployment method and device, and computer equipment
CN109359728B (en) Method, storage medium and apparatus for calculating optimal fixed point bits for neural network compression

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant