WO2024259853A1 - 一种数据处理方法、装置、计算机设备及存储介质 - Google Patents

一种数据处理方法、装置、计算机设备及存储介质 Download PDF

Info

Publication number
WO2024259853A1
WO2024259853A1 PCT/CN2023/128556 CN2023128556W WO2024259853A1 WO 2024259853 A1 WO2024259853 A1 WO 2024259853A1 CN 2023128556 W CN2023128556 W CN 2023128556W WO 2024259853 A1 WO2024259853 A1 WO 2024259853A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
layer
operator
quantization
network layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/128556
Other languages
English (en)
French (fr)
Inventor
程雅慧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Publication of WO2024259853A1 publication Critical patent/WO2024259853A1/zh
Priority to US19/292,293 priority Critical patent/US20250363103A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Definitions

  • the present application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer equipment and storage medium.
  • AI models artificial intelligence models
  • the structure of artificial intelligence models i.e. AI models
  • the weight parameters of AI models have become larger and larger, so the hardware requirements for forward reasoning have become higher and higher.
  • the embodiments of the present application provide a data processing method, apparatus, computer device and storage medium, which can improve the speed of model quantization and achieve the universality of model quantization.
  • An embodiment of the present application provides a data processing method, including:
  • a forged operator for quantization is used to quantize the target structure, and the forged operator includes a quantization operator and an anti-quantization operator;
  • An embodiment of the present application provides a data processing device, including:
  • a search module is used to perform a structural search on the initial model when the initial model is obtained, so as to obtain a model structure diagram of the initial model;
  • a quantization module used to quantize the target structure according to the type of the target structure in the model structure diagram, using a forged operator for quantization, where the forged operator includes a quantization operator and an anti-quantization operator;
  • the business model determination module is used to obtain a business model for business processing based on the quantified initial model.
  • the quantization module includes:
  • the first quantization unit is used to add the forged operator at the input position of the first network layer with weights when the target structure in the model structure diagram includes a first network layer with weights, and use the forged operator to quantize the weights of the first network layer with weights.
  • the quantization module further includes:
  • the second quantization unit is used to add the forged operator respectively at the multiple output positions of the first network layer with weights when the first network layer with weights has multiple output positions.
  • the quantization module further includes:
  • the third quantization unit is used to add the forged operator at the output position of the first network layer with weights when the first network layer with weights has an output position and the output position of the first network layer with weights is not connected to the activation layer.
  • the fake operator is not added to the output position of the first network layer with weights.
  • the quantization module further includes:
  • a fourth quantization unit configured to add a forgery operator at an output position of the overlay layer when the target structure in the model structure diagram includes an overlay layer, a first input branch of the overlay layer, and a second input branch of the overlay layer;
  • the first input branch includes a first network layer and a normalization layer with weights connected in sequence, the normalization layer is connected to the superposition layer, the type of the second input branch is the input type of non-weighted operation, and the first network layer with weights is a convolutional layer.
  • the quantization module further includes:
  • the fifth quantization unit is used to add a spurious operator at an input position of the second input branch.
  • the quantization module further includes:
  • a sixth quantization unit is used to add a pseudo operator for quantization at the input position of the first input branch, add a pseudo operator at the input position of the second input branch, and add a pseudo operator at the output position of the first superimposed layer when the target structure in the model structure diagram includes a first superimposed layer and the type of the first input branch of the first superimposed layer and the type of the second input branch of the first superimposed layer are both input types of non-weighted operations.
  • the quantization module further includes:
  • a seventh quantization unit configured to, when the target structure in the model structure diagram includes a first splicing layer and the number of input branches of the first splicing layer is the same as the number of output branches of the first splicing layer, quantize each of the N output branches of the first splicing layer; At the output positions of the output branches, pseudo operators for quantization are respectively added; N is a positive integer greater than 1; and the quantization operators of the pseudo operators added at each output branch are the same.
  • the business model determination module includes:
  • a first training unit is used to determine the quantized initial model as the first model, and perform fine-tuning training on the first model to obtain a second model;
  • the first determination unit is used to obtain a business model for business processing based on the second model;
  • the business model includes a second network layer with a weight, and a quantization operator added at the input position of the second network layer with a weight; in the business model, the second network layer with a weight is obtained based on the first network layer with a weight, and the weight of the second network layer with a weight is obtained by processing the weight of the first network layer with a weight in the second model using the quantization operator.
  • the first network layer with weights in the target structure is a convolutional layer, and the target structure further includes a normalization layer connected to the convolutional layer, and an activation layer connected to the normalization layer;
  • the business model determination module also includes:
  • the first fusion unit is used to fuse the first network layer with weights, the normalization layer and the activation layer to obtain a second network layer with weights.
  • the target structure includes an overlay layer, a first input branch of the overlay layer, and a second input branch of the overlay layer;
  • the first input branch includes a first network layer and a normalization layer connected in sequence with weights;
  • the normalization layer is connected to the overlay layer, the type of the second input branch is an input type of a non-weighted operation, and the first network layer with weights is a convolutional layer;
  • the business model determination module also includes:
  • the second fusion unit is used to fuse the first network layer with weights, the normalization layer and the superposition layer to obtain a second network layer with weights.
  • the business model determination module further includes:
  • a second training unit is used to determine the quantized initial model as the first model, and perform fine-tuning training on the first model to obtain a second model;
  • the second determination unit is used to obtain a business model for business processing based on the second model;
  • the business model includes a second superimposed layer, a quantization operator added at the first input branch of the second superimposed layer, and a quantization operator added at the second input branch of the second superimposed layer; in the business model, the second superimposed layer is obtained according to the quantization operator and the first superimposed layer in the second model.
  • the business model determination module further includes:
  • the third training unit is used to determine the quantized initial model as the first model and perform fine-tuning training on the first model. Get the second model;
  • the third determination unit is used to obtain a business model for business processing based on the second model;
  • the business model includes a second splicing layer, and a quantization operator added at the input position of each input branch of the second splicing layer;
  • the quantization operator at the input position of each input branch of the second splicing layer is the quantization operator added at the output position of each output branch of the first splicing layer in the second model, which is obtained by equivalently forward propagating to the input position of each input branch of the first splicing layer in the second model.
  • the device further comprises:
  • a feature acquisition module used for acquiring image features of an image to be recognized when an image to be recognized is obtained if the business processing of the business model is image recognition;
  • a feature input module is used to input image features into the business model;
  • the optimization structure in the business model includes a target network layer and a quantization operator added at an input position of the target network layer;
  • the target network layer is obtained according to the quantization operator and the original network layer of the target structure in the initial model;
  • a precision conversion module for converting the precision of the input feature of the optimized structure from a first precision to a second precision based on a quantization operator added at an input position of the target network layer; the first precision is higher than the second precision;
  • a feature extraction module used for performing feature extraction processing on input features belonging to the second precision through a target network layer, and the target network layer outputs output features corresponding to the input features;
  • the category determination module is used to obtain the predicted category of the image to be identified based on the output features.
  • the present application provides a computer device, including: a processor, a memory, and a network interface;
  • the processor is connected to the memory and the network interface, wherein the network interface is used to provide a data communication function, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method provided in the embodiment of the present application.
  • an embodiment of the present application provides a computer-readable storage medium, which stores a computer program.
  • the computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the method provided by the embodiment of the present application.
  • an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium; a processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method in the embodiment of the present application.
  • the computer device with the model quantization function when the computer device with the model quantization function obtains the initial model, it does not need to manually add a forged operator to the initial model. Instead, after searching the model structure diagram of the initial model, the computer device directly adds a forged operator according to the initial model.
  • the type of target structure in the model structure diagram of the initial model is automatically quantized by using a forged operator for quantization (the forged operator here may include a quantization operator and an anti-quantization operator), and then after the quantization is completed, the business model for business processing can be determined based on the quantized initial model.
  • the data processing method provided in the embodiment of the present application does not require human participation in model quantization.
  • the forged operator for quantization can be used according to the type of target structure in the model structure diagram to automatically quantize the accelerable structure to be quantized (i.e., the target structure) in the initial model, which can effectively reduce labor costs, improve the speed of model quantization, and achieve the versatility of model quantization.
  • FIG1 is a schematic diagram of a network architecture provided in an embodiment of the present application.
  • FIG2 is a schematic diagram of a structure of multiple structures to be quantized provided in an embodiment of the present application
  • FIG3 is a flow chart of a data processing method provided in an embodiment of the present application.
  • FIG4 is a schematic diagram 1 of quantifying a target structure provided in an embodiment of the present application.
  • FIG5 is a second schematic diagram of quantifying a target structure provided by an embodiment of the present application.
  • FIG6 is a third schematic diagram of quantifying a target structure provided in an embodiment of the present application.
  • FIG7 is a fourth schematic diagram of quantifying a target structure provided in an embodiment of the present application.
  • FIG8 is a fifth schematic diagram of quantifying a target structure provided by an embodiment of the present application.
  • FIG9 is a schematic diagram 1 of processing a structure to be optimized in a second model provided by an embodiment of the present application.
  • FIG10 is a second schematic diagram of processing a structure to be optimized in a second model provided by an embodiment of the present application.
  • FIG11 is a third schematic diagram of processing a structure to be optimized in a second model provided by an embodiment of the present application.
  • FIG12 is a fourth schematic diagram of processing a structure to be optimized in a second model provided by an embodiment of the present application.
  • FIG13 is a fifth schematic diagram of processing a structure to be optimized in a second model provided by an embodiment of the present application.
  • FIG14 is a flow chart of a data processing method provided in an embodiment of the present application.
  • FIG15 is a schematic diagram of a scenario for optimizing a business model provided by an embodiment of the present application.
  • FIG16 is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application.
  • FIG. 17 is a schematic diagram of a computer device provided in an embodiment of the present application.
  • the embodiment of the present application proposes a general quantification method for artificial intelligence models with complex structures.
  • the quantification method involves the field of artificial intelligence, wherein the so-called artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or digital computer-controlled calculations to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
  • AI artificial intelligence
  • Artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence.
  • Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.
  • Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies.
  • Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating/interactive systems, mechatronics and other technologies.
  • Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning/deep learning, autonomous driving, smart transportation and other major directions.
  • machine learning is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.
  • Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence.
  • Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.
  • the deep learning algorithm is composed of a number of computing units, and the embodiments of the present application may refer to these computing units as operators (OP).
  • the operator corresponds to the computing logic in the network layer.
  • the operator corresponding to the convolution layer i.e., the convolution operator
  • FC layer fully-connected layer
  • the operator corresponding to the activation layer i.e., the activation operator
  • the operator used as the activation function in the network model e.g., tanh, ReLU, etc.
  • Computer Vision is a science that studies how to make machines "see”. To put it more specifically, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further Do graphic processing to make the computer processing into an image that is more suitable for human observation or transmission to instruments for detection.
  • Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, automatic driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.
  • Natural language processing is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.
  • the key technologies of speech technology include automatic speech recognition technology, speech synthesis technology and voiceprint recognition technology. Enabling computers to listen, see, speak and feel is the future development direction of human-computer interaction, among which speech has become one of the most promising human-computer interaction methods in the future.
  • the embodiment of the present application can adopt a general quantization method to quantize the artificial intelligence model, so that the quantized model has the following advantages: less storage overhead and bandwidth requirements, faster computing speed, lower energy consumption and occupied area, acceptable precision loss, support for low precision (for example, int8), etc.
  • the network architecture may include a server 10S and a terminal device cluster.
  • the terminal device cluster may include one or more terminal devices, and the number of terminal devices will not be limited here.
  • the terminal device cluster may specifically include terminal devices 100a, terminal devices 100b, terminal devices 100c, ..., terminal devices 100n.
  • terminal devices 100a, terminal devices 100b, terminal devices 100c, ..., terminal devices 100n can be respectively connected to the above-mentioned server 10S through a network, so that each terminal device can exchange data with the server 10S through the network connection.
  • the network connection here does not limit the connection method, and can be directly or indirectly connected through a wired communication method, or directly or indirectly connected through a wireless communication method, or through other methods, and the present application does not limit it here.
  • each terminal device in the terminal device cluster may include: smart phones, tablet computers, laptops, desktop computers, smart speakers, smart watches, car terminals, smart TVs and other smart terminals with data processing functions.
  • each terminal device in the terminal device cluster shown in Figure 1 can be installed with a business application (i.e., an application client).
  • the application client runs in each terminal device, data can be exchanged with the server 10S shown in Figure 1 above.
  • the application client may include application clients such as social clients, multimedia clients (e.g., video clients), entertainment clients (e.g., game clients), information flow clients, education clients, and live broadcast clients.
  • the application client can be an independent client or an embedded sub-client integrated in a client (e.g., a social client, an education client, and a multimedia client, etc.), which is not limited here.
  • the server 10S in the embodiment of the present application may be a server corresponding to the application client.
  • the server 10S may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.
  • the embodiment of the present application does not limit the number of servers.
  • the embodiment of the present application can select a terminal device from the multiple terminal devices shown in Figure 1 as the business terminal device used by the business object.
  • the embodiment of the present application can use the terminal device 100a shown in Figure 1 as a business terminal device, and the business terminal device can be integrated with a business application (i.e., an application client).
  • the business terminal device can implement data interaction with the server 10S through the business data platform corresponding to the application client.
  • the business object here (e.g., a user) can specify that certain network layers in the initial model do not participate in quantization, wherein the network layer specified by the business object that does not participate in quantization can be referred to as a designated network layer (i.e., a network layer belonging to a blacklist) in the embodiment of the present application.
  • a designated network layer i.e., a network layer belonging to a blacklist
  • the computer device with the model quantization function can be a server, or any terminal device in the terminal device cluster shown in FIG1 , for example, terminal device 100b, and the specific form of the computer device will not be limited here.
  • the computer device in the embodiment of the present application can be a server (for example, server 10S shown in FIG1 ) as an example to illustrate the specific implementation method of the computer device automatically quantizing the initial model through a universal quantization tool.
  • the general quantization tool in the embodiment of the present application utilizes rich reasoning experience to design various structures to be quantized that are suitable for low-precision (for example, int8) acceleration, that is, different quantization rules are set for different types of structures to be quantized to obtain an initial rule set, so that the obtained initial model can be automatically quantized based on the quantization rule set determined by the initial rule set.
  • low-precision for example, int8
  • the quantization rule set here is the initial rule set obtained by the general quantization tool; if the business object is configured with a specified network layer that does not participate in quantization, the quantization rule set here is based on the specified network layer configured by the business object, and the initial rule set is filtered to obtain the rule set.
  • the quantization priority of the specified network layer configured by the business object is higher than the general initial rule set, and has the highest quantization priority.
  • the quantization rule set here may include N quantization rules, where N is a positive integer, and one quantization rule corresponds to a structure to be quantized.
  • N is a positive integer
  • one quantization rule corresponds to a structure to be quantized.
  • the computer device obtains the initial model to be optimized, it needs to perform a structural search on the initial model to obtain a model structure diagram of the initial model, and then query whether there is a target structure in the model structure diagram, where the target structure belongs to the N structures to be quantified corresponding to the quantization rule set. If the target structure exists in the model structure diagram, the computer device can automatically determine the target quantization rule that matches the target structure from the N quantization rules, and then quantize the target structure based on the target quantization rule and the forgery operator, and after the quantization is completed, determine the business model for business processing based on the quantized initial model.
  • the fake operator (e.g., fake op) is mainly used to quantize and then dequantize the parameters and inputs input to the fake operator, which is equivalent to the QDQ operator, and its structure consists of a quantization operator (quant op) and a dequantization operator (dequant op).
  • the fake operator can be used to store the quantization information of the current layer, which may specifically include a scaling factor (scale) and an offset (zero-point).
  • the embodiment of the present application can also determine the accuracy of each layer of operators based on the positions of the quantization operator and the dequantization operator in the fake operator.
  • quantization operator is mainly used to convert features belonging to the first precision into features belonging to the second precision
  • dequantization operator is mainly used to convert features belonging to the second precision back to features belonging to the first precision, where the first precision is higher than the second precision.
  • quantization can convert a floating-point real number (e.g., FP32) into an integer number (e.g., INT8)
  • dequantization can convert an integer number (e.g., INT8) back into a floating-point real number (e.g., FP32).
  • the initial model here can include models in various artificial intelligence scenarios, for example, a detection model for content review, a detection model for key part identification (for example, a face detection model) and an image recognition model, etc., and the application scenario of the initial model will not be limited here.
  • the computer device can automatically quantize the accelerable structure to be quantified in the initial model according to a pre-set set of quantization rules, so as to determine the business model for business processing based on the quantized initial model.
  • This quantization method does not require manual design of different quantization schemes for different models, but can automatically quantize the accelerable structure to be quantified in the initial model, which can not only effectively reduce labor costs, but also improve the model quantization speed.
  • the quantization method of the embodiment of the present application does not require manual insertion of counterfeit operators, which reduces the code intrusiveness, thereby achieving model quantization universality.
  • Figure 2 is a schematic diagram of a structure of multiple structures to be quantified provided in an embodiment of the present application.
  • the computer device in the embodiment of the present application can be a computer device with a model quantization function, and the computer device can be any terminal device in the terminal device cluster shown in Figure 1 above, for example, the terminal device 100b, and the computer device can also be the server 10S shown in Figure 1 above, and the computer device will not be limited here.
  • the embodiments of the present application can design various patterns suitable for low-precision acceleration based on the rich reasoning experience in a deep learning reasoning framework, and then different quantization rules can be designed based on different patterns.
  • the pattern belongs to the minimum granularity type searched by the present application, and the pattern can be a module composed of multiple network layers or a module composed of a single network layer.
  • the rule set 20R is the initial rule set acquired by the computer device in the embodiment of the present application, and the rule set 20R may include M general quantization rules, where M is a positive integer, and one quantization rule corresponds to one structure to be quantized.
  • M is a positive integer
  • the number of quantization rules in the rule set 20R may be 7, for example, and may specifically include quantization rule R1, quantization rule R2, quantization rule R3, quantization rule R4, quantization rule R5, quantization rule R6, and quantization rule R7.
  • quantization rules R1, quantization rule R2 and quantization rule R3 are all quantization rules designed for the network layer with weights, which is the structure to be quantized.
  • the network layer with weights here is a network layer with weighted operations, such as convolution layer (conv), transposed convolution layer (e.g., transpose conv) and general matrix multiplication (e.g., gemm); when the network layer with weights is a convolution layer, quantization rules R4 and quantization rules R5 can be further designed.
  • a quantization rule R1 corresponding to the structure G1 may indicate that a forged operator is added at the input position of the network layer with weights and at multiple output positions of the network layer with weights, and the weights of the network layer with weights are quantized using the forged operator.
  • the structure G1 is an attention structure
  • the structure may include a convolution operator (conv operator), a matrix reshape operator (reshape operator), a permutation operator (permute operator), and a bmm matrix multiplication operator.
  • the quantization rule R2 may indicate that while retaining high-precision calculations, a pseudo operator is added at the input position of the network layer with weights, and the weights of the network layer with weights are quantized using the pseudo operator; while retaining low-precision calculations, pseudo operators are added at the input position of the network layer with weights and at the output position of the network layer with weights, respectively, and the weights of the network layer with weights are quantized using the pseudo operator.
  • the network layer with weights is generally followed by an activation layer. If a forged operator is added at the output position, unacceptable precision loss will be caused while retaining high-precision calculations.
  • the quantization rule R2 corresponding to the structure G1 can add forged operators at the input position of the network layer with weights and at the output position of the network layer with weights, respectively, and use the forged operator to quantize the weights of the network layer with weights.
  • the quantization rule R3 corresponding to the structure G3 can add a forged operator at the input position of the network layer with weights, and use the forged operator to quantize the weights of the network layer with weights, that is, the forged operator may not be added at the output position of the network layer with weights.
  • the structure composed of a convolutional layer, a normalization layer (e.g., BN) and an activation layer (e.g., ReLU) can automatically fuse the operators of the original network layer during the model reasoning stage. Therefore, the quantization rule R4 corresponding to the structure G4 can indicate that a fake operator is added at the input position of the convolutional layer, and the fake operator is used to quantize the weights of the convolutional layer.
  • the accuracy of the input of the non-weighted operation of the superimposed layer determines the output accuracy of the operator after the structure is fused.
  • the quantized operator after the output of the superimposed layer will not be fused, which will lead to the process of accuracy conversion of the fused operator, thereby affecting the speed.
  • the superimposed layer includes the first input branch (with the weighted operation
  • the quantization rule R5 corresponding to the structure G5 may indicate that when a high speed needs to be maintained, a forged operator is added at the input position of the convolution layer, at the input position of the second input branch of the superimposed layer, and at the output position of the superimposed layer, and the weight of the convolution layer is quantized by the forged operator; and when there is no requirement for speed, a forged operator may be added at the input position of the convolution layer and at the output position of the superimposed layer, and the weight of the convolution layer is quantized by the forged operator, that is, the forged operator may not be added at the input position of the
  • the output branch of the superimposed layer must be quantized.
  • the quantization rule R6 corresponding to the structure G6 may indicate that a counterfeit operation is added at the two input positions of the superimposed layer and at the output position of the superimposed layer, respectively. son.
  • the quantization operator i.e., the Q operator
  • the dequantization operator i.e., the DQ operator
  • the forged operator is used at each output position of the splicing layer (for example, Concat)
  • this will make the calculation after the splicing layer belong to the second precision.
  • the splicing operator corresponding to the splicing layer is also quantized, it is necessary to make the quantization operator in the forged operator added at the two output branches the same.
  • the quantization rule R7 corresponding to the structure G7 can indicate that at the output position of each output branch of the splicing layer, a forged operator with the same quantization operator is added respectively, where N is a positive integer greater than 1, and the embodiment of the present application can take 2 as an example.
  • the quantization priority of the specified network layer of the business object has the highest quantization priority.
  • the computer device can first filter out the quantization rules (e.g., quantization rule R3, quantization rule R4) corresponding to the structure to be quantized including the specified network layer, obtain the filtered first rule set, and then based on the filtered first rule set, detect whether there is an accelerable structure to be quantized (i.e., the same target structure as the 5 structures after filtering) in the model structure diagram of the initial model.
  • the quantization rules e.g., quantization rule R3, quantization rule R4
  • Figure 3 is a flow chart of a data processing method provided in an embodiment of the present application.
  • the method can be executed by a computer device with a model quantization function, and the computer device can be a terminal device (for example, any terminal device in the terminal device cluster shown in Figure 1 above, for example, terminal device 100b, or a server (for example, server 10S shown in Figure 1 above), which is not limited here.
  • the embodiment of the present application takes the method as an example of a computer device with a model quantization function, and the method can at least include the following steps S101-step S103:
  • Step S101 When the computer device obtains the initial model, it performs a structural search on the initial model to obtain a model structure diagram of the initial model.
  • the computer device when it obtains the initial model to be optimized, it can call a general quantization tool, perform a structural search on the initial model based on the search graph tool in the general quantization tool, and then obtain a model structure graph of the initial model.
  • Step S102 The computer device quantizes the target structure using a pseudo operator for quantization according to the type of the target structure in the model structure diagram.
  • the counterfeit operator here may include a quantization operator and an anti-quantization operator.
  • the type of the target structure here may be a first structure, a second structure or a third structure.
  • the first structure here refers to a structure including a network layer with a weight.
  • the network layer with a weight in the target structure of the initial model may be referred to as a first network layer with a weight.
  • the structures G1, G2, G3, G4 and G5 shown in Figure 2 above all belong to the first structure.
  • the second structure here (for example, the structure G6 shown in Figure 2 above) includes an overlay layer whose input branch type is an input type of a non-weighted operation.
  • the overlay layer whose input branch types included in the target structure of the initial model are all input types of non-weighted operations may be referred to as a first overlay layer.
  • the third structure here (for example, the structure G7 shown in Figure 2 above) includes a splicing layer with the same number of input branches and output branches.
  • the splicing layer with the same number of input branches and output branches included in the target structure of the initial model may be referred to as a first splicing layer.
  • the computer device can obtain M general quantization rules designed in advance based on reasoning experience, and then the rule set composed of these M quantization rules can be determined as the initial rule set (for example, the rule set 20R shown in Figure 2 above).
  • M is a positive integer.
  • the computer device also needs to obtain the business configuration of the business object for the initial model.
  • the business configuration here refers to the configuration information set by the business object (for example, the user) for the initial model, such as the business platform to be deployed subsequently by the initial model, the designated network layer specified by the business object that does not participate in the quantization, etc., which will not be limited here.
  • the initial rule set includes M quantization rules, and one quantization rule corresponds to a structure to be quantized, the initial rule set is related to the M structures to be quantized. Further, the computer device can determine the quantization rule set ultimately used for model quantization based on the business configuration and the initial rule set.
  • the computer device needs to query the specified network layer in the M structures to be quantified to obtain the query result. If the query result indicates that there is a structure to be quantified including the specified network layer in the M structures to be quantified, the computer device can determine the queried structure to be quantified as a filtering structure, and then delete the quantization rule corresponding to the filtering structure in the initial rule set, and determine the deleted initial rule set as the quantization rule set.
  • the computer device needs to query the activation layer in the seven structures to be quantified shown in FIG2 to obtain the query result. Since the structure G3 and the structure G4 shown in FIG2 both include activation layers, the query result indicates that there is a structure to be quantified that includes the designated network layer among the seven structures to be quantified.
  • the computer device can determine the structure G3 and the structure G4 as filter structures respectively, and then delete the quantization rules corresponding to the filter structures (i.e., quantization rules R3 and quantization rules R4) in the rule set 20R, and then determine the deleted rule set 20R as the quantization rule set.
  • the quantization rule set here can include FIG2 2 shows the quantization rules R1, R2, R5, R6 and R7.
  • the computer device can directly determine the initial rule set as the quantization rule set.
  • the computer device can directly determine the rule set 20R shown in Figure 2 as the quantization rule set.
  • the quantization rule set here can include quantization rule R1, quantization rule R2, quantization rule R3, quantization rule R4, quantization rule R5, quantization rule R6 and quantization rule R7 shown in Figure 2.
  • the quantization rule set may include N quantization rules, and one quantization rule corresponds to a structure to be quantized
  • the computer device needs to use a depth-first search (DFS) in the model structure diagram to find all target structures that meet the N structures to be quantized in the above quantization rule set, and then according to the type of the target structure in the model structure diagram, the target quantization rule matching the target structure can be determined from the N quantization rules, and the adding position of the forged operator in the initial model can be determined, and then the forged operator can be added in the determined adding position to achieve quantization of the target structure.
  • DFS depth-first search
  • the computer device can add a pseudo operator for quantization at the input position of the first network layer with weights, and use the pseudo operator to quantize the weights of the first network layer with weights.
  • the computer device In the case where the first network layer with weights has multiple output positions, the computer device also needs to add pseudo operators for quantization at the multiple output positions of the first network layer with weights, respectively.
  • the computer device needs to obtain a quantization rule that matches the target structure from the quantization rule set, and then the matched quantization rule (for example, the quantization rule R1 shown in FIG. 2 above) can be determined as the target quantization rule. Then, based on the quantization rule R1, the computer device can determine the input position of the first network layer with a weight, the output position of the first network layer with a weight, and the weight position of the first network layer with a weight as the adding position, and then a forged operator can be added at the adding position. Among them, the forged operator added at each adding position can be the same or different, and it will not be limited here.
  • the computer device when the first network layer with weights has an output position and the output position of the first network layer with weights is not connected to an activation layer, the computer device also needs to add a pseudo operator for quantization at the output position of the first network layer with weights.
  • the computer device may not add a pseudo algorithm for quantization at the output position of the first network layer with weights. son.
  • FIG. 4 is a schematic diagram of quantizing a target structure provided in an embodiment of the present application.
  • the structure 40G may be a target structure found by the computer device in the model structure diagram of the initial model, and the structure 40G includes a first network layer with a weight (e.g., a convolutional layer), and the first network layer with a weight has an output position, and the output position is not connected to an activation layer.
  • a weight e.g., a convolutional layer
  • the first network layer with weights in the structure 40G can be used to perform feature extraction processing on X, thereby obtaining the output of the first network layer with weights (i.e., the output of the structure 40G, e.g., Y). If X here belongs to the first precision, then the output of the structure 40G is also the first precision. Furthermore, when quantizing the structure 40G, the computer device can search for a quantization rule that matches the structure 40G in the quantization rule set, and then determine the found quantization rule as the target quantization rule (e.g., the quantization rule R2 shown in FIG2 above).
  • the target quantization rule e.g., the quantization rule R2 shown in FIG2 above.
  • the computer device can determine the input position of the first network layer with weight as position A1, the weight position in the first network layer with weight as position A2, and the output position of the first network layer with weight as position A3 based on the target quantization rule, and then the position A1, position A2 and position A3 can be respectively determined as the adding positions of the forged operator in the structure 40G.
  • the computer device may add a forged operator (e.g., forged operator 4F1) at position A1, so that X is converted to the second precision after passing through the quantization operator in forged operator 4F1, and then restored to the first precision after passing through the inverse quantization operator in forged operator 4F1.
  • the computer device may also add a forged operator at position A2 to quantize the weight of the first network layer with weights, so that the weight of the first network layer with weights is converted to the second precision after passing through the quantization operator in forged operator 4F2, and then restored to the first precision after passing through the inverse quantization operator in forged operator 4F2.
  • the computer device may also add a forged operator (e.g., forged operator 4F3) at position A3, so that the output of the first network layer with weights is converted to the second precision after passing through the quantization operator in forged operator 4F3, and then restored to the first precision after passing through the inverse quantization operator in forged operator 4F3.
  • a forged operator e.g., forged operator 4F3
  • the embodiment of the present application detects a target structure such as structure 40G, it can automatically quantize structure 40G through quantization rules and forgery operators (including forgery operators 4F1, 4F2 and 4F3) that match structure 40G to obtain the quantized structure 40G (for example, the quantized structure 41G shown in Figure 4).
  • quantization rules and forgery operators including forgery operators 4F1, 4F2 and 4F3 that match structure 40G to obtain the quantized structure 40G (for example, the quantized structure 41G shown in Figure 4).
  • the target structure includes a first network layer with weights, a normalization layer, and an activation layer connected in sequence, and when the first network layer with weights is a convolutional layer, such a target structure can be subsequently fused. Therefore, the embodiment of the present application can also quantize such a target structure, that is, the computer device can obtain a quantization rule that matches the target structure from the quantization rule set, and then can quantize the matching The quantization rule (for example, the quantization rule R4 shown in FIG2 ) assigned to the target quantization rule is determined as the target quantization rule. Then, the computer device can add a pseudo operator at the input position of the first network layer with weights based on the quantization rule R4, and use the quantization operator to quantize the weights of the first network layer with weights.
  • the quantization rule for example, the quantization rule R4 shown in FIG2
  • Figure 5 is a schematic diagram 2 of quantizing a target structure provided in an embodiment of the present application.
  • the structure 50G can be the target structure found by the computer device in the model structure diagram of the initial model, and the structure 50G can include a convolution layer, a normalization layer, and an activation layer connected in sequence.
  • the precision of the convolution layer, normalization layer, and activation layer in the structure 50G are all the first precision (e.g., FP32)
  • the output (e.g., Y) obtained after the convolution calculation, normalization calculation, and activation calculation of the input corresponding to the structure 50G is still the first precision.
  • the computer device can search for a quantization rule that matches the structure 50G in the quantization rule set according to the type of the structure 50G, and then determine the found quantization rule as the target quantization rule (e.g., the quantization rule R4 shown in FIG2 above).
  • a quantization rule that matches the structure 50G in the quantization rule set according to the type of the structure 50G, and then determine the found quantization rule as the target quantization rule (e.g., the quantization rule R4 shown in FIG2 above).
  • the computer device can determine the input position of the convolution layer in structure 50G as position B1 and the weight position of the convolution layer in structure 50G as position B2 based on the target quantization rule, and then the position B1 and position B2 can be respectively determined as the adding positions of the forged operator in structure 50G.
  • the computer device may add a forged operator (e.g., forged operator 5F1) at position B1, so that the input belonging to the first precision is converted to the second precision after passing through the quantization operator in forged operator 5F1, and then restored to the first precision after passing through the inverse quantization operator in forged operator 5F1.
  • the computer device may also use forged operator 5F2 to quantize the weights of the convolution layer, that is, add forged operator 5F2 at position B2, so that the weight belonging to the first precision is converted to the second precision after passing through the quantization operator in forged operator 5F2, and then restored to the first precision after passing through the inverse quantization operator in forged operator 5F2.
  • the forged operator 5F1 and the forged operator 5F2 may be the same or different, and will not be limited here.
  • the embodiment of the present application detects a target structure such as structure 50G, it can automatically quantize structure 50G through quantization rules and forgery operators (including forgery operators 5F1 and 5F2) that match structure 50G to obtain the quantized structure 50G (for example, the quantized structure 51G shown in Figure 5).
  • the computer device may add a forged operator at the output position of the overlay layer, and may also add a forged operator at the input position of the second input branch.
  • the first input branch here includes a first network layer and a normalization layer connected in sequence with weights, the normalization layer is connected to the overlay layer, the type of the second input branch is an input type of a non-weighted operation, and the first network layer with weights is a convolutional layer.
  • the computer device when the first network layer with weights is a convolution layer, and the output position of the convolution layer is connected to the normalization layer, and the output position of the normalization layer is connected to the superposition layer, the computer device not only needs to add a pseudo operator for quantization at the input position of the first network layer with weights, and use the pseudo operator to quantize the weights of the first network layer with weights, but also needs to add a pseudo operator at the output position of the superposition layer.
  • the computer device since the accuracy of the second input branch determines the output accuracy of the fused network layer with weights, in order to increase the speed, the computer device can also add a pseudo operator at the input position of the second input branch.
  • Figure 6 is a schematic diagram of quantizing a target structure provided in an embodiment of the present application.
  • the structure 60G can be the target structure found by the computer device in the model structure diagram of the initial model, and the structure 60G is a jump link structure associated with the superposition layer.
  • the first input branch of the superposition layer may include a convolutional layer and a normalization layer connected in sequence, and the type of the second input branch is the input type of a non-weighted operation, that is, the input X2 corresponding to the second input branch can be an input of a non-weighted operation.
  • the superposition layer in the structure 60G can superimpose the first input (e.g., X1) and the second input (e.g., X2) after convolution calculation and normalization calculation to obtain the output of the structure 60G (e.g., Y). Since the first input and the second input are both of the first precision, the output of the structure 60G is still of the first precision.
  • the computer device can search for a quantization rule that matches the structure 60G in the quantization rule set, and then determine the found quantization rule as the target quantization rule (e.g., the quantization rule R5 shown in FIG2 above).
  • the computer device can determine the input position of the convolution layer in structure 60G as position C1, and the weight position of the convolution layer as position C2, the input position of the second input branch as position C3, and the output position of the superposition layer as position C4 based on the target quantization rule, and then the position C1, position C2, position C3 and position C4 can be respectively determined as the addition positions of the forged operator in the initial model.
  • the computer device may add a forged operator (e.g., forged operator 6F1) at position C1, so that the first input (e.g., X1) belonging to the first precision is converted to the second precision after passing through the quantization operator in the forged operator 6F1, and then restored to the first precision after passing through the dequantization operator in the forged operator 6F1.
  • a forged operator e.g., forged operator 6F1
  • the computer device may also add a forged operator (e.g., forged operator 6F2) at position C2, and use the forged operator 6F2 to quantize the weights of the convolution layer, so that the weights belonging to the first precision are converted to the second precision after passing through the quantization operator in the forged operator 6F2, and then restored to the first precision after passing through the dequantization operator in the forged operator 6F2.
  • a forged operator e.g., forged operator 6F2
  • the computer device may also add a forged operator (e.g., forged operator 6F3) at position C3, so that the second input (e.g., X2) belonging to the first precision is converted to the second precision after passing through the quantization operator in the forged operator 6F3, and then restored to the first precision after passing through the dequantization operator in the forged operator 6F2.
  • the computer device can also add a forged operator at position C4 (for example, the forged operator 6F4) is used to convert the output of the superimposed layer into the second precision after passing through the quantization operator in the forged operator 6F4, and then restore it to the first precision after passing through the inverse quantization operator in the forged operator 6F4.
  • the forged operators added at each adding position here may be the same or different, and will not be limited here.
  • the embodiment of the present application detects a target structure such as structure 60G, it can automatically quantize structure 60G through quantization rules and forgery operators (including forgery operators 6F1, forgery operators 6F2, forgery operators 6F3, and forgery operators 6F4) that match structure 60G to obtain the quantized structure 60G (for example, the quantized structure 61G shown in Figure 6).
  • quantization rules and forgery operators including forgery operators 6F1, forgery operators 6F2, forgery operators 6F3, and forgery operators 6F4
  • the target structure includes a first superimposed layer
  • the type of the first input branch of the first superimposed layer and the type of the second input branch of the first superimposed layer are both input types of non-weighted operations
  • the computer device needs to add a pseudo operator for quantization at the input position of the first input branch, add a pseudo operator at the input position of the second input branch, and add a pseudo operator at the output position of the first superimposed layer.
  • Figure 7 is a schematic diagram 4 of quantizing a target structure provided in an embodiment of the present application.
  • the structure 70G can be the target structure found by the computer device in the model structure diagram of the initial model, and the structure 70G can be an overlay including two input branches and one output branch.
  • the superposition processing of X1 and X2 can be performed through the superposition layer in the structure 70G, thereby obtaining the output of the superposition layer (i.e., the output of the structure 70G, e.g., Y). If both X1 and X2 here belong to the first precision, then the output of the structure 70G is also the first precision.
  • the computer device can search for a quantization rule that matches the structure 70G in the quantization rule set, and then the found quantization rule can be determined as the target quantization rule for quantizing the structure 70G (e.g., the quantization rule R6 shown in FIG2 above).
  • the computer device can determine the input position of the first input branch of the superimposed layer (i.e., the input position of X1) as position D1, the input position of the second input branch of the superimposed layer (i.e., the input position of X2) as position D2, and the output position of the superimposed layer as position D3 based on the target quantization rule, and then the position D1, position D2 and position D3 can be respectively determined as the adding positions of the forged operator in the structure 70G.
  • the input position of the first input branch of the superimposed layer i.e., the input position of X1
  • the input position of the second input branch of the superimposed layer i.e., the input position of X2
  • the output position of the superimposed layer as position D3 based on the target quantization rule
  • the computer device may add a pseudo operator (e.g., pseudo operator 7F1) at position D1, so that the first input (e.g., X1) input to structure 71G is converted to the second precision after passing through the quantization operator in pseudo operator 7F1, and then restored to the first precision after passing through the dequantization operator in pseudo operator 7F1.
  • the computer device may also add a pseudo operator (e.g., pseudo operator 7F2) at position D2, so that the second input (e.g., X2) input to structure 71G is converted to the second precision after passing through the quantization operator in pseudo operator 7F2, and then restored to the first precision after passing through the dequantization operator in pseudo operator 7F2.
  • the computer device may also add a pseudo operator at position D3.
  • a forged operator (for example, forged operator 7F3) is formed so that the output of the superimposed layer is converted to the second precision after passing through the quantization operator in the forged operator 7F3, and then restored to the first precision after passing through the inverse quantization operator (for example, inverse quantization operator 2) in the forged operator 7F3.
  • the inverse quantization operator for example, inverse quantization operator 2
  • the embodiment of the present application detects a target structure such as structure 70G, it can automatically quantize structure 70G through quantization rules and forgery operators (including forgery operators 7F1, forgery operators 7F2 and forgery operators 7F3) that match structure 70G to obtain the quantized structure 70G (for example, the quantized structure 71G shown in Figure 7).
  • quantization rules and forgery operators including forgery operators 7F1, forgery operators 7F2 and forgery operators 7F3
  • the target structure includes a first splicing layer, and the number of input branches of the first splicing layer is the same as the number of output branches of the first splicing layer
  • the computer device can add a pseudo operator for quantization at the output position of each of the N output branches of the first splicing layer; wherein N here can be a positive integer greater than 1, and in order to be able to optimize the structure later, the quantization operators of the pseudo operators added at each output branch here are the same.
  • Figure 8 is a schematic diagram 5 of quantizing a target structure provided in an embodiment of the present application.
  • the structure 80G can be the target structure found by the computer device in the model structure diagram of the initial model.
  • the structure 80G can include a splicing layer, and the number of input branches of the splicing layer is the same as the number of output branches.
  • the embodiment of the present application can take two input branches and two output branches as an example.
  • the concatenation layer in the structure 80G can concatenate the inputs of the two input branches (e.g., X1 and X2) to obtain the output of the concatenation layer (e.g., Y1 and Y2). If both X1 and X2 here belong to the first precision, then both outputs of the structure 80G are of the first precision. It can be understood that when quantizing the structure 80G, the computer device can search for a quantization rule that matches the structure 80G in the quantization rule set, and then determine the found quantization rule as the target quantization rule (e.g., the quantization rule R7 shown in FIG2 above).
  • the target quantization rule e.g., the quantization rule R7 shown in FIG2 above.
  • the computer device can determine the output position of the first output branch of the splicing layer as position E1 and the output position of the second output branch of the splicing layer as position E2 based on the target quantization rule, and then the position E1 and position E2 can be respectively determined as the addition positions of the forged operator in the structure 80G.
  • the computer device may add a fake operator (e.g., fake operator 8F1) at position E1, so that the first output of the splicing layer is converted to the second precision after passing through the quantization operator in the fake operator 8F1, and then restored to the first precision after passing through the inverse quantization operator (e.g., inverse quantization operator 1) in the fake operator 8F1.
  • the computer device may also add a fake operator (e.g., fake operator 8F2) at position E2, so that the second output of the splicing layer is converted to the second precision after passing through the quantization operator in the fake operator 8F2, and then restored to the first precision after passing through the inverse quantization operator (e.g., inverse quantization operator 2) in the fake operator 8F2.
  • the fake operator 8F1 is the same as the quantization operator in the fake operator 8F2, and the inverse quantization operators in the fake operator 8F1 and the fake operator 8F2 may be the same or different. The same will not be limited here.
  • the embodiment of the present application detects a target structure such as structure 80G, it can automatically quantize structure 80G through quantization rules and forgery operators (including forgery operators 8F1 and 8F2) that match structure 80G to obtain the quantized structure 80G (for example, the quantized structure 81G shown in Figure 8).
  • Step S103 The computer device obtains a service model for service processing based on the quantized initial model.
  • the computer device can quantize various structures to be quantized that can be accelerated in the model structure diagram, and after the quantization is completed, the computer device can determine the quantized initial model as the first model. In order to make the accuracy of the quantized model meet the business requirements, the computer device can also fine-tune the first model to obtain a second model, and then obtain a business model for business processing based on the second model.
  • the computer device can obtain sample business data and sample business labels corresponding to the sample business data, and then input the sample features of the sample business data into the first model, and predict the sample business data through the forged operator added in the first model and each network layer in the first model to obtain the predicted business label of the sample business data.
  • the computer device can fine-tune the first model based on the sample business label and the predicted business label to obtain the second model, and then optimize the second model to obtain a business model for business processing.
  • the sample business data here is determined based on the business processing performed by the initial model.
  • the sample business data here can be a face/non-face training set generated by using local slicing, flipping, Gaussian blurring, etc. for face images in a certain face database (e.g., face dataset (Annotated Facial Landmarks in the Wild)).
  • face database e.g., face dataset (Annotated Facial Landmarks in the Wild)
  • the sample business label here can be used to indicate the true category of the sample business data (e.g., face or non-face).
  • the face (or other biometric) recognition technology involved when the above embodiments of the present application are applied to specific products or technologies, the relevant data collection, use and processing processes should comply with the requirements of national laws and regulations. Before collecting facial information, the information processing rules should be informed and the separate consent of the target object should be sought. The facial information should be processed in strict accordance with the requirements of laws and regulations and the personal information processing rules, and technical measures should be taken to ensure the security of relevant data.
  • the sample business data here may be the acquired multimedia data (for example, video, audio, text, picture, etc.).
  • the sample business label here may be used to indicate the actual impact of the content quality of the sample business data. The higher the impact, the higher the content quality of the sample business data.
  • the sample business data here may be the acquired image data.
  • the sample business label here may be used to indicate the true category of the sample business data (for example, of course, the business processing of the initial model may also include other processing, which will not be exemplified one by one here.
  • the implementation of the data capture technology solution involved for example, multimedia data captured during content review processing or image data captured during image recognition processing.
  • the relevant data collection, use and processing processes should comply with the requirements of national laws and regulations, conform to the principles of legality, legitimacy and necessity, do not involve the acquisition of data types prohibited or restricted by laws and regulations, and will not hinder the normal operation of the target website.
  • the embodiment of the present application needs to optimize the structure to be optimized in the second model so that as many operators as possible in the second model are integrated to obtain a business model.
  • the second model is a quantized initial model that has been fine-tuned and trained
  • the model structure diagram of the second model and the model structure diagram of the first model are the same as the model structure diagram of the initial model.
  • the embodiment of the present application can also refer to the network layer with weights in the second model as the first network layer with weights, and so on.
  • the superposition layer whose input branches in the second model are all non-weighted operation input types is called the first superposition layer;
  • the splicing layer with the same number of input branches and output branches in the second model is called the first splicing layer.
  • the structure to be optimized in the second model also includes a first network layer with a weight.
  • the business model finally obtained based on the second model may include a network layer with a weight (i.e., the second network layer) and a quantization operator added at the input position of the second network layer with a weight; in the business model, the second network layer with a weight is obtained based on the first network layer with a weight, and the weight of the second network layer with a weight is obtained by processing the weight of the first network layer with a weight in the second model using a quantization operator.
  • FIG. 9 is a schematic diagram 1 of processing a structure to be optimized in the second model provided in an embodiment of the present application.
  • the structure 91G here may be the structure to be optimized in the second model, and the structure 91G is obtained by the computer device after quantizing the target structure in the initial model and fine-tuning the training, and the structure 91G is the same as the structure 41G shown in FIG. 4 above.
  • the structure 91G may include a pseudo operator F1 (i.e., a pseudo operator composed of a quantization operator 1 and an inverse quantization operator 1), a pseudo operator F2 (i.e., a pseudo operator composed of a quantization operator 2 and an inverse quantization operator 2), and a pseudo operator F3 (i.e., a pseudo operator composed of a quantization operator 3 and an inverse quantization operator 3).
  • a pseudo operator F1 i.e., a pseudo operator composed of a quantization operator 1 and an inverse quantization operator 1
  • a pseudo operator F2 i.e., a pseudo operator composed of a quantization operator 2 and an inverse quantization operator 2
  • a pseudo operator F3 i.e., a pseudo operator composed of a quantization operator 3 and an inverse quantization operator 3
  • the computer device when it optimizes the structure 91G, it can obtain the second network layer with weights according to the first network layer with weights in the structure 91G, and then use a quantization operator to process the weights of the first network layer with weights in the structure 91G to obtain the weights of the second network layer with weights.
  • the computer device can filter out inverse quantization operator 1 and inverse quantization operator 2, and use quantization operator 2 and quantization operator 3 to process the weights of the first network layer with weights in structure 91G to obtain the weights of the second network layer with weights.
  • the computer device optimizes the structure 91G in the second model, it can obtain the optimized structure in the business model (i.e., the structure 92G shown in FIG9 ).
  • the structure 92G may include a second network layer with a weight, and a quantization operator (e.g., quantization operator 1) added at the input position of the second network layer with a weight.
  • the computer device when the first network layer with weights in the target structure of the initial model is a convolutional layer, and the target structure also includes a normalization layer connected to the convolutional layer and an activation layer connected to the normalization layer, the computer device also needs to fuse the first network layer with weights, the normalization layer and the activation layer to obtain a second network layer with weights.
  • Figure 10 is a schematic diagram 2 of processing a structure to be optimized in the second model provided in an embodiment of the present application.
  • the structure 101G here can be the structure to be optimized in the second model, and the structure 101G is obtained after the computer device quantizes the target structure in the initial model and fine-tunes the training.
  • the first network layer with weights in the target structure here is a convolution layer, and the target structure also includes a normalization layer connected to the convolution layer and an activation layer connected to the normalization layer.
  • the structure 101G is the same as the structure 51G shown in Figure 5 above.
  • the structure 101G may include a forged operator F1 (i.e., a forged operator composed of a quantization operator 1 and an inverse quantization operator 1) and a forged operator F2 (i.e., a forged operator composed of a quantization operator 2 and an inverse quantization operator 2).
  • a forged operator F1 i.e., a forged operator composed of a quantization operator 1 and an inverse quantization operator 1
  • a forged operator F2 i.e., a forged operator composed of a quantization operator 2 and an inverse quantization operator 2
  • the computer device can obtain the fused convolution layer in the structure 102G according to the convolution layer in the structure 101G.
  • the computer device can fuse the convolution layer, the normalization layer, and the activation layer in the structure 101G to obtain the fused convolution layer (i.e., the second network layer with weights).
  • the computer device can also use a quantization operator to process the weights of the convolution layer in structure 101G to obtain the weights of the fused convolution layer.
  • a quantization operator For example, the computer device can filter out inverse quantization operator 1 and inverse quantization operator 2, and use quantization operator 2 to process the weights of the convolution layer in structure 101G to obtain the weights of the fused convolution layer.
  • the computer device optimizes the structure 101G in the second model, it can obtain the optimized structure in the business model (i.e., the structure 102G shown in FIG10).
  • the structure 102G may include a second network layer with a weight, and a quantization operator (e.g., quantization operator 1) added at the input position of the second network layer with a weight.
  • the target structure includes an overlay layer, a first input branch of the overlay layer, and a second input branch of the overlay layer.
  • the computer device can also fuse the first network layer with weights, the normalization layer and the superposition layer to obtain a second network layer with weights.
  • the normalization layer is connected to the superposition layer
  • the type of the second input branch is the input type of a non-weighted operation
  • the first network layer with weights is a convolutional layer.
  • Figure 11 is a schematic diagram three of processing a structure to be optimized in the second model provided in an embodiment of the present application.
  • the structure 111G here can be the structure to be optimized in the second model, and the structure 111G is obtained by the computer device quantizing the target structure in the initial model and fine-tuning the training.
  • the target structure here includes an overlay layer, a first input branch of the overlay layer, and a second input branch of the overlay layer.
  • the first input branch includes a first network layer and a normalization layer with weights connected in sequence, and the first network layer with weights is a convolutional layer, and the normalization layer is connected to the overlay layer.
  • the type of the second input branch is an input type of a non-weighted operation.
  • the structure 111G is the same as the structure 61G shown in Figure 6 above.
  • the structure 111G may include a fake operator F1 (i.e., a fake operator composed of a quantization operator 1 and an inverse quantization operator 1), a fake operator F2 (i.e., a fake operator composed of a quantization operator 2 and an inverse quantization operator 2), a fake operator F3 (i.e., a fake operator composed of a quantization operator 3 and an inverse quantization operator 3), and a fake operator F4 (i.e., a fake operator composed of a quantization operator 4 and an inverse quantization operator 4).
  • a fake operator F1 i.e., a fake operator composed of a quantization operator 1 and an inverse quantization operator 1
  • a fake operator F2 i.e., a fake operator composed of a quantization operator 2 and an inverse quantization operator 2
  • a fake operator F3 i.e., a fake operator composed of a quantization operator 3 and an inverse quantization operator 3
  • a fake operator F4 i.e
  • the computer device when the computer device optimizes the structure 111G, it can obtain the fused convolution layer in the structure 112G according to the convolution layer in the structure 111G.
  • the computer device can fuse the convolution layer, the normalization layer, and the activation layer in the structure 111G to obtain the fused convolution layer (i.e., the second network layer with weights).
  • the computer device can also use a quantization operator to process the weights of the convolution layer in structure 111G to obtain the weights of the fused convolution layer.
  • a quantization operator can filter out inverse quantization operator 1, inverse quantization operator 2, and inverse quantization operator 3, and use quantization operator 2 and quantization operator 4 to process the weights of the convolution layer in structure 111G to obtain the weights of the fused convolution layer.
  • the computer device optimizes the structure 111G in the second model, it can obtain the optimized structure in the business model (i.e., the structure 112G shown in FIG11).
  • the structure 112G may include a second network layer with a weight, and a quantization operator (e.g., quantization operator 1 and quantization operator 3) added at the input position of the second network layer with a weight.
  • a quantization operator e.g., quantization operator 1 and quantization operator 3
  • the business model finally obtained based on the second model may include the second superimposed layer, the quantization operator added to the first input branch of the second superimposed layer, and the quantization operator added to the second input branch of the second superimposed layer.
  • the second stacked layer is obtained according to the quantization operator and the first stacked layer in the second model.
  • Figure 12 is a schematic diagram 4 of processing a structure to be optimized in the second model provided in an embodiment of the present application.
  • the structure 121G here can be the structure to be optimized in the second model, and the structure 121G is obtained by the computer device quantizing the target structure in the initial model and fine-tuning the training.
  • the target structure here can include a first superimposed layer, and the type of the first input branch of the first superimposed layer and the type of the second input branch of the first superimposed layer are both input types of non-weighted operations.
  • the structure 121G can be the same as the structure 71G shown in Figure 7 above.
  • the structure 121G may include a forged operator F1 (i.e., a forged operator composed of a quantization operator 1 and an inverse quantization operator 1), a forged operator F2 (i.e., a forged operator composed of a quantization operator 2 and an inverse quantization operator 2), and a forged operator F3 (i.e., a forged operator composed of a quantization operator 3 and an inverse quantization operator 3).
  • the second superposition layer in the structure 122G can be obtained according to the quantization operator and the first superposition layer in the second model.
  • the computer device can filter out the inverse quantization operator 1 and the inverse quantization operator 2, and use the quantization operator 3 to process the first superposition layer to obtain the second superposition layer.
  • the structure 122G may include a second superimposed layer, a quantization operator added at the first input branch of the second superimposed layer (i.e., quantization operator 1), and a quantization operator added at the second input branch of the second superimposed layer (i.e., quantization operator 2).
  • the embodiments of the present application can appropriately move the positions of the quantization operator and the dequantization operator according to the equivalent propagation rule so as to fuse as many operators of the model to be optimized as possible.
  • the business model finally obtained based on the second model may include the second splicing layer and the quantization operator added at the input position of each input branch of the second splicing layer.
  • the quantization operator at the input position of each input branch of the second splicing layer is obtained by equivalently propagating the quantization operator at the output position of each output branch of the first splicing layer added in the second model to the input position of each input branch of the first splicing layer in the second model.
  • FIG. 13 is a schematic diagram of processing a structure to be optimized in the second model provided by an embodiment of the present application.
  • the structure 131G here can be The structure to be optimized 131G is obtained by the computer device after quantizing the target structure in the initial model.
  • the target structure may include a first splicing layer, and the number of input branches of the first splicing layer is the same as the number of output branches of the first splicing layer, and the number here is 2 as an example.
  • the structure 131G is the same as the structure 81G shown in FIG. 8 above.
  • the structure 131G can include a pseudo operator F1 (i.e., a pseudo operator composed of a quantization operator 1 and an inverse quantization operator 1) and a pseudo operator F2 (i.e., a pseudo operator composed of a quantization operator 2 and an inverse quantization operator 2), and the quantization operator 1 here is equal to the quantization operator 2.
  • a pseudo operator F1 i.e., a pseudo operator composed of a quantization operator 1 and an inverse quantization operator 1
  • a pseudo operator F2 i.e., a pseudo operator composed of a quantization operator 2 and an inverse quantization operator 2
  • the computer device when the computer device optimizes the structure 131G, it can propagate the position of the quantization operator 1 from the first output position of the first splicing layer to the first input position of the first splicing layer (i.e., the input position corresponding to X1). Similarly, it is also necessary to propagate the position of the quantization operator 2 from the second output position of the first splicing layer to the second input position of the first splicing layer (i.e., the input position corresponding to X2), thereby obtaining the structure 132G. At this time, the computer device can refer to the splicing layer in the structure 132G as the second splicing layer.
  • the computer device optimizes the structure 131G in the second model, it can obtain the optimized structure in the business model (i.e., the structure 132G shown in FIG13 ).
  • the structure 132G may include a second splicing layer and a quantization operator added at the input position of each input branch of the second splicing layer.
  • the computer device can convert the precision of X1 from the first precision to the second precision through the quantization operator 1, and convert the precision of X2 from the first precision to the second precision through the quantization operator 2, and then through the second splicing layer in the structure 132G, X1 belonging to the second precision and X2 belonging to the second precision can be spliced, so that the parameters obtained after the subsequent splicing process are still the second precision.
  • the business object corresponding to the initial model does not need to manually add a fake operator at the corresponding position in the initial model, but directly submits any initial model to the computer device, so that the computer device calls a universal quantization tool to obtain a pre-set set of quantization rules, and then based on the universal quantization tool and the set of quantization rules, the quantization of the quantifiable structure to be accelerated in the initial model is performed.
  • the entire quantization process can be completed within the universal quantization tool, which can effectively reduce labor costs, improve the model quantization speed, and achieve quantization universality.
  • the universal quantization tool also supports fine-tuning training of the quantized initial model, the subsequent business model can obtain the highest acceleration ratio while meeting the business indicators. Because the structure to be optimized in the second model is optimized, the subsequent business model can improve the query rate per second (QPS) of the model, and can also reduce the model deployment memory and video memory occupancy.
  • QPS query rate per second
  • FIG. 14 is a flow chart of a data processing method provided by an embodiment of the present application.
  • the method can be performed by a terminal device having a model quantization function (for example, the terminal device cluster shown in FIG. 1 above).
  • the method may be executed by any terminal device, for example, the terminal device 100a), or by a server with a model quantization function (for example, the server 10S shown in FIG. 1 ), or by a terminal device with a model application function and a server with a model quantization function interacting with each other, which is not limited here.
  • the method may include at least steps S201 to S209.
  • Step S201 when the initial model is acquired, a structural search is performed on the initial model to obtain a model structure diagram of the initial model.
  • Step S202 quantize the target structure using a pseudo operator for quantization according to the type of the target structure in the model structure diagram.
  • the counterfeit operator includes a quantization operator and an anti-quantization operator.
  • Step S203 determining a service model for service processing based on the quantized initial model.
  • the data processing method in the embodiment of the present application may include a model quantization process and a model application process.
  • steps S201 to S203 describe the model quantization process
  • the specific implementation of the model training process can refer to the description of steps S101 to S103 in the embodiment corresponding to FIG. 3 above, which will not be further described here.
  • Figure 15 is a schematic diagram of a scenario for optimizing a business model provided by an embodiment of the present application.
  • the computer device in the embodiment of the present application can be a computer device with a model quantization function, and the computer device can be any terminal device in the terminal device cluster shown in Figure 1 above, for example, terminal device 100b, and the computer device can also be the server 10S shown in Figure 1 above, and the computer device will not be limited here.
  • the model 151W shown in FIG15 is the initial model obtained by the computer device.
  • the computer device can call a universal quantization tool to perform a structural search on the model 151W to obtain a model structure diagram of the model 151W.
  • the computer device can query whether there is a target structure belonging to the N structures to be quantized in the model structure diagram of the model 151W based on the N structures to be quantized corresponding to the pre-set quantization rule set.
  • the computer device can determine the target quantization rule (for example, the quantization rule R4 shown in FIG. 2 above) that matches the structure 15G1 from the N quantization rules in the quantization rule set. Further, the computer device can quantize the structure 15G1 based on the target quantization rule and the forgery operator to obtain the quantized target structure (for example, the structure 15G2 shown in FIG. 15).
  • the specific implementation method of the computer device quantizing the structure 15G1 can refer to the specific implementation method of quantizing the structure 50G shown in FIG. 5 above, which will not be described in detail here.
  • the computer device can obtain the model 152W (i.e., the first model) shown in FIG15. Further, the computer device can fine-tune the model 152W, and then the fine-tune the trained model 152W can be called the second model. Then the computer device can traverse the structure to be optimized (i.e., the target structure that has been quantized and trained) in the second model, and then the optimized structure can be obtained according to the quantization operator in the structure to be optimized and the original network layer in the structure to be optimized.
  • the structure to be optimized i.e., the target structure that has been quantized and trained
  • the computer device traverses to the structure 15G2 in the second model, it can refer to the optimization processing method shown in FIG10 above, and optimize the structure 15G2 to obtain the optimized structure corresponding to the structure 15G2 (for example, the structure 15G3 shown in FIG15).
  • the computer device can determine the optimized second model as the third model, and then perform model inference on the third model based on the business platform deployed by the third model to obtain a business model for business processing (for example, model 153W shown in Figure 15).
  • a business model for business processing for example, model 153W shown in Figure 15.
  • the embodiment of the present application can automatically perform optimization processes such as model quantization, fine-tuning training and model reasoning on the initial model through a general quantization tool, so that the subsequently obtained business model can improve the model's query rate per second, and further reduce the model deployment memory and video memory occupancy.
  • the business model here can be applicable to a variety of application scenarios. Therefore, the business processing of the business model here can be key part detection processing, content review processing, or image recognition processing, which will not be limited here.
  • the embodiment of the present application can take image recognition as an example to illustrate the application process of the business model, wherein the model application process can be specifically described in the following steps S204-S208.
  • Step S204 If the business processing of the business model is image recognition, when an image to be recognized is obtained, image features of the image to be recognized are obtained.
  • the computer device when the computer device obtains the image to be recognized, it can perform encoding processing on the image to be recognized to obtain image features of the image to be recognized.
  • Step S205 input the image features into the business model.
  • the computer device can input the image features of the image to be identified into the business model.
  • the business model here is a model obtained after model quantization, fine-tuning training and inference processing of the high-precision initial model, this means that there is an optimized structure in the business model (for example, structure 15G3 shown in Figure 15).
  • the optimized structure here includes a target network layer and a quantization operator added at the input position of the target network layer (for example, quantization operator 1 before the input position of structure 15G3 shown in Figure 15); the target network layer is obtained based on the quantization operator and the original network layer of the target structure in the initial model.
  • Step S206 based on the quantization operator added at the input position of the target network layer, the precision of the input feature of the optimized structure is converted from the first precision to the second precision.
  • the computer device can perform a quantization operation on the input features of the optimization structure based on the quantization operator added at the input position of the target network layer to convert the precision of the input features from the first precision to the second precision.
  • the first precision is higher than the second precision, which means that the embodiment of the present application can convert the precision of the parameters input to the optimization structure from high precision to low precision through the quantization operator, thereby achieving the purpose of acceleration.
  • Step S207 performing feature extraction processing on the input features belonging to the second precision through the target network layer, and the target network layer outputs output features corresponding to the input features;
  • the output features of the target network layer are still the first precision.
  • Step S208 obtaining a predicted category of the image to be identified based on the output features.
  • the computer device can transmit the output features of the optimized structure (i.e., the output features of the target network layer) to the next network layer of the optimized structure in the business model until the business model predicts the influence of the image to be identified for multiple classification categories, and then selects the classification category corresponding to the maximum influence among the influences of these multiple classification categories as the predicted category of the image to be identified.
  • the output features of the optimized structure i.e., the output features of the target network layer
  • the computer device can predict through the business model that the influence of the image to be identified on category 1 is 15%, the influence on category 2 is 80%, and the influence on category 3 is 5%. Then, the computer device can use the classification category corresponding to the maximum influence (i.e., category 2) as the predicted category of the image to be identified.
  • category 1 e.g., human
  • category 2 e.g., cat
  • category 3 e.g., dog
  • the business object corresponding to the initial model does not need to manually add a fake operator at the corresponding position in the initial model, but directly submits any initial model to the computer device, so that the computer device calls a universal quantization tool to obtain a pre-set set of quantization rules, and then based on the universal quantization tool and the set of quantization rules, the quantization of the quantifiable structure to be accelerated in the initial model is performed.
  • the entire quantization process can be completed within the universal quantization tool, which can effectively reduce labor costs, improve the model quantization speed, and achieve quantization universality.
  • the universal quantization tool also supports fine-tuning training of the quantized initial model
  • the subsequent business model can obtain the highest acceleration ratio while meeting the business indicators, that is, the computer device can quickly and accurately perform business processing (for example, image recognition processing, etc.) through the business model.
  • the data processing device 1 may include: a search module 10, a quantization module 20, a business model determination module 30, a feature acquisition module 40, a feature input module 50, a precision conversion module 60, a feature extraction module 70 and a category determination module 80.
  • the search module 10 is used to perform a structural search on the initial model when the initial model is obtained to obtain the initial model.
  • Model structure diagram
  • the quantization module 20 is used to quantize the target structure according to the type of the target structure in the model structure diagram by using a pseudo operator for quantization; the pseudo operator includes a quantization operator and an anti-quantization operator.
  • the quantization module 20 includes: a first quantization unit 201 , a second quantization unit 202 , a third quantization unit 203 , a fourth quantization unit 204 , a fifth quantization unit 205 , a sixth quantization unit 206 and a seventh quantization unit 207 .
  • the first quantization unit 201 is used to add a pseudo operator for quantization at the input position of the first network layer with weights when the target structure in the model structure diagram includes a first network layer with weights, and use the pseudo operator to quantize the weights of the first network layer with weights.
  • the second quantization unit 202 is used to add a pseudo operator for quantization at the multiple output positions of the first network layer with weights, respectively, when the first network layer with weights has multiple output positions.
  • the third quantization unit 203 is used to add a pseudo operator for quantization at the output position of the first network layer with weights when the first network layer with weights has an output position and the output position of the first network layer with weights is not connected to the activation layer.
  • the fourth quantization unit 204 is used to add a forged operator at the output position of the overlay layer when the target structure in the model structure diagram includes an overlay layer, a first input branch of the overlay layer, and a second input branch of the overlay layer;
  • the first input branch includes a first network layer and a normalization layer with weights connected in sequence, the normalization layer is connected to the superposition layer, the type of the second input branch is the input type of non-weighted operation, and the first network layer with weights is a convolutional layer.
  • the fifth quantization unit 205 is used to add a spurious operator at the input position of the second input branch.
  • the sixth quantization unit 206 is used to add a pseudo operator for quantization at the input position of the first input branch, add a pseudo operator at the input position of the second input branch, and add a pseudo operator at the output position of the first superimposed layer when the target structure in the model structure diagram includes a first superimposed layer and the type of the first input branch of the first superimposed layer and the type of the second input branch of the first superimposed layer are both input types of non-weighted operations.
  • the seventh quantization unit 207 is used to add a pseudo operator for quantization at the output position of each of the N output branches of the first splicing layer, when the target structure in the model structure diagram includes a first splicing layer and the number of input branches of the first splicing layer is the same as the number of output branches of the first splicing layer; N is a positive integer greater than 1; and the quantization operators of the pseudo operators added at each output branch are the same.
  • the specific implementation methods of the first quantization unit 201, the second quantization unit 202, the third quantization unit 203, the fourth quantization unit 204, the fifth quantization unit 205, the sixth quantization unit 206 and the seventh quantization unit 207 can be referred to the description of step S102 in the embodiment corresponding to Figure 3 above, and will not be further elaborated here.
  • the service model determination module 30 is used to obtain a service model for service processing based on the quantized initial model.
  • the business model determination module 30 includes: a first training unit 301 , a first determination unit 302 , a first fusion unit 303 , a second fusion unit 304 , a second training unit 305 , a second determination unit 306 , a third training unit 307 and a third determination unit 308 .
  • the first training unit 301 is used to determine the quantized initial model as the first model, and perform fine-tuning training on the first model to obtain a second model;
  • the first determination unit 302 is used to obtain a business model for business processing based on the second model;
  • the business model includes a second network layer with a weight, and a quantization operator added at the input position of the second network layer with a weight; in the business model, the second network layer with a weight is obtained based on the first network layer with a weight, and the weight of the second network layer with a weight is obtained by processing the weight of the first network layer with a weight in the second model using a quantization operator.
  • the first network layer with weights in the target structure is a convolutional layer, and the target structure further includes a normalization layer connected to the convolutional layer, and an activation layer connected to the normalization layer;
  • the first fusion unit 303 is used to fuse the first network layer with weights, the normalization layer and the activation layer to obtain a second network layer with weights.
  • the target structure includes an overlay layer, a first input branch of the overlay layer, and a second input branch of the overlay layer;
  • the first input branch includes a first network layer and a normalization layer connected in sequence with weights;
  • the normalization layer is connected to the overlay layer, the type of the second input branch is an input type of a non-weighted operation, and the first network layer with weights is a convolutional layer;
  • the second fusion unit 304 is used to fuse the first network layer with weights, the normalization layer and the superposition layer to obtain a second network layer with weights.
  • the second training unit 305 is used to determine the quantized initial model as the first model, and perform fine-tuning training on the first model to obtain the second model;
  • the second determination unit 306 is used to obtain a business model for business processing based on the second model; the business model includes a second superimposed layer, a quantization operator added at the first input branch of the second superimposed layer, and a quantization operator added at the second input branch of the second superimposed layer; in the business model, the second superimposed layer is obtained according to the quantization operator and the first superimposed layer in the second model.
  • the third training unit 307 is used to determine the quantized initial model as the first model, and perform micro-training on the first model. Adjust the training to obtain the second model;
  • the third determination unit 308 is used to obtain a business model for business processing based on the second model;
  • the business model includes a second splicing layer, and a quantization operator added at the input position of each input branch of the second splicing layer; in the business model, the quantization operator at the input position of each input branch of the second splicing layer is obtained by equivalently forward propagating the quantization operator at the output position of each output branch of the first splicing layer added in the second model to the input position of each input branch of the first splicing layer in the second model.
  • the specific implementation methods of the first training unit 301, the first determination unit 302, the first fusion unit 303, the second fusion unit 304, the second training unit 305, the second determination unit 306, the third training unit 307 and the third determination unit 308 can refer to the description of step S103 in the embodiment corresponding to Figure 3 above, and will not be repeated here.
  • the feature acquisition module 40 is used to acquire image features of an image to be recognized when an image to be recognized is acquired if the business processing of the business model is image recognition.
  • the feature input module 50 is used to input image features into the business model; the optimized structure in the business model includes a target network layer and a quantization operator added at the input position of the target network layer; the target network layer is obtained based on the quantization operator and the original network layer of the target structure in the initial model.
  • the precision conversion module 60 is used to convert the precision of the input feature of the optimized structure from a first precision to a second precision based on a quantization operator added at the input position of the target network layer; the first precision is higher than the second precision.
  • the feature extraction module 70 is used to perform feature extraction processing on the input features belonging to the second precision through the target network layer, and the target network layer outputs the output features corresponding to the input features.
  • the category determination module 80 is used to obtain a predicted category of the image to be identified based on the output features.
  • the specific implementation of the search module 10, the quantization module 20, the business model determination module 30, the feature acquisition module 40, the feature input module 50, the precision conversion module 60, the feature extraction module 70 and the category determination module 80 can refer to the description of steps S201 to S208 in the embodiment corresponding to the above-mentioned FIG. 14, which will not be further described here.
  • the description of the beneficial effects of using the same method will not be repeated.
  • the computer device 1000 may include: at least one processor 1001, for example, a CPU, at least one network interface 1004, a memory 1005, and at least one communication bus 1002.
  • the communication bus 1002 is used to realize the connection and communication between these components.
  • the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface).
  • the memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage.
  • the memory 1005 may also optionally be at least one A storage device located away from the aforementioned processor 1001.
  • the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application.
  • the computer device may also include a user interface 1003 shown in FIG17.
  • the computer device may also include the user interface 1003, wherein the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), etc.
  • the network interface 1004 is mainly used for network communication;
  • the user interface 1003 is mainly used for providing an input interface for the user; and
  • the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:
  • the target structure is quantized using a forged operator for quantization;
  • the forged operator includes a quantization operator and an anti-quantization operator;
  • the computer device 1000 described in the embodiment of the present application can execute the description of the data processing method in the embodiments corresponding to FIG. 3 and FIG. 14 above, and can also execute the description of the data processing device 1 in the embodiment corresponding to FIG. 16 above, which will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated.
  • An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program.
  • the computer program includes program instructions.
  • the program instructions are executed by a processor, the data processing method provided in each step of Figure 3 and Figure 14 is implemented.
  • the computer-readable storage medium may be the data transmission device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device.
  • the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the computer device.
  • the computer-readable storage medium may also include both the internal storage unit of the computer device and the external storage device.
  • the computer-readable storage medium is used to store the computer program and other programs and data required by the computer device.
  • the computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
  • the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium.
  • a processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device can execute the above embodiments.
  • the description of the data processing method or device will not be repeated here.
  • the description of the beneficial effects of the same method will not be repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • Databases & Information Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Image Analysis (AREA)

Abstract

本申请实施例公开了一种数据处理方法、装置、计算机设备及存储介质，可应用于人工智能场景，包括：在获取到初始模型时，对初始模型进行结构搜索，得到初始模型的模型结构图；根据模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对目标结构进行量化；伪造算子包括量化算子和反量化算子；基于量化后的初始模型，得到用于业务处理的业务模型。采用本申请实施例，可以提升模型量化的速度，实现模型量化的通用性。

Description

一种数据处理方法、装置、计算机设备及存储介质
本申请要求于2023年06月19日提交中国专利局、申请号为202310731911.0、发明名称为“一种数据处理的方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及一种数据处理方法、装置、计算机设备及存储介质。
背景技术
随着人工智能技术的不断发展，人工智能模型(即AI模型)的结构也被设计的越来越复杂，AI模型的权重参数也越来越庞大，那么在进行前向推理时对硬件的要求也越来越高。而在一些对实时性要求很高，同时算力资源有限的终端设备上，要想很好的部署AI模型，就需要对AI模型进行一些"压缩"处理，而量化就是一种模型压缩的方式。
传统量化方法往往需要人工手动去修改AI模型，以插入相应的伪造算子(即包括量化算子和反量化算子)，进而导致代码侵入性很高，不利于代码的复用，即针对不同的模型结构需要设计不同的量化方案，不仅影响量化速度，还导致模型量化不具备通用性。
发明内容
本申请实施例提供一种数据处理方法、装置、计算机设备及存储介质，可以提升模型量化的速度，实现模型量化的通用性。
本申请实施例一方面提供一种数据处理方法，包括：
在获取到初始模型时，对初始模型进行结构搜索，得到初始模型的模型结构图；
根据模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对目标结构进行量化，伪造算子包括量化算子和反量化算子；
基于量化后的初始模型，得到用于业务处理的业务模型。
本申请实施例一方面提供一种数据处理装置，包括：
搜索模块，用于在获取到初始模型时，对初始模型进行结构搜索，得到初始模型的模型结构图；
量化模块，用于根据模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对目标结构进行量化，伪造算子包括量化算子和反量化算子；
业务模型确定模块，用于基于量化后的初始模型，得到用于业务处理的业务模型。
其中，该量化模块包括：
第一量化单元，用于在模型结构图中的目标结构包括具有权重的第一网络层的情况下，在具有权重的第一网络层的输入位置处，添加该伪造算子，且采用伪造算子对具有权重的第一网络层的权重进行量化。
其中，该量化模块还包括：
第二量化单元，用于在具有权重的第一网络层具有多个输出位置的情况下，在具有权重的第一网络层的多个输出位置处，分别添加该伪造算子。
其中，该量化模块还包括：
第三量化单元，用于在具有权重的第一网络层具有一个输出位置，且具有权重的第一网络层的输出位置处未连接激活层的情况下，在具有权重的第一网络层的输出位置处，添加该伪造算子。
其中，在具有权重的第一网络层具有一个输出位置，且具有权重的第一网络层的输出位置处连接激活层的情况下，不在具有权重的第一网络层的输出位置处，添加该伪造算子。
其中，该量化模块还包括：
第四量化单元，用于在模型结构图中的目标结构包括叠加层、叠加层的第一输入分支和叠加层的第二输入分支的情况下，在叠加层的输出位置处添加伪造算子；
其中，第一输入分支包括依次连接的具有权重的第一网络层和归一化层，归一化层与叠加层连接，第二输入分支的类型为非加权操作的输入类型，具有权重的第一网络层为卷积层。
其中，该量化模块还包括：
第五量化单元，用于在第二输入分支的输入位置处添加伪造算子。
其中，该量化模块还包括：
第六量化单元，用于在模型结构图中的目标结构包括第一叠加层，且第一叠加层的第一输入分支的类型与第一叠加层的第二输入分支的类型均为非加权操作的输入类型的情况下，在第一输入分支的输入位置处添加用于进行量化的伪造算子，在第二输入分支的输入位置处添加伪造算子，且在第一叠加层的输出位置处添加伪造算子。
其中，该量化模块还包括：
第七量化单元，用于在模型结构图中的目标结构包括第一拼接层，且第一拼接层的输入分支与第一拼接层的输出分支的数量相同的情况下，在第一拼接层的N个输出分支的每 个输出分支的输出位置处，分别添加用于进行量化的伪造算子；N为大于1的正整数；每个输出分支处所添加的伪造算子的量化算子均相同。
其中，该业务模型确定模块包括：
第一训练单元，用于将量化后的初始模型确定为第一模型，对第一模型进行微调训练，得到第二模型；
第一确定单元，用于基于第二模型，得到用于业务处理的业务模型；业务模型包括具有权重的第二网络层、和添加在具有权重的第二网络层的输入位置处的量化算子；在业务模型中，具有权重的第二网络层是根据具有权重的第一网络层得到的，且具有权重的第二网络层的权重是采用量化算子，对第二模型中的具有权重的第一网络层的权重进行处理后所得到的。
其中，目标结构中的具有权重的第一网络层为卷积层，且目标结构还包括与卷积层连接的归一化层，以及与归一化层连接的激活层；
该业务模型确定模块还包括：
第一融合单元，用于对具有权重的第一网络层、归一化层和激活层进行融合，得到具有权重的第二网络层。
其中，目标结构包括叠加层、叠加层的第一输入分支和叠加层的第二输入分支；第一输入分支包括依次连接的具有权重的第一网络层和归一化层；归一化层与叠加层连接，第二输入分支的类型为非加权操作的输入类型，具有权重的第一网络层为卷积层；
该业务模型确定模块还包括：
第二融合单元，用于对具有权重的第一网络层、归一化层和叠加层进行融合，得到具有权重的第二网络层。
其中，该业务模型确定模块还包括：
第二训练单元，用于将量化后的初始模型确定为第一模型，对第一模型进行微调训练，得到第二模型；
第二确定单元，用于基于第二模型，得到用于业务处理的业务模型；业务模型包括第二叠加层、添加在第二叠加层的第一输入分支处的量化算子、和添加在第二叠加层的第二输入分支处的量化算子；在业务模型中，第二叠加层是根据量化算子以及第二模型中的第一叠加层所得到的。
其中，该业务模型确定模块还包括：
第三训练单元，用于将量化后的初始模型确定为第一模型，对第一模型进行微调训练， 得到第二模型；
第三确定单元，用于基于第二模型，得到用于业务处理的业务模型；业务模型包括第二拼接层、和添加在第二拼接层的每个输入分支的输入位置处的量化算子；在业务模型中，第二拼接层的每个输入分支的输入位置处的量化算子是将添加在第二模型中的第一拼接层的每个输出分支的输出位置处的量化算子，分别等价向前传播至第二模型中的第一拼接层的每个输入分支的输入位置处后所得到的。
其中，该装置还包括：
特征获取模块，用于若业务模型的业务处理为图像识别，则在获取到待识别图像时，获取待识别图像的图像特征；
特征输入模块，用于将图像特征输入至业务模型；业务模型中的优化结构包括目标网络层、和添加在目标网络层的输入位置处的量化算子；目标网络层是根据量化算子，以及初始模型中的目标结构的原网络层所得到的；
精度转换模块，用于基于添加在目标网络层的输入位置处的量化算子，将优化结构的输入特征的精度由第一精度转换为第二精度；第一精度高于第二精度；
特征提取模块，用于通过目标网络层，对属于第二精度的输入特征进行特征提取处理，由目标网络层输出输入特征对应的输出特征；
类别确定模块，用于基于输出特征，得到待识别图像的预测类别。
本申请一方面提供了一种计算机设备，包括：处理器、存储器、网络接口；
处理器与存储器、网络接口相连，其中，网络接口用于提供数据通信功能，存储器用于存储计算机程序，处理器用于调用计算机程序，以使得计算机设备执行本申请实施例提供的方法。
本申请实施例一方面提供了一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，该计算机程序适于由处理器加载并执行，以使得具有该处理器的计算机设备执行本申请实施例提供的方法。
本申请实施例一方面提供了一种计算机程序产品，该计算机程序产品包括计算机程序，该计算机程序存储在计算机可读存储介质中；计算机设备的处理器从计算机可读存储介质读取该计算机程序，处理器执行该计算机程序，使得该计算机设备执行本申请实施例中的方法。
在本申请实施例中，具有模型量化功能的计算机设备在获取到初始模型时，无需人工手动向初始模型添加伪造算子，而是在对初始模型的模型结构图进行查找后，直接根据初 始模型的模型结构图中的目标结构的种类，自动采用用于进行量化的伪造算子(这里的伪造算子可以包括量化算子和反量化算子)，对目标结构进行量化，进而在量化结束后，可以基于量化后的初始模型，确定用于业务处理的业务模型。换言之，本申请实施例提供的数据处理方法来进行模型量化无需人工参与，对于任意一种初始模型而言，均可以根据模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，自动对初始模型中存在的可加速的待量化结构(即目标结构)进行量化，不仅可以有效降低人工成本，提升了模型量化的速度，还实现了模型量化的通用性。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种网络架构的结构示意图；
图2是本申请实施例提供的一种多个待量化结构的结构示意图；
图3是本申请实施例提供的一种数据处理方法的流程示意图；
图4是本申请实施例提供的一种对目标结构进行量化的示意图一；
图5是本申请实施例提供的一种对目标结构进行量化的示意图二；
图6是本申请实施例提供的一种对目标结构进行量化的示意图三；
图7是本申请实施例提供的一种对目标结构进行量化的示意图四；
图8是本申请实施例提供的一种对目标结构进行量化的示意图五；
图9是本申请实施例提供的一种对第二模型中的待优化结构进行处理的示意图一；
图10是本申请实施例提供的一种对第二模型中的待优化结构进行处理的示意图二；
图11是本申请实施例提供的一种对第二模型中的待优化结构进行处理的示意图三；
图12是本申请实施例提供的一种对第二模型中的待优化结构进行处理的示意图四；
图13是本申请实施例提供的一种对第二模型中的待优化结构进行处理的示意图五；
图14是本申请实施例提供的一种数据处理方法的流程示意图；
图15是本申请实施例提供的一种对业务模型进行模型优化的场景示意图；
图16是本申请实施例提供的一种数据处理装置的结构示意图；
图17是本申请实施例提供的一种计算机设备的示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
其中，本申请实施例针对复杂结构的人工智能模型，提出了一种通用型的量化方法，该量化方法涉及人工智能领域，其中，所谓人工智能(Artificial Intelligence，AI)是利用数字计算机或者数字计算机控制的计算模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。换句话说，人工智能是计算机科学的一个综合技术，它企图了解智能的实质，并生产出一种新的能以人类智能相似的方式做出反应的智能机器。人工智能也就是研究各种智能机器的设计原理与实现方法，使机器具有感知、推理与决策的功能。
人工智能技术是一门综合学科，涉及领域广泛，既有硬件层面的技术也有软件层面的技术。人工智能基础技术一般包括如传感器、专用人工智能芯片、云计算、分布式存储、大数据处理技术、操作/交互系统、机电一体化等技术。人工智能软件技术主要包括计算机视觉技术、语音处理技术、自然语言处理技术以及机器学习/深度学习、自动驾驶、智慧交通等几大方向。
其中，机器学习(Machine Learning，ML)是一门多领域交叉学科，涉及概率论、统计学、逼近论、凸分析、算法复杂度理论等多门学科。专门研究计算机怎样模拟或实现人类的学习行为，以获取新的知识或技能，重新组织已有的知识结构使之不断改善自身的性能。机器学习是人工智能的核心，是使计算机具有智能的根本途径，其应用遍及人工智能的各个领域。机器学习和深度学习通常包括人工神经网络、置信网络、强化学习、迁移学习、归纳学习、式教学习等技术。
深度学习算法由一个个计算单元组成，本申请实施例可以将这些计算单元称为算子(Operator，OP)。在网络模型中，算子对应网络层中的计算逻辑，例如：卷积层(Convolution Layer)对应的算子(即卷积算子)可以用于表示做一次卷积计算；全连接层(Fully-connected Layer，FC layer)中的权值求和过程也是一个算子；激活层(activating layer)对应的算子(即激活算子)为在网络模型中被用做激活函数的算子(例如，tanh、ReLU等)。
计算机视觉技术(Computer Vision，CV)是一门研究如何使机器“看”的科学，更进一步的说，就是指用摄影机和电脑代替人眼对目标进行识别和测量等机器视觉，并进一步 做图形处理，使电脑处理成为更适合人眼观察或传送给仪器检测的图像。作为一个科学学科，计算机视觉研究相关的理论和技术，试图建立能够从图像或者多维数据中获取信息的人工智能系统。计算机视觉技术通常包括图像处理、图像识别、图像语义理解、图像检索、OCR、视频处理、视频语义理解、视频内容/行为识别、三维物体重建、3D技术、虚拟现实、增强现实、同步定位与地图构建、自动驾驶、智慧交通等技术，还包括常见的人脸识别、指纹识别等生物特征识别技术。
自然语言处理(Nature Language processing，NLP)是计算机科学领域与人工智能领域中的一个重要方向。它研究能实现人与计算机之间用自然语言进行有效通信的各种理论和方法。自然语言处理是一门融语言学、计算机科学、数学于一体的科学。因此，这一领域的研究将涉及自然语言，即人们日常使用的语言，所以它与语言学的研究有着密切的联系。自然语言处理技术通常包括文本处理、语义理解、机器翻译、机器人问答、知识图谱等技术。
语音技术(Speech Technology)的关键技术有自动语音识别技术和语音合成技术以及声纹识别技术。让计算机能听、能看、能说、能感觉，是未来人机交互的发展方向，其中语音成为未来最被看好的人机交互方式之一。
应当理解，随着深度学习技术在多个领域的快速应用，具体包括计算机视觉、自然语言处理、语言技术等，出现了大量的基于深度学习的网络模型。为了解决这些模型因模型结构越来越复杂而导致不便于部署在一些低成本的终端设备中这一问题，本申请实施例可以采用一种通用型的量化方法，对人工智能模型进行量化，进而使得量化后的模型存在以下优点：具有更少的存储开销和带宽需求、更快的计算速度、更低的能耗与占用面积、尚可接受的精度损失，支持低精度(例如，int8)等。
请参见图1，图1是本申请实施例提供的一种网络架构的结构示意图。如图1所示，该网络架构可以包括服务器10S和终端设备集群。该终端设备集群可以包括一个或者多个终端设备，这里将不对终端设备的数量进行限制。如图1所示，该终端设备集群具体可以包括终端设备100a、终端设备100b、终端设备100c、…、终端设备100n。如图1所示，终端设备100a、终端设备100b、终端设备100c、…、终端设备100n可以分别与上述服务器10S进行网络连接，以便于每个终端设备可以通过该网络连接与服务器10S进行数据交互。其中，这里的网络连接不限定连接方式，可以通过有线通信方式进行直接或间接地连接，也可以通过无线通信方式进行直接或间接地连接，还可以通过其他方式，本申请在此不做限制。
其中，该终端设备集群中的每个终端设备均可以包括：智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表、车载终端、智能电视等具有数据处理功能的智能终端。应当理解，如图1所示的终端设备集群中的每个终端设备均可以安装有业务应用(即应用客户端)，当该应用客户端运行于各终端设备中时，可以分别与上述图1所示的服务器10S之间进行数据交互。其中，该应用客户端可以包括社交客户端、多媒体客户端(例如，视频客户端)、娱乐客户端(例如，游戏客户端)、信息流客户端、教育客户端、直播客户端等应用客户端。其中，该应用客户端可以为独立的客户端，也可以为集成在某客户端(例如，社交客户端、教育客户端以及多媒体客户端等)中的嵌入式子客户端，在此不做限定。
如图1所示，本申请实施例中的服务器10S可以为该应用客户端对应的服务器。该服务器10S可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云计算服务的云服务器。其中，本申请实施例将不对服务器的数量进行限制。
为便于理解，本申请实施例可以在图1所示的多个终端设备中选择一个终端设备作为业务对象所使用的业务终端设备。例如，本申请实施例可以将图1所示的终端设备100a作为业务终端设备，该业务终端设备中可以集成有业务应用(即应用客户端)。此时，该业务终端设备可以通过该应用客户端对应的业务数据平台与服务器10S之间实现数据交互。其中，在模型量化场景中，为了有效控制人工智能模型中的各个模块的计算精度，在对某一具有复杂结构的初始模型进行优化时，这里的业务对象(例如，用户)可以指定该初始模型中的某些网络层不参与量化，其中，本申请实施例可以将业务对象所指定的不参与量化的网络层称之为指定网络层(即属于黑名单中的网络层)。
在本申请实施例中，具有模型量化功能的计算机设备可以为服务器，也可以为图1所示的终端设备集群中的任意一个终端设备，例如，终端设备100b，这里将不对计算机设备的具体形式进行限定。为便于理解，本申请实施例中的计算机设备可以以服务器(例如，图1所示的服务器10S)为例，用以阐述该计算机设备通过通用型量化工具，对初始模型进行自动量化的具体实施方式。
其中，本申请实施例中的通用型量化工具利用了丰富的推理经验，设计各种适合低精度(例如，int8)加速的待量化结构，即针对不同种类的待量化结构，设定不同的量化规则，以得到初始规则集合，使得后续可以直接根据该初始规则集合所确定的量化规则集合，自动对获取到的初始模型进行量化。
可以理解的是，若业务对象未配置不参与量化的指定网络层，则这里的量化规则集合为通用型量化工具所获取到的初始规则集合；若业务对象配置有不参与量化的指定网络层，则这里的量化规则集合是基于业务对象所配置的指定网络层，对初始规则集合进行过滤后所得到的规则集合。这意味着在本申请实施例中，业务对象配置的指定网络层的量化优先级高于通用的初始规则集合，具有最高量化优先级。
其中，这里的量化规则集合可以包括N个量化规则，N为正整数，一个量化规则对应一种待量化结构。该计算机设备在获取到待优化的初始模型时，需要对初始模型进行结构搜索，以得到初始模型的模型结构图，进而可以在该模型结构图中查询是否存在目标结构，这里的目标结构属于量化规则集合所对应的N种待量化结构。若模型结构图中存在目标结构，则该计算机设备可以自动从N个量化规则中确定与目标结构相匹配的目标量化规则，进而可以基于目标量化规则以及伪造算子，对目标结构进行量化，且在量化结束后，基于量化后的初始模型，确定用于业务处理的业务模型。
其中，这里的伪造算子(例如，fake op)主要用于将输入到该伪造算子的参数和输入先量化后再反量化，即等同于QDQ算子，其结构组成是量化算子(quant op)以及反量化算子(dequant op)。该伪造算子可以用于存储当前层的量化信息，具体可以包括缩放因子(scale)和偏移量(zero-point)，本申请实施例还可以基于伪造算子中的量化算子和反量化算子的位置来决定每一层算子的精度。
可以理解的是，量化算子主要用于将属于第一精度的特征转换为属于第二精度的特征，反量化算子主要用于将属于第二精度的特征转换回属于第一精度的特征，这里的第一精度高于第二精度。例如，量化可以将浮点型实数(例如，FP32)转化为整型数(例如，INT8)，而反量化可以将整型数(例如，INT8)转化回浮点型实数(例如，FP32)。
其中，这里的初始模型可以包括各种人工智能场景下的模型，例如，用于进行内容审核的检测模型、用于进行关键部位识别的检测模型(例如，人脸检测模型)以及图像识别模型等，这里将不对初始模型的应用场景进行限定。换言之，对于任意一种复杂结构的初始模型而言，该计算机设备均可以根据预先设定好的量化规则集合，自动对初始模型中存在的可加速的待量化结构进行量化，以基于量化后的初始模型，确定用于业务处理的业务模型，这种量化方法无需人工针对不同模型设计不同的量化方案，而是能够自动对初始模型中存在的可加速的待量化结构进行量化，不仅可以有效降低人工成本，还能够提升模型量化速度，此外，本申请实施例这种量化方法因不需要人工插入伪造算子，使得代码侵入性降低，从而实现了模型量化通用性。
为便于理解，进一步地，请参见图2，图2是本申请实施例提供的一种多个待量化结构的结构示意图。其中，本申请实施例中的计算机设备可以为具有模型量化功能的计算机设备，该计算机设备可以为上述图1所示的终端设备集群中的任意一个终端设备，例如，终端设备100b，该计算机设备也可以为上述图1所示的服务器10S，这里将不对计算机设备进行限定。
应当理解，本申请实施例可以基于某一深度学习推理框架中的丰富的推理经验，设计出各种适合低精度加速的待量化结构(Patterns)，进而可以基于不同的待量化结构设计出不同的量化规则。其中，这里的待量化结构属于本申请搜索的最小粒度类型，该待量化结构可以为由多个网络层所构成的模块，也可以为单个网络层所构成的模块。
如图2所示，规则集合20R是本申请实施例中的计算机设备获取到的初始规则集合，该规则集合20R可以包括M个通用型的量化规则，M为正整数，一个量化规则对应一种待量化结构。为便于阐述，该规则集合20R中的量化规则的数量可以以7个为例，具体可以包括量化规则R1、量化规则R2、量化规则R3、量化规则R4、量化规则R5、量化规则R6以及量化规则R7。
其中，量化规则R1、量化规则R2以及量化规则R3均为针对具有权重的网络层这一待量化结构所设计的量化规则，这里的具有权重的网络层即为具有加权操作的网络层，例如，卷积层(conv)、转置卷积层(例如，transpose conv)以及通用矩阵乘法(例如，gemm)；而在具有权重的网络层为卷积层时，可以进一步设计出量化规则R4和量化规则R5。
比如，如图2所示，在结构G1(例如，注意力结构(Attention结构))中具有权重的网络层具有多个输出位置的情况下，该结构G1对应的量化规则R1可以指示在具有权重的网络层的输入位置处、以及具有权重的网络层的多个输出位置处，分别添加伪造算子，且采用伪造算子对具有权重的网络层的权重进行量化。例如，在结构G1为注意力结构(Attention结构)时，该结构可以包括卷积算子(conv算子)、矩阵变维算子(reshape算子)、排列算子(permute算子)、bmm矩阵乘算子。
在结构G2包括具有权重的网络层时，该量化规则R2可以指示在保留高精度计算时，在具有权重的网络层的输入位置处添加伪造算子，且采用伪造算子对具有权重的网络层的权重进行量化；在保留低精度计算时，具有权重的网络层的输入位置处、具有权重的网络层的输出位置处，分别添加伪造算子，且采用伪造算子对具有权重的网络层的权重进行量化。
其中，根据推理经验可知，具有权重的网络层一般后面会跟激活层，若在该网络层的 输出位置处添加伪造算子，那么在保留高精度计算时，将会导致不可接受的精度损失，基于此，在结构G2中包括的具有权重的网络层具有一个输出位置，且该具有权重的第一网络层的输出位置未连接激活层的情况下，该结构G1对应的量化规则R2可以在具有权重的网络层的输入位置处、以及具有权重的网络层的输出位置处，分别添加伪造算子，且采用伪造算子对具有权重的网络层的权重进行量化。而在结构G3中包括的具有权重的网络层具有一个输出位置，且该具有权重的第一网络层的输出位置连接激活层的情况下，该结构G3对应的量化规则R3可以在具有权重的网络层的输入位置处添加伪造算子，采用伪造算子对具有权重的网络层的权重进行量化，即可以不在该具有权重的网络层的输出位置处添加伪造算子。
另外，根据推理经验可知，由卷积层、归一化层(例如，BN)以及激活层(例如，ReLU)构成的结构(例如，图2所示的结构G4)在模型推理阶段能够自动对原网络层的算子进行融合，因此，该结构G4对应的量化规则R4可以指示在卷积层的输入位置处添加伪造算子，且采用伪造算子对卷积层的权重进行量化。
根据推理经验可知，叠加层的非加权操作的输入的精度决定了结构融合后的算子的输出精度，换言之，若叠加层的非加权操作的输入不参与量化，则叠加层的输出后的量化算子将不会被融合，这样将导致融合后的算子存在精度转换的过程，进而影响速度，基于此，在结构G5包括叠加层、叠加层的第一输入分支(包括依次连接的卷积层和归一化层)、和叠加层的第二输入分支(即类型为非加权操作的输入类型)在叠加层包括第一输入分支(具有加权操作的分支)和第二输入分支的情况下，该结构G5对应的量化规则R5可以指示在需要保持较高的速度时，在卷积层的输入位置处、叠加层的第二输入分支的输入位置处以及叠加层的输出位置处，分别添加伪造算子，且采用伪造算子对卷积层的权重进行量化；而在对速度无要求时，可以在卷积层的输入位置处、以及叠加层的输出位置处，分别添加伪造算子，且采用伪造算子对卷积层的权重进行量化，即可以不在叠加层的第二输入分支的输入位置处添加伪造算子。例如，图2所示的结构G5可以为与叠加层(例如，Add)相关联的跳跃链接(skip connections)结构，例如，残差网络(resnet)或轻量级网络(Efficient Net)中的跳跃链接结构。
根据推理经验可知，叠加层的两个输入分支中存在无加权操作的低精度输入时，那么叠加层的输出分支必须要量化，基于此，在图2所示的结构G6可以包括叠加层，且该叠加层的两个输入分支的类型均为非加权操作的输入类型的情况下，该结构G6对应的量化规则R6可以指示在叠加层的两个输入位置处、和叠加层的输出位置处，分别添加伪造算 子。
其中，根据推理经验可知，在一定情况下，伪造算子中的量化算子(即Q算子)可以等价向前传播，伪造算子中的反量化算子(即DQ算子)可以等价向后传播，因此如果在拼接层(例如，Concat)的每个输出位置处伪造算子，这样将使得拼接层后的计算属于第二精度，但是要想拼接层对应的拼接算子也被量化，则需要令两个输出分支处所添加的伪造算子中的量化算子相同，基于此，在图2所示的结构G7包括拼接层，且拼接层的输入分支与拼接层的输出分支的数量(例如，N个)相同的情况下，结构G7对应的量化规则R7可以指示在拼接层的每个输出分支的输出位置处，分别添加具有相同量化算子的伪造算子，其中，这里的N为大于1的正整数，本申请实施例可以以2为例。
可以理解的是，后续在对初始模型进行量化时，业务对象的指定网络层的量化优先级具有最高量化优先级。比如，若业务对象针对初始模型的业务配置包括不参与量化的指定网络层(例如，激活层)，此时，计算机设备可以先过滤掉包括指定网络层的待量化结构所对应的量化规则(例如，量化规则R3、量化规则R4)，得到过滤后的第一规则集合，再基于过滤后的第一规则集合，检测初始模型的模型结构图中是否存在可加速的待量化结构(即与过滤后的5种结构相同的目标结构)。
其中，具有模型量化功能的计算机设备根据通用型量化工具，自动对初始模型中存在的可加速的待量化结构进行量化的具体实现方式可以参见下述图3-图15所对应的实施例。
进一步地，请参见图3，图3是本申请实施例提供的一种数据处理方法的流程示意图。如图3所示，该方法可以由具有模型量化功能的计算机设备执行，该计算机设备可以为终端设备(例如，上述图1所示的终端设备集群中的任意一个终端设备，例如，终端设备100b，也可以为服务器(例如，上述图1所示的服务器10S)，在此不做限定。为便于理解，本申请实施例以该方法由具有模型量化功能的计算机设备为例进行说明，该方法至少可以包括以下步骤S101-步骤S103：
步骤S101，计算机设备在获取到初始模型时，对初始模型进行结构搜索，得到初始模型的模型结构图。
具体地，该计算机设备在获取到待优化的初始模型时，可以调用通用型量化工具，基于该通用型量化工具中的搜索图工具，对初始模型进行结构搜索，进而可以得到该初始模型的模型结构图。
步骤S102，计算机设备根据模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对目标结构进行量化。
其中，这里的伪造算子可以包括量化算子和反量化算子。可以理解的是，这里的目标结构的种类可以为第一结构、第二结构或第三结构。这里的第一结构是指包括具有权重的网络层的结构，本申请实施例可以将初始模型的目标结构中的具有权重的网络层称之为具有权重的第一网络层。其中，上述图2所示的结构G1、结构G2、结构G3、结构G4以及结构G5均属于第一结构。这里的第二结构(例如，上述图2所示的结构G6)包括输入分支的类型为非加权操作的输入类型的叠加层，本申请实施例可以将初始模型的目标结构中所包括的输入分支的类型均为非加权操作的输入类型的叠加层，称之为第一叠加层。这里的第三结构(例如，上述图2所示的结构G7)包括输入分支与输出分支的数量相同的拼接层，本申请实施例可以将初始模型的目标结构中所包括的输入分支与输出分支的数量相同的拼接层称之为第一拼接层。
应当理解，该计算机设备可以获取预先根据推理经验所设计的包括M个通用型的量化规则，进而可以将这M个量化规则所构成的规则集合确定为初始规则集合(例如，上述图2所示的规则集合20R)。其中，这里的M为正整数。与此同时，该计算机设备还需要获取业务对象针对初始模型的业务配置。其中，这里的业务配置是指业务对象(例如，用户)针对初始模型所设定的配置信息，例如，该初始模型后续所要部署的业务平台、业务对象所指定的不参与量化的指定网络层等，这里将不对其进行限定。可以理解的是，由于初始规则集合包括M个量化规则，且一个量化规则对应一种待量化结构，因此，该初始规则集合与M个待量化结构相关。进一步地，该计算机设备可以基于该业务配置以及初始规则集合，确定最终用于进行模型量化的量化规则集合。
其中，当业务对象指定某一网络层不参与量化时，这意味着业务配置包括不参与量化的指定网络层，此时，该计算机设备需要在M个待量化结构中查询指定网络层，得到查询结果。若查询结果指示M个待量化结构中存在包括指定网络层的待量化结构，则该计算机设备可以将查询到的待量化结构确定为过滤结构，进而可以在初始规则集合中删除过滤结构对应的量化规则，将删除后的初始规则集合确定为量化规则集合。
如图2所示，若业务配置包括不参与量化的指定网络层为激活层，则该计算机设备需要在图2所示的7个待量化结构中查询激活层，以得到查询结果。由于图2所示的结构G3、以及结构G4均包括激活层，因此该查询结果指示7个待量化结构中存在包括指定网络层的待量化结构，此时，该计算机设备可以将结构G3和结构G4分别确定为过滤结构，然后在规则集合20R中删除该过滤结构对应的量化规则(即量化规则R3和量化规则R4)，进而将删除后的规则集合20R确定为量化规则集合。这时，这里的量化规则集合可以包括图 2所示的量化规则R1、量化规则R2、量化规则R5、量化规则R6以及量化规则R7。
可选的，当业务对象未指定不参与量化的网络层时，这意味着业务配置中不存在指定网络层，此时，该计算机设备可以直接将初始规则集合确定为量化规则集合。如图2所示，若业务配置不包括指定网络层，则该计算机设备可以直接将图2所示的规则集合20R确定为量化规则集合。此时，这里的量化规则集合可以包括图2所示的量化规则R1、量化规则R2、量化规则R3、量化规则R4、量化规则R5、量化规则R6以及量化规则R7。
其中，由于量化规则集合可以包括N个量化规则，且一个量化规则对应一种待量化结构，因此，该计算机设备需要在模型结构图中使用深度优先搜索(Depth First Search，DFS)，来找出所有符合上述量化规则集合中的N个待量化结构的目标结构，进而可以根据模型结构图中的目标结构的种类，从N个量化规则中，确定与目标结构相匹配的目标量化规则，确定伪造算子在初始模型中的添加位置，进而可以在确定的添加位置中添加伪造算子，以实现对目标结构的量化。
比如，在模型结构图中的目标结构包括具有权重的第一网络层的情况下，该计算机设备可以在具有权重的第一网络层的输入位置处，添加用于进行量化的伪造算子，且采用伪造算子对具有权重的第一网络层的权重进行量化。
其中，在具有权重的第一网络层具有多个输出位置的情况下，该计算机设备还需要在具有权重的第一网络层的多个输出位置处，分别添加用于进行量化的伪造算子。
换言之，在目标结构包括具有权重的第一网络层，且具有权重的第一网络层具有多个输出位置的情况下，该计算机设备需要从量化规则集合中，获取与目标结构相匹配的量化规则，进而可以将该匹配到的量化规则(例如，上述图2所示的量化规则R1)确定为目标量化规则。然后，该计算机设备可以基于该量化规则R1，将具有权重的第一网络层的输入位置、具有权重的第一网络层的输出位置、以及具有权重的第一网络层的权重位置分别确定为添加位置，进而可以在添加位置处添加伪造算子。其中，每个添加位置处添加的伪造算子可以相同，也可以不同，这里将不对其进行限定。
可选的，在具有权重的第一网络层具有一个输出位置，且该具有权重的第一网络的输出位置处未连接激活层的情况下，该计算机设备还需要在具有权重的第一网络层的输出位置处，添加用于进行量化的伪造算子。
可选的，在具有权重的第一网络层具有一个输出位置，且具有权重的第一网络层的输出位置处连接激活层的情况下，为了保留高精度的计算，减少量化输出所导致的精度丢失，该计算机设备可以不在具有权重的第一网络层的输出位置处，添加用于进行量化的伪造算 子。
为便于理解，进一步地，请参见图4，图4是本申请实施例提供的一种对目标结构进行量化的示意图一。如图4所示，该结构40G可以为该计算机设备在初始模型的模型结构图中所查找到的目标结构，该结构40G包括具有权重的第一网络层(例如，卷积层)，该具有权重的第一网络层的具有一个输出位置，且该输出位置处并未连接激活层。
如图4所示，本申请实施例中的输入(例如，X)在传输至该结构40G时，可以通过该结构40G中的具有权重的第一网络层，对X进行特征提取处理，从而得到该具有权重的第一网络层的输出(即结构40G的输出，例如，Y)。若这里的X属于第一精度，那么该结构40G的输出也为第一精度。进一步地，在对结构40G进行量化时，该计算机设备可以在量化规则集合中，查找与结构40G相匹配的量化规则，进而可以将查找到的量化规则确定为目标量化规则(例如，上述图2所示的量化规则R2)。
基于此，该计算机设备可以基于该目标量化规则，将具有权重的第一网络层的输入位置确定为位置A1，将具有权重的第一网络层中的权重位置确定为位置A2，将具有权重的第一网络层的输出位置确定为位置A3，进而可以将该位置A1、位置A2和位置A3分别确定为伪造算子在结构40G中的添加位置。
进一步地，该计算机设备可以在位置A1处添加伪造算子(例如，伪造算子4F1)，使得X在经过伪造算子4F1中的量化算子后转换为第二精度，再经过伪造算子4F1中的反量化算子后，还原为第一精度。同理，该计算机设备还可以在位置A2处添加伪造算子，对具有权重的第一网络层的权重进行量化，使得该具有权重的第一网络层的权重在经过伪造算子4F2中的量化算子后，转换为第二精度，再经过伪造算子4F2中的反量化算子后，还原为第一精度。该计算机设备还可以在位置A3处添加伪造算子(例如，伪造算子4F3)，使得具有权重的第一网络层的输出在经过伪造算子4F3中的量化算子后，转换为第二精度，再经过伪造算子4F3中的反量化算子后，还原为第一精度。
由此可见，本申请实施例在检测到结构40G这样的目标结构时，可以通过与结构40G相匹配的量化规则以及伪造算子(包括伪造算子4F1、伪造算子4F2和伪造算子4F3)，自动对结构40G进行量化，以得到量化后的结构40G(例如，图4所示的量化结构41G)。
又比如，由于推理经验可知，在模型结构图中的目标结构包括依次连接的具有权重的第一网络层，归一化层以及激活层，且具有权重的第一网络层为卷积层的情况下，后续可以对这样的目标结构进行融合，因此本申请实施例还可以对这样的目标结构进行量化，即计算机设备可以从量化规则集合中，获取与目标结构相匹配的量化规则，进而可以将该匹 配到的量化规则(例如，上述图2所示的量化规则R4)确定为目标量化规则。然后，该计算机设备可以基于该量化规则R4，在具有权重的第一网络层的输入位置添加伪造算子，且采用量化算子，对具有权重的第一网络层的权重进行量化。
为便于理解，进一步地，请参见图5，图5是本申请实施例提供的一种对目标结构进行量化的示意图二。如图5所示，该结构50G可以为该计算机设备在初始模型的模型结构图中所查找到的目标结构，该结构50G可以包括依次连接的卷积层、归一化层以及激活层。
如图5所示，在结构50G中的卷积层、归一化层以及激活层的精度均为第一精度(例如，FP32)时，这意味着该卷积层中的输入(例如，X)的精度和权重的精度也为第一精度，换言之，该结构50G对应的输入在经过卷积计算、归一化计算以及激活计算后所得到的输出(例如，Y)仍为第一精度。可以理解的是，在对结构50G进行量化时，计算机设备可以根据结构50G的种类，在量化规则集合中查找与结构50G相匹配的量化规则，进而可以将查找到的量化规则确定为目标量化规则(例如，上述图2所示的量化规则R4)。
基于此，该计算机设备可以基于该目标量化规则，将结构50G中的卷积层的输入位置确定为位置B1，且将结构50G中的卷积层的权重位置确定为位置B2，进而可以将该位置B1和位置B2分别确定为伪造算子在结构50G中的添加位置。
进一步地，该计算机设备可以在位置B1处添加伪造算子(例如，伪造算子5F1)，使得属于第一精度的输入在经过伪造算子5F1中的量化算子后，转换为第二精度，再经过伪造算子5F1中的反量化算子后，还原为第一精度。与此同时，该计算机设备还可以采用伪造算子5F2，对卷积层的权重进行量化，即在位置B2处添加伪造算子5F2，使得属于第一精度的权重在经过伪造算子5F2中的量化算子后，转换为第二精度，再经过伪造算子5F2中的反量化算子后，还原为第一精度。其中，这里的伪造算子5F1与伪造算子5F2可以相同，也可以不同，这里将不对其进行限定。
由此可见，本申请实施例在检测到结构50G这样的目标结构时，可以通过与结构50G相匹配的量化规则以及伪造算子(包括伪造算子5F1和伪造算子5F2)，自动对结构50G进行量化，以得到量化后的结构50G(例如，图5所示的量化结构51G)。
又比如，在模型结构图中的目标结构包括叠加层、叠加层的第一输入分支和叠加层的第二输入分支的情况下，该计算机设备可以在叠加层的输出位置处添加伪造算子，也可以在第二输入分支的输入位置处添加伪造算子。其中，这里的第一输入分支包括依次连接的具有权重的第一网络层和归一化层，该归一化层与叠加层连接，第二输入分支的类型为非加权操作的输入类型，具有权重的第一网络层为卷积层。
换言之，在模型结构图中的目标结构包括的具有权重的第一网络层为卷积层，且该卷积层的输出位置与归一化层连接，该归一化层的输出位置与叠加层连接的情况下，该计算机设备不仅需要在具有权重的第一网络层的输入位置处，添加用于进行量化的伪造算子，采用伪造算子对具有权重的第一网络层的权重进行量化，还需要在叠加层的输出位置处添加伪造算子。此外，由于第二输入分支的精度决定了融合后的具有权重的网络层的输出精度，为了提高速度，该计算机设备还可以在第二输入分支的输入位置处添加伪造算子。
为便于理解，进一步地，请参见图6，图6是本申请实施例提供的一种对目标结构进行量化的示意图三。如图6所示，该结构60G可以为该计算机设备在初始模型的模型结构图中所查找到的目标结构，该结构60G是一种与叠加层相关联的跳跃链接结构。其中，该叠加层的第一输入分支可以包括依次连接的卷积层和归一化层，第二输入分支的类型为非加权操作的输入类型，即该第二输入分支对应的输入X2可以为一个非加权操作的输入。
如图6所示，该结构60G中的叠加层可以对经过卷积计算、归一化计算后的第一输入(例如，X1)和第二输入(例如，X2)进行叠加，得到结构60G的输出(例如，Y)。由于该第一输入和第二输入均属于第一精度，因此该结构60G的输出仍为第一精度。可以理解的是，在对结构60G进行量化时，该计算机设备可以在量化规则集合中，查找与结构60G相匹配的量化规则，进而可以将查找到的量化规则确定为目标量化规则(例如，上述图2所示的量化规则R5)。
基于此，为了提高后续模型处理速度，该计算机设备可以基于该目标量化规则，将结构60G中的卷积层的输入位置确定为位置C1，且将卷积层的权重位置确定为位置C2，将第二输入分支的输入位置确定为位置C3，将叠加层的输出位置确定为位置C4，进而可以将该位置C1、位置C2、位置C3和位置C4分别确定为伪造算子在初始模型中的添加位置。
进一步地，该计算机设备可以在位置C1处添加伪造算子(例如，伪造算子6F1)，使得属于第一精度的第一输入(例如，X1)在经过伪造算子6F1中的量化算子后，转换为第二精度，再经过伪造算子6F1中的反量化算子后，还原为第一精度。同理，该计算机设备还可以在位置C2处添加伪造算子(例如，伪造算子6F2)，采用伪造算子6F2，对卷积层的权重进行量化，使得属于第一精度的权重在经过伪造算子6F2中的量化算子后，转换为第二精度，再经过伪造算子6F2中的反量化算子后，还原为第一精度。该计算机设备还可以在位置C3处添加伪造算子(例如，伪造算子6F3)，使得属于第一精度的第二输入(例如，X2)在经过伪造算子6F3中的量化算子后，转换为第二精度，再经过伪造算子6F2中的反量化算子后，还原为第一精度。该计算机设备还可以在位置C4处添加伪造算子(例 如，伪造算子6F4)，使得叠加层的输出在经过伪造算子6F4中的量化算子后，转换为第二精度，再经过伪造算子6F4中的反量化算子后，还原为第一精度。其中，这里的各添加位置处所添加的伪造算子可以相同，也可以不同，这里将不对其进行限定。
由此可见，本申请实施例在检测到结构60G这样的目标结构时，可以通过与结构60G相匹配的量化规则以及伪造算子(包括伪造算子6F1、伪造算子6F2、伪造算子6F3、伪造算子6F4)，自动对结构60G进行量化，以得到量化后的结构60G(例如，图6所示的量化结构61G)。
又比如，在模型结构图中的目标结构包括第一叠加层，且该第一叠加层的第一输入分支的类型与该第一叠加层的第二输入分支的类型均为非加权操作的输入类型的情况下，该计算机设备需要分别在第一输入分支的输入位置处添加用于进行量化的伪造算子、在第二输入分支的输入位置处添加伪造算子，且在第一叠加层的输出位置处添加伪造算子。
为便于理解，进一步地，请参见图7，图7是本申请实施例提供的一种对目标结构进行量化的示意图四。如图7所示，该结构70G可以为该计算机设备在初始模型的模型结构图中所查找到的目标结构，该结构70G可以为包括两个输入分支和一个输出分支的叠加层。
如图7所示，本申请实施例中的两个输入(例如，X1和X2)在传输至该结构70G时，可以通过该结构70G中的叠加层，对X1和X2进行叠加处理，从而得到该叠加层的输出(即结构70G的输出，例如，Y)。若这里的X1和X2均属于第一精度，那么该结构70G的输出也为第一精度。可以理解的是，在对结构70G进行量化时，该计算机设备可以在量化规则集合中，查找与结构70G相匹配的量化规则，进而可以将查找到的量化规则确定为用于对结构70G进行量化的目标量化规则(例如，上述图2所示的量化规则R6)。
基于此，该计算机设备可以基于该目标量化规则，将叠加层的第一输入分支的输入位置(即X1的输入位置)确定为位置D1，将叠加层的第二输入分支的输入位置(即X2的输入位置)确定为位置D2，将叠加层的输出位置确定为位置D3，进而可以将该位置D1、位置D2和位置D3分别确定为伪造算子在结构70G中的添加位置。
进一步地，该计算机设备可以在位置D1处添加伪造算子(例如，伪造算子7F1)，使得输入至结构71G的第一输入(例如，X1)在经过伪造算子7F1中的量化算子后转换为第二精度，再经过伪造算子7F1中的反量化算子后，还原为第一精度。同理，该计算机设备还可以在位置D2处添加伪造算子(例如，伪造算子7F2)，使得输入至结构71G的第二输入(例如，X2)在经过伪造算子7F2中的量化算子后转换为第二精度，再经过伪造算子7F2中的反量化算子后，还原为第一精度。该计算机设备还可以在位置D3处添加伪 造算子(例如，伪造算子7F3)，使得叠加层的输出在经过伪造算子7F3中的量化算子后，转换为第二精度，再经过伪造算子7F3中的反量化算子(例如，反量化算子2)后，还原为第一精度。
由此可见，本申请实施例在检测到结构70G这样的目标结构时，可以通过与结构70G相匹配的量化规则以及伪造算子(包括伪造算子7F1、伪造算子7F2和伪造算子7F3)，自动对结构70G进行量化，以得到量化后的结构70G(例如，图7所示的量化结构71G)。
又比如，在模型结构图中的目标结构包括第一拼接层，且该第一拼接层的输入分支与第一拼接层的输出分支的数量相同的情况下，该计算机设备可以子啊第一拼接层的N个输出分支的每个输出分支的输出位置处，分别添加用于进行量化的伪造算子；其中，这里的N可以为大于1的正整数，且为了后续能够优化结构，这里的每个输出分支处所添加的伪造算子的量化算子均相同。
为便于理解，进一步地，请参见图8，图8是本申请实施例提供的一种对目标结构进行量化的示意图五。如图8所示，该结构80G可以为该计算机设备在初始模型的模型结构图中所查找到的目标结构，该结构80G可以包括拼接层，该拼接层的输入分支的数量与输出分支的数量相同，为便于阐述，本申请实施例可以以两个输入分支和两个输出分支为例。
如图8所示，该结构80G中的拼接层可以对两个输入分支的输入(例如，X1和X2)进行拼接处理，从而得到拼接层的输出(例如，Y1和Y2)。若这里的X1和X2均属于第一精度，那么该结构80G的两个输出均为第一精度。可以理解的是，在对结构80G进行量化时，该计算机设备可以在量化规则集合中，查找与结构80G相匹配的量化规则，进而可以将查找到的量化规则确定为目标量化规则(例如，上述图2所示的量化规则R7)。
基于此，该计算机设备可以基于该目标量化规则，将拼接层的第一输出分支的输出位置确定为位置E1，且将拼接层的第二输出分支的输出位置确定为位置E2，进而可以将该位置E1和位置E2分别确定为伪造算子在结构80G中的添加位置。
进一步地，该计算机设备可以在位置E1处添加伪造算子(例如，伪造算子8F1)，使得拼接层的第一输出在经过伪造算子8F1中的量化算子后，转换为第二精度，再经过伪造算子8F1中的反量化算子(例如，反量化算子1)后，还原为第一精度。同理，该计算机设备还可以在位置E2处添加伪造算子(例如，伪造算子8F2)，使得拼接层的第二输出在经过伪造算子8F2中的量化算子后，转换为第二精度，再经过伪造算子8F2中的反量化算子(例如，反量化算子2)后，还原为第一精度。其中，该伪造算子8F1与伪造算子8F2中的量化算子相同，而伪造算子8F1与伪造算子8F2中的反量化算子可以相同，也可以不 同，这里将不对其进行限定。
由此可见，本申请实施例在检测到结构80G这样的目标结构时，可以通过与结构80G相匹配的量化规则以及伪造算子(包括伪造算子8F1和伪造算子8F2)，自动对结构80G进行量化，以得到量化后的结构80G(例如，图8所示的量化结构81G)。
步骤S103，计算机设备基于量化后的初始模型，得到用于业务处理的业务模型。
具体地，该计算机设备可以对模型结构图中存在的各种可加速的待量化结构分别进行量化，在量化结束后，该计算机设备可以将量化后的初始模型确定为第一模型。为了使得量化后的模型精度满足业务需求，该计算机设备还可以对第一模型进行微调训练，得到第二模型，进而可以基于该第二模型，得到用于业务处理的业务模型。
其中，在对第一模型进行微调训练时，该计算机设备可以获取样本业务数据以及样本业务数据对应的样本业务标签，进而可以将样本业务数据的样本特征输入至第一模型，通过第一模型中添加的伪造算子以及第一模型中的各网络层，对样本业务数据进行预测，得到样本业务数据的预测业务标签。此时，该计算机设备可以基于样本业务标签以及预测业务标签，对第一模型进行微调训练，得到第二模型，再对第二模型进行优化处理，得到用于业务处理的业务模型。
其中，这里的样本业务数据是基于初始模型所进行的业务处理所确定的。比如，若初始模型的业务处理为关键部位检测(例如，人脸检测)处理，则这里的样本业务数据可以为对获取到的某一人脸数据库(例如人脸数据集(Annotated Facial Landmarks in the Wild))中的人脸图像，采用局部切块、翻转、高斯模糊等方式，所生成人脸/非人脸训练集。此时，这里的样本业务标签可以用于指示样本业务数据的真实类别(例如，人脸或非人脸)。
其中，在本申请实施例中，涉及到的人脸(或其他生物特征)识别技术，在本申请以上实施例运用到具体产品或技术中时，相关数据收集、使用和处理过程应该遵守国家法律法规要求，收集人脸信息前应该告知信息处理规则并征求目标对象的单独同意，并严格遵照法律法规要求和个人信息处理规则处理人脸信息，采取技术措施保障相关数据安全。
又比如，若初始模型的业务处理为内容审核处理，则这里的样本业务数据可以为获取到的多媒体数据(例如，视频、音频、文本、图片等)，此时，这里的样本业务标签可以用于指示样本业务数据的内容质量的真实影响度，影响度越高，意味着该样本业务数据的内容质量越高。
又比如，若初始模型的业务处理为图像识别处理，则这里的样本业务数据可以为获取到的图片数据，此时，这里的样本业务标签可以用于指示样本业务数据的真实类别(例如， 人、猫、狗等)。当然，该初始模型的业务处理还可以包括其他处理，这里将不对其进行一一举例。
在本申请中，涉及到的数据抓取技术方案实施(例如，内容审核处理时所抓取到的多媒体数据或图像识别处理时所抓取到的图片数据)，在本申请以上实施例运用到具体产品或技术中时，相关数据收集、使用和处理过程应该遵守国家法律法规要求，符合合法、正当、必要的原则，不涉及获取法律法规禁止或限制的数据类型，不会妨碍目标网站的正常运行。
可以理解的是，当第二模型的网络结构比较复杂时，本申请实施例需要对第二模型中的待优化结构进行优化处理，使得第二模型中的尽可能多的算子进行融合，以得到业务模型。其中，由于第二模型是经过微调训练的已量化的初始模型，因此，该第二模型的模型结构图、第一模型的模型结构图均与初始模型的模型结构图相同，本申请实施例可以将第二模型中具有权重的网络层也称之为具有权重的第一网络层，依次类推，将第二模型中的输入分支的类型均为非加权操作的输入类型的叠加层称之为第一叠加层；将第二模型中的输入分支与输出分支的数量相同的拼接层称之为第一拼接层。
比如，在初始模型的目标结构包括具有权重的第一网络层的情况下，第二模型中的待优化结构也包括具有权重的第一网络层，那么，基于该第二模型所最终得到的业务模型可以包括具有权重的网络层(即第二网络层)、添加在具有权重的第二网络层的输入位置处的量化算子；在该业务模型中，具有权重的第二网络层是根据具有权重的第一网络层得到的，且该具有权重的第二网络层的权重是采用量化算子，对第二模型中的具有权重的第一网络层的权重进行处理后所得到的。
为便于理解，进一步地，请参见图9，图9是本申请实施例提供的一种对第二模型中的待优化结构进行处理的示意图一。如图9所示，这里的结构91G可以为第二模型中的待优化结构，该结构91G是该计算机设备对初始模型中的目标结构进行量化，并微调训练后所得到的，该结构91G与上述图4所示的结构41G的结构相同。
其中，该结构91G中可以包括伪造算子F1(即由量化算子1和反量化算子1所构成的伪造算子)、伪造算子F2(即由量化算子2和反量化算子2所构成的伪造算子)以及伪造算子F3(即由量化算子3和反量化算子3所构成的伪造算子)。
应当理解，该计算机设备在对结构91G进行优化处理时，可以根据结构91G中的具有权重的第一网络层，得到具有权重的第二网络层，进而可以采用量化算子，对结构91G中的具有权重的第一网络层的权重进行处理，得到具有权重的第二网络层的权重，比如，该 计算机设备可以过滤掉反量化算子1和反量化算子2，并采用量化算子2以及量化算子3，对结构91G中的具有权重的第一网络层的权重进行处理，得到具有权重的第二网络层的权重。
这意味着该计算机设备对第二模型中的结构91G进行优化处理后，可以得到业务模型中的优化结构(即图9所示的结构92G)。如图9所示，该结构92G可以包括具有权重的第二网络层，添加在该具有权重的第二网络层的输入位置处的量化算子(例如，量化算子1)。
又比如，在初始模型的目标结构中的具有权重的第一网络层为卷积层，且该目标结构还包括与卷积层连接的归一化层以及与归一化层连接的激活层的情况下，该计算机设备还需要对具有权重的第一网络层、归一化层和激活层进行融合，得到具有权重的第二网络层。
为便于理解，进一步地，请参见图10，图10是本申请实施例提供的一种对第二模型中的待优化结构进行处理的示意图二。如图10所示，这里的结构101G可以为第二模型中的待优化结构，该结构101G是该计算机设备对初始模型中的目标结构进行量化，并微调训练后所得到的，这里的目标结构中具有权重的第一网络层为卷积层，且该目标结构还包括与该卷积层连接的归一化层以及与归一化层连接的激活层。该结构101G与上述图5所示的结构51G的结构相同。
其中，该结构101G中可以包括伪造算子F1(即由量化算子1和反量化算子1所构成的伪造算子)和伪造算子F2(即由量化算子2和反量化算子2所构成的伪造算子)。应当理解，该计算机设备在对结构101G进行优化处理时，可以根据结构101G中的卷积层，得到结构102G中的融合后的卷积层。比如，该计算机设备可以对结构101G中的卷积层、归一化层和激活层进行融合，得到融合后的卷积层(即具有权重的第二网络层)。
与此同时，该计算机设备还可以采用量化算子，对结构101G中的卷积层的权重进行处理，得到融合后的卷积层的权重，比如，该计算机设备可以过滤掉反量化算子1和反量化算子2，并采用量化算子2，对结构101G中的卷积层的权重进行处理，得到融合后的卷积层的权重。
这意味着该计算机设备对第二模型中的结构101G进行优化处理后，可以得到业务模型中的优化结构(即图10所示的结构102G)。如图10所示，该结构102G可以包括具有权重的第二网络层，添加在该具有权重的第二网络层的输入位置处的量化算子(例如，量化算子1)。
又比如，在初始模型的目标结构包括叠加层、叠加层的第一输入分支和叠加层的第二 输入分支，且第一输入分支包括依次连接的具有权重的第一网络层和归一化层的情况下，该计算机设备还可以对具有权重的第一网络层、归一化层和叠加层进行融合，得到具有权重的第二网络层。其中，这里的归一化层与叠加层连接，第二输入分支的类型为非加权操作的输入类型，具有权重的第一网络层为卷积层。
为便于理解，进一步地，请参见图11，图11是本申请实施例提供的一种对第二模型中的待优化结构进行处理的示意图三。如图11所示，这里的结构111G可以为第二模型中的待优化结构，该结构111G是该计算机设备对初始模型中的目标结构进行量化，并微调训练后所得到的，这里的目标结构包括叠加层，叠加层的第一输入分支、叠加层的第二输入分支。其中，该第一输入分支包括依次连接的具有权重的第一网络层和归一化层，且该具有权重的第一网络层为卷积层，归一化层与叠加层连接。该第二输入分支的类型为非加权操作的输入类型。该结构111G与上述图6所示的结构61G的结构相同。
其中，该结构111G中可以包括伪造算子F1(即由量化算子1和反量化算子1所构成的伪造算子)、伪造算子F2(即由量化算子2和反量化算子2所构成的伪造算子)、伪造算子F3(即由量化算子3和反量化算子3所构成的伪造算子)、伪造算子F4(即由量化算子4和反量化算子4所构成的伪造算子)。
应当理解，该计算机设备在对结构111G进行优化处理时，可以根据结构111G中的卷积层，得到结构112G中的融合后的卷积层。比如，该计算机设备可以对结构111G中的卷积层、归一化层和激活层进行融合，得到融合后的卷积层(即具有权重的第二网络层)。
与此同时，该计算机设备还可以采用量化算子，对结构111G中的卷积层的权重进行处理，得到融合后的卷积层的权重，比如，该计算机设备可以过滤掉反量化算子1、反量化算子2和反量化算子3，并采用量化算子2以及量化算子4，对结构111G中的卷积层的权重进行处理，得到融合后的卷积层的权重。
这意味着该计算机设备对第二模型中的结构111G进行优化处理后，可以得到业务模型中的优化结构(即图11所示的结构112G)。如图11所示，该结构112G可以包括具有权重的第二网络层，添加在该具有权重的第二网络层的输入位置处的量化算子(例如，量化算子1和量化算子3)。
又比如，在初始模型的目标结构包括第一叠加层，且第一叠加层的第一输入分支的类型与第一叠加层的第二输入分支的类型均为非加权操作的输入类型的情况下，那么，基于该第二模型所最终得到的业务模型可以包括第二叠加层、添加在第二叠加层的第一输入分支处的量化算子、和添加在第二叠加层的第二输入分支处的量化算子。在该业务模型中， 第二叠加层是根据量化算子以及第二模型中的第一叠加层所得到的。
为便于理解，进一步地，请参见图12，图12是本申请实施例提供的一种对第二模型中的待优化结构进行处理的示意图四。如图12所示，这里的结构121G可以为第二模型中的待优化结构，该结构121G是该计算机设备对初始模型中的目标结构进行量化，并微调训练后所得到的，这里的目标结构可以包括第一叠加层，且第一叠加层的第一输入分支的类型与第一叠加层的第二输入分支的类型均为非加权操作的输入类型。该结构121G可以为上述图7所示的结构71G的结构相同。
其中，该结构121G中可以包括伪造算子F1(即由量化算子1和反量化算子1所构成的伪造算子)、伪造算子F2(即由量化算子2和反量化算子2所构成的伪造算子)以及伪造算子F3(即由量化算子3和反量化算子3所构成的伪造算子)。应当理解，该计算机设备在对结构121G进行优化处理时，可以根据量化算子以及第二模型中的第一叠加层，得到结构122G中的第二叠加层。比如，该计算机设备可以过滤掉反量化算子1和反量化算子2，并采用量化算子3对第一叠加层进行处理后，得到第二叠加层。
这意味着该计算机设备对第二模型中的结构121G进行优化处理后，可以得到业务模型中的优化结构(即图12所示的结构122G)。如图12所示，该结构122G可以包括第二叠加层、添加在第二叠加层的第一输入分支处的量化算子(即量化算子1)、和添加在第二叠加层的第二输入分支处的量化算子(即量化算子2)。
可以理解的是，本申请实施例可以根据等价传播规则适当移动量化算子和反量化算子的位置，以便于尽可能多的将待优化模型的多个算子进行融合，简单来说，需要尽可能的将反量化算子推迟，以推迟反量化操作，尽可能的将量化算子提前，以提前量化操作，这样将使得参数以低精度的形式在模型中的尽量多的网络层中进行传播，从而提升模型速度。
基于此，在初始模型的目标结构包括第一拼接层，且第一拼接层的输入分支与第一拼接层的输出分支的数量相同的情况下，那么，基于该第二模型所最终得到的业务模型可以包括第二拼接层、添加在第二拼接层的每个输入分支的输入位置处的量化算子。在该业务模型中，第二拼接层的每个输入分支的输入位置处的量化算子是将添加在第二模型中的第一拼接层的每个输出分支的输出位置处的量化算子，分别等价向前传播至第二模型中的第一拼接层的每个输入分支的输入位置处后所得到的。
为便于理解，进一步地，请参见图13，图13是本申请实施例提供的一种对第二模型中的待优化结构进行处理的示意图五。如图13所示，这里的结构131G可以为第二模型中 的待优化结构，该结构131G是该计算机设备对初始模型中的目标结构进行量化后所得到的，该目标结构可以包括第一拼接层，且第一拼接层的输入分支与第一拼接层的输出分支的数量相同，这里的数量以2为例。其中，该结构131G与上述图8所示的结构81G的结构相同。
其中，该结构131G中可以包括伪造算子F1(即由量化算子1和反量化算子1所构成的伪造算子)和伪造算子F2(即由量化算子2和反量化算子2所构成的伪造算子)，且这里的量化算子1等于量化算子2。
基于此，该计算机设备在对结构131G进行优化处理时，可以将量化算子1的位置由第一拼接层的第一输出位置等价向前传播至第一拼接层的第一输入位置(即X1对应的输入位置)，同理，还需要将量化算子2的位置由第一拼接层的第二输出位置等价向前传播至第一拼接层的第二输入位置(即X2对应的输入位置)，进而可以得到结构132G。此时，该计算机设备可以将结构132G中的拼接层称之为第二拼接层。
这意味着该计算机设备对第二模型中的结构131G进行优化处理后，可以得到业务模型中的优化结构(即图13所示的结构132G)。如图13所示，该结构132G可以包括第二拼接层、添加在第二拼接层的每个输入分支的输入位置处的量化算子。在结构132G中，该计算机设备可以通过量化算子1将X1的精度由第一精度转换为第二精度，通过量化算子2将X2的精度由第一精度转换为第二精度，进而可以通过结构132G中的第二拼接层，将属于第二精度的X1与属于第二精度的X2进行拼接处理，从而使得后续拼接处理后所得到的参数仍为第二精度。
在本申请实施例中，初始模型对应的业务对象无需手动在初始模型中的相应位置处添加伪造算子，而是直接将任意一种初始模型提交至计算机设备，以使计算机设备调用通用型量化工具，来获取预先设定好的量化规则集合，进而基于该通用型量化工具以及量化规则集合，对初始模型中存在的可加速的待量化结构进行量化，整个量化过程可在通用型量化工具内部完成，从而可以有效降低人工成本，提升了模型量化速度，实现了量化通用性。此外，由于该通用型量化工具还支持对量化后的初始模型进行微调训练，使得后续得到的业务模型在满足业务指标的同时还能够获取最高的加速比。又因对第二模型中的待优化结构进行优化处理，这样将使得后续得到的业务模型能够提升模型的每秒查询率(query per second，QPS)，进而还可减少模型部署内存以及显存占用等。
进一步地，请参见图14，图14是本申请实施例提供的一种数据处理方法的流程示意图。该方法可以由具有模型量化功能的终端设备(例如，上述图1所示的终端设备集群中 的任意一个终端设备，例如，终端设备100a)执行，也可以由具有模型量化功能的服务器(例如，上述图1所示的服务器10S)执行，还可以由具备模型应用功能的终端设备和具备模型量化功能的服务器交互执行，在此不做限定。该方法至少可以包括步骤S201-步骤S209。
步骤S201，在获取到初始模型时，对初始模型进行结构搜索，得到初始模型的模型结构图。
步骤S202，根据模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对目标结构进行量化。
其中，伪造算子包括量化算子和反量化算子。
步骤S203，基于量化后的初始模型，确定用于业务处理的业务模型。
其中，本申请实施例中的数据处理方法可以包括模型量化过程以及模型应用过程。可以理解的是，该步骤S201-步骤S203阐述了模型量化过程，该模型训练过程的具体实施方式可参见上述图3所对应实施例中对步骤S101-步骤S103的描述，这里将不再继续进行赘述。
为便于理解，进一步地，请参见图15，图15是本申请实施例提供的一种对业务模型进行模型优化的场景示意图。其中，本申请实施例中的计算机设备可以为具有模型量化功能的计算机设备，该计算机设备可以为上述图1所示的终端设备集群中的任意一个终端设备，例如，终端设备100b，该计算机设备也可以为上述图1所示的服务器10S，这里将不对计算机设备进行限定。
图15所示的模型151W为该计算机设备所获取到的初始模型。该计算机设备在获取到图15所示的模型151W时，可以调用通用型量化工具，对模型151W进行结构搜索，以得到该模型151W的模型结构图。进一步地，该计算机设备可以基于预先设定好的量化规则集合对应的N个待量化结构，在模型151W的模型结构图中查询是否存在属于N个待量化结构的目标结构。
由于该模型151W的模型结构图中的结构15G1属于可加速的待量化结构，因此，该计算机设备可以从量化规则集合中的N个量化规则中确定与结构15G1相匹配的目标量化规则(例如，上述图2所示的量化规则R4)，进一步地，该计算机设备可以基于目标量化规则以及伪造算子，对结构15G1进行量化，以得到量化后的目标结构(例如，图15所示的结构15G2)。其中，该计算机设备对结构15G1进行量化的具体实施方式可以参见上述图5所示的对结构50G进行量化的具体实施方式，这里将不再对其进行赘述。
在量化结束后，该计算机设备可以得到图15所示的模型152W(即第一模型)，进一步地，该计算机设备可以对模型152W进行微调训练，进而可以将微调训练后的模型152W称之为第二模型，然后该计算机设备可以在第二模型中遍历待优化结构(即已经量化并训练的目标结构)，进而可以根据待优化结构中的量化算子以及待优化结构中的原网络层，得到优化结构。比如，该计算机设备在遍历到第二模型中的结构15G2时，可以参见上述图10所示的优化处理方式，对结构15G2进行优化处理，以得到结构15G2对应的优化结构(例如，图15所示的结构15G3)。
在遍历完成时，该计算机设备可以将优化处理后的第二模型确定为第三模型，进而基于第三模型所部署的业务平台，对第三模型进行模型推理，得到用于业务处理的业务模型(例如，图15所示的模型153W)。
由此可见，本申请实施例在获取到初始模型时，通过通用型量化工具，能够自动对初始模型进行模型量化、微调训练以及模型推理等优化过程，从而使得后续得到的业务模型能够提升模型的每秒查询率，进而还可减少模型部署内存以及显存占用等。
可以理解的是，这里的业务模型可以适用于多种应用场景，因此，这里的业务模型的业务处理可以为关键部位检测处理，也可以为内容审核处理，还可以为图像识别处理，这里将不对其进行限定。为便于理解，本申请实施例可以以图像识别为例，用以阐述业务模型的应用过程，其中，该模型应用过程具体可以参见下述步骤S204-步骤S208的描述。
步骤S204，若业务模型的业务处理为图像识别，则在获取到待识别图像时，获取待识别图像的图像特征。
具体地，若业务模型的业务处理为图像识别，则该计算机设备在获取到待识别图像时，可以对该待识别图像进行编码处理，以得到待识别图像的图像特征。
步骤S205，将图像特征输入至业务模型。
具体地，该计算机设备可以将待识别图像的图像特征输入至业务模型，由于这里的业务模型是对高精度的初始模型进行模型量化、微调训练以及推理处理后所得到的模型，这意味着该业务模型中存在优化结构(例如，图15所示的结构15G3)。其中，这里的优化结构包括目标网络层和添加在目标网络层的输入位置处的量化算子(例如，图15所示的结构15G3的输入位置之前量化算子1)；该目标网络层是根据量化算子，以及初始模型中的目标结构的原网络层所得到的。
步骤S206，基于添加在目标网络层的输入位置处的量化算子，将优化结构的输入特征的精度由第一精度转换为第二精度。
具体地，该计算机设备可以基于添加在目标网络层的输入位置处的量化算子，对优化结构的输入特征进行量化操作，以将输入特征的精度由第一精度转换为第二精度。其中，第一精度高于第二精度，这意味着本申请实施例可以通过量化算子，将输入至该优化结构的参数的精度由高精度转换低精度，从而达到加速的目的。
步骤S207，通过目标网络层，对属于第二精度的输入特征进行特征提取处理，由目标网络层输出输入特征对应的输出特征；
其中，目标网络层的输出特征仍为第一精度。
步骤S208，基于输出特征，得到待识别图像的预测类别。
具体地，该计算机设备可以将优化结构的输出特征(即目标网络层的输出特征)传输至该业务模型中的优化结构的下一网络层，直到由该业务模型预测出该待识别图像针对多个分类类别的影响度，进而可以在这多个分类类别的影响度中选择最大影响度对应的分类类别作为该待识别图像的预测类别。
比如，若业务模型能够识别的分类类别的数量为3个，具体可以包括类别1(例如，人)、类别2(例如，猫)，类型3(例如，狗)，则该计算机设备在通过该业务模型可以预测出待识别图像针对类别1的影响度为15％，针对类别2的影响度为80％，针对类别3的影响度为5％，然后，该计算机设备可以将最大影响度对应的分类类别(即类别2)，作为待识别图像的预测类别。
在本申请实施例中，初始模型对应的业务对象无需手动在初始模型中的相应位置处添加伪造算子，而是直接将任意一种初始模型提交至计算机设备，以使计算机设备调用通用型量化工具，来获取预先设定好的量化规则集合，进而基于该通用型量化工具以及量化规则集合，对初始模型中存在的可加速的待量化结构进行量化，整个量化过程可在通用型量化工具内部完成，从而可以有效降低人工成本，提升了模型量化速度，实现了量化通用性。此外，由于该通用型量化工具还支持对量化后的初始模型进行微调训练，使得后续得到的业务模型在满足业务指标的同时还能够获取最高的加速比，即该计算机设备可以通过业务模型，快速且准确的进行业务处理(例如，图像识别处理等)。
进一步地，请参见图16，图16是本申请实施例提供的一种数据处理装置的结构示意图。如图16所示，该数据处理装置1可以包括：搜索模块10，量化模块20，业务模型确定模块30，特征获取模块40，特征输入模块50，精度转换模块60，特征提取模块70以及类别确定模块80。
该搜索模块10，用于在获取到初始模型时，对初始模型进行结构搜索，得到初始模型 的模型结构图。
该量化模块20，用于根据模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对目标结构进行量化；伪造算子包括量化算子和反量化算子。
其中，该量化模块20包括：第一量化单元201，第二量化单元202，第三量化单元203，第四量化单元204，第五量化单元205，第六量化单元206以及第七量化单元207。
该第一量化单元201，用于在模型结构图中的目标结构包括具有权重的第一网络层的情况下，在具有权重的第一网络层的输入位置处，添加用于进行量化的伪造算子，且采用伪造算子对具有权重的第一网络层的权重进行量化。
该第二量化单元202，用于在具有权重的第一网络层具有多个输出位置的情况下，在具有权重的第一网络层的多个输出位置处，分别添加用于进行量化的伪造算子。
该第三量化单元203，用于在具有权重的第一网络层具有一个输出位置，且具有权重的第一网络层的输出位置处未连接激活层的情况下，在具有权重的第一网络层的输出位置处，添加用于进行量化的伪造算子。
其中，在具有权重的第一网络层具有一个输出位置，且具有权重的第一网络层的输出位置处连接激活层的情况下，不在具有权重的第一网络层的输出位置处，添加用于进行量化的伪造算子。
该第四量化单元204，用于在模型结构图中的目标结构包括叠加层、叠加层的第一输入分支和叠加层的第二输入分支的情况下，在叠加层的输出位置处添加伪造算子；
其中，第一输入分支包括依次连接的具有权重的第一网络层和归一化层，归一化层与叠加层连接，第二输入分支的类型为非加权操作的输入类型，具有权重的第一网络层为卷积层。
该第五量化单元205，用于在第二输入分支的输入位置处添加伪造算子。
该第六量化单元206，用于在模型结构图中的目标结构包括第一叠加层，且第一叠加层的第一输入分支的类型与第一叠加层的第二输入分支的类型均为非加权操作的输入类型的情况下，在第一输入分支的输入位置处添加用于进行量化的伪造算子，在第二输入分支的输入位置处添加伪造算子，且在第一叠加层的输出位置处添加伪造算子。
该第七量化单元207，用于在模型结构图中的目标结构包括第一拼接层，且第一拼接层的输入分支与第一拼接层的输出分支的数量相同的情况下，在第一拼接层的N个输出分支的每个输出分支的输出位置处，分别添加用于进行量化的伪造算子；N为大于1的正整数；每个输出分支处所添加的伪造算子的量化算子均相同。
其中，该第一量化单元201，第二量化单元202，第三量化单元203，第四量化单元204，第五量化单元205，第六量化单元206以及第七量化单元207的具体实现方式可以参见上述图3所对应实施例中对步骤S102的描述，这里将不再继续进行赘述。
该业务模型确定模块30，用于基于量化后的初始模型，得到用于业务处理的业务模型。
其中，该业务模型确定模块30包括：第一训练单元301，第一确定单元302，第一融合单元303，第二融合单元304，第二训练单元305，第二确定单元306，第三训练单元307以及第三确定单元308。
该第一训练单元301，用于将量化后的初始模型确定为第一模型，对第一模型进行微调训练，得到第二模型；
该第一确定单元302，用于基于第二模型，得到用于业务处理的业务模型；业务模型包括具有权重的第二网络层、和添加在具有权重的第二网络层的输入位置处的量化算子；在业务模型中，具有权重的第二网络层是根据具有权重的第一网络层得到的，且具有权重的第二网络层的权重是采用量化算子，对第二模型中的具有权重的第一网络层的权重进行处理后所得到的。
其中，目标结构中的具有权重的第一网络层为卷积层，且目标结构还包括与卷积层连接的归一化层，以及与归一化层连接的激活层；
该第一融合单元303，用于对具有权重的第一网络层、归一化层和激活层进行融合，得到具有权重的第二网络层。
其中，目标结构包括叠加层、叠加层的第一输入分支和叠加层的第二输入分支；第一输入分支包括依次连接的具有权重的第一网络层和归一化层；归一化层与叠加层连接，第二输入分支的类型为非加权操作的输入类型，具有权重的第一网络层为卷积层；
该第二融合单元304，用于对具有权重的第一网络层、归一化层和叠加层进行融合，得到具有权重的第二网络层。
该第二训练单元305，用于将量化后的初始模型确定为第一模型，对第一模型进行微调训练，得到第二模型；
该第二确定单元306，用于基于第二模型，得到用于业务处理的业务模型；业务模型包括第二叠加层、添加在第二叠加层的第一输入分支处的量化算子、和添加在第二叠加层的第二输入分支处的量化算子；在业务模型中，第二叠加层是根据量化算子以及第二模型中的第一叠加层所得到的。
该第三训练单元307，用于将量化后的初始模型确定为第一模型，对第一模型进行微 调训练，得到第二模型；
该第三确定单元308，用于基于第二模型，得到用于业务处理的业务模型；业务模型包括第二拼接层、和添加在第二拼接层的每个输入分支的输入位置处的量化算子；在业务模型中，第二拼接层的每个输入分支的输入位置处的量化算子是将添加在第二模型中的第一拼接层的每个输出分支的输出位置处的量化算子，分别等价向前传播至第二模型中的第一拼接层的每个输入分支的输入位置处后所得到的。
其中，该第一训练单元301，第一确定单元302，第一融合单元303，第二融合单元304，第二训练单元305，第二确定单元306，第三训练单元307以及第三确定单元308的具体实现方式可以参见上述图3所对应实施例中对步骤S103的描述，这里将不再继续进行赘述。
该特征获取模块40，用于若业务模型的业务处理为图像识别，则在获取到待识别图像时，获取待识别图像的图像特征。
该特征输入模块50，用于将图像特征输入至业务模型；业务模型中的优化结构包括目标网络层、和添加在目标网络层的输入位置处的量化算子；目标网络层是根据量化算子，以及初始模型中的目标结构的原网络层所得到的。
该精度转换模块60，用于基于添加在目标网络层的输入位置处的量化算子，将优化结构的输入特征的精度由第一精度转换为第二精度；第一精度高于第二精度。
该特征提取模块70，用于通过目标网络层，对属于第二精度的输入特征进行特征提取处理，由目标网络层输出输入特征对应的输出特征。
该类别确定模块80，用于基于输出特征，得到待识别图像的预测类别。
其中，该搜索模块10，量化模块20，业务模型确定模块30，特征获取模块40，特征输入模块50，精度转换模块60，特征提取模块70以及类别确定模块80的具体实现方式可以参见上述图14所对应实施例中对步骤S201-步骤S208的描述，这里将不再继续进行赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
进一步地，请参见图17，图17是本申请实施例提供的一种计算机设备的示意图。如图17所示，该计算机设备1000可以包括：至少一个处理器1001，例如，CPU，至少一个网络接口1004，存储器1005，至少一个通信总线1002。其中，通信总线1002用于实现这些组件之间的连接通信。网络接口1004可选地可以包括标准的有线接口、无线接口(如WI-FI接口)。存储器1005可以是高速RAM存储器，也可以是非不稳定的存储器(non-volatile memory)，例如至少一个磁盘存储器。存储器1005可选地还可以是至少一 个位于远离前述处理器1001的存储装置。如图17所示，作为一种计算机存储介质的存储器1005可以包括操作系统、网络通信模块、用户接口模块以及设备控制应用程序。其中，在一些实施例中，该计算机设备还可以包括图17所示的用户接口1003，比如，若该计算机设备为图1所示的具有模型量化功能的终端设备(例如，终端设备100b)，则该计算机设备还可以包括该用户接口1003，其中，该用户接口1003可以包括显示屏(Display)、键盘(Keyboard)等。
在图17所示的计算机设备1000中，网络接口1004主要用于进行网络通信；而用户接口1003主要用于为用户提供输入的接口；而处理器1001可以用于调用存储器1005中存储的设备控制应用程序，以实现：
在获取到初始模型时，对初始模型进行结构搜索，得到初始模型的模型结构图；
根据模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对目标结构进行量化；伪造算子包括量化算子和反量化算子；
基于量化后的初始模型，得到用于业务处理的业务模型。
应当理解，本申请实施例中所描述的计算机设备1000可执行前文图3和图14所对应实施例中对该数据处理方法的描述，也可执行前文图16所对应实施例中对该数据处理装置1的描述，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
本申请实施例还提供一种计算机可读存储介质，该计算机可读存储介质存储有计算机程序，该计算机程序包括程序指令，该程序指令被处理器执行时实现图3和图14中各个步骤所提供的数据处理方法，具体可参见图3以及图14各个步骤所提供的实现方式，在此不再赘述。
计算机可读存储介质可以是前述任一实施例提供的数据传输装置或者计算机设备的内部存储单元，例如计算机设备的硬盘或内存。该计算机可读存储介质也可以是该计算机设备的外部存储设备，例如该计算机设备上配备的插接式硬盘，智能存储卡(smart media card，SMC)，安全数字(secure digital，SD)卡，闪存卡(flash card)等。进一步地，该计算机可读存储介质还可以既包括该计算机设备的内部存储单元也包括外部存储设备。该计算机可读存储介质用于存储该计算机程序以及该计算机设备所需的其他程序和数据。该计算机可读存储介质还可以用于暂时地存储已经输出或者将要输出的数据。
本申请实施例还提供了一种计算机程序产品，该计算机程序产品包括计算机程序，该计算机程序存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机程序，处理器执行该计算机程序，使得该计算机设备可执行前文各实施例中对 数据处理方法或装置的描述，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
本申请实施例的说明书和权利要求书及附图中的术语“第一”、“第二”等是用于区别不同对象，而非用于描述特定顺序。此外，术语“包括”以及它们任何变形，意图在于覆盖不排他的包括。例如包括了一系列步骤或单元的过程、方法、装置、产品或设备没有限定于已列出的步骤或模块，而是可选地还包括没有列出的步骤或模块，或可选地还包括对于这些过程、方法、装置、产品或设备固有的其他步骤单元。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
以上所揭露的仅为本申请较佳实施例而已，当然不能以此来限定本申请之权利范围，因此依本申请权利要求所作的等同变化，仍属本申请所涵盖的范围。

Claims (19)

  1. 一种数据处理方法，其特征在于，包括：
    在获取到初始模型时，对所述初始模型进行结构搜索，得到所述初始模型的模型结构图；
    根据所述模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对所述目标结构进行量化，所述伪造算子包括量化算子和反量化算子；
    基于量化后的初始模型，得到用于业务处理的业务模型。
  2. 根据权利要求1所述的数据处理方法，其特征在于，所述根据所述模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对所述目标结构进行量化，包括：
    在所述模型结构图中的目标结构包括具有权重的第一网络层的情况下，在所述具有权重的第一网络层的输入位置处，添加所述伪造算子，且采用所述伪造算子对所述具有权重的第一网络层的权重进行量化。
  3. 根据权利要求2所述的数据处理方法，其特征在于，所述数据处理方法还包括：
    在所述具有权重的第一网络层具有多个输出位置的情况下，在所述具有权重的第一网络层的多个输出位置处，分别添加所述伪造算子。
  4. 根据权利要求2所述的数据处理方法，其特征在于，所述数据处理方法还包括：
    在所述具有权重的第一网络层具有一个输出位置，且所述具有权重的第一网络层的输出位置处未连接激活层的情况下，在所述具有权重的第一网络层的输出位置处，添加所述伪造算子。
  5. 根据权利要求2所述的数据处理方法，其特征在于，所述数据处理方法还包括：
    在所述具有权重的第一网络层具有一个输出位置，且所述具有权重的第一网络层的输出位置处连接激活层的情况下，不在所述具有权重的第一网络层的输出位置处，添加所述伪造算子。
  6. 根据权利要求2所述的数据处理方法，其特征在于，所述数据处理方法还包括：
    在所述模型结构图中的目标结构包括叠加层、所述叠加层的第一输入分支和所述叠加 层的第二输入分支的情况下，在所述叠加层的输出位置处添加所述伪造算子；
    其中，所述第一输入分支包括依次连接的所述具有权重的第一网络层和归一化层，所述归一化层与所述叠加层连接，所述第二输入分支的类型为非加权操作的输入类型，所述具有权重的第一网络层为卷积层。
  7. 根据权利要求6所述的数据处理方法，其特征在于，所述数据处理方法还包括：
    在所述第二输入分支的输入位置处添加所述伪造算子。
  8. 根据权利要求1所述的数据处理方法，其特征在于，所述根据所述模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对所述目标结构进行量化，包括：
    在所述模型结构图中的目标结构包括第一叠加层，且所述第一叠加层的第一输入分支的类型与所述第一叠加层的第二输入分支的类型均为非加权操作的输入类型的情况下，在所述第一输入分支的输入位置处添加所述伪造算子，在所述第二输入分支的输入位置处添加所述伪造算子，且在所述第一叠加层的输出位置处添加所述伪造算子。
  9. 根据权利要求1所述的数据处理方法，其特征在于，所述根据所述模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对所述目标结构进行量化，包括：
    在所述模型结构图中的目标结构包括第一拼接层，且所述第一拼接层的输入分支与所述第一拼接层的输出分支的数量相同的情况下，在所述第一拼接层的N个输出分支的每个输出分支的输出位置处，分别添加所述伪造算子，N为大于1的正整数，每个输出分支处所添加的所述伪造算子的量化算子均相同。
  10. 根据权利要求2所述的数据处理方法，其特征在于，所述基于量化后的初始模型，得到用于业务处理的业务模型，包括：
    将量化后的初始模型确定为第一模型，对所述第一模型进行微调训练，得到第二模型；
    基于所述第二模型，得到用于业务处理的业务模型，
    其中，所述业务模型包括具有权重的第二网络层、和添加在所述具有权重的第二网络层的输入位置处的量化算子，在所述业务模型中，所述具有权重的第二网络层是根据具有权重的第一网络层得到的，且所述具有权重的第二网络层的权重是采用量化算子，对所述第二模型中的具有权重的第一网络层的权重进行处理后所得到的。
  11. 根据权利要求10所述的数据处理方法，其特征在于，所述目标结构中的具有权重的第一网络层为卷积层，且所述目标结构还包括与所述卷积层连接的归一化层，以及与所述归一化层连接的激活层；
    所述数据处理方法还包括：
    对所述具有权重的第一网络层、所述归一化层和所述激活层进行融合，得到所述具有权重的第二网络层。
  12. 根据权利要求10所述的数据处理方法，其特征在于，所述目标结构包括叠加层、所述叠加层的第一输入分支和所述叠加层的第二输入分支；所述第一输入分支包括依次连接的所述具有权重的第一网络层和归一化层；所述归一化层与所述叠加层连接，所述第二输入分支的类型为非加权操作的输入类型，所述具有权重的第一网络层为卷积层；
    所述数据处理方法还包括：
    对所述具有权重的第一网络层、所述归一化层和所述叠加层进行融合，得到所述具有权重的第二网络层。
  13. 根据权利要求8所述的数据处理方法，其特征在于，所述基于量化后的初始模型，得到用于业务处理的业务模型，包括：
    将量化后的初始模型确定为第一模型，对所述第一模型进行微调训练，得到第二模型；
    基于所述第二模型，得到用于业务处理的业务模型，
    其中，所述业务模型包括第二叠加层、添加在所述第二叠加层的第一输入分支处的量化算子、和添加在所述第二叠加层的第二输入分支处的量化算子，在所述业务模型中，所述第二叠加层是根据量化算子以及所述第二模型中的第一叠加层所得到的。
  14. 根据权利要求9所述的数据处理方法，其特征在于，所述基于量化后的初始模型，得到用于业务处理的业务模型，包括：
    将量化后的初始模型确定为第一模型，对所述第一模型进行微调训练，得到第二模型；
    基于所述第二模型，得到用于业务处理的业务模型，
    其中，所述业务模型包括第二拼接层、和添加在所述第二拼接层的每个输入分支的输入位置处的量化算子，在所述业务模型中，所述第二拼接层的每个输入分支的输入位置处 的量化算子是将添加在所述第二模型中的第一拼接层的每个输出分支的输出位置处的量化算子，分别等价向前传播至所述第二模型中的第一拼接层的每个输入分支的输入位置处后所得到的。
  15. 根据权利要求1所述的数据处理方法，其特征在于，所述数据处理方法还包括：
    若所述业务模型的业务处理为图像识别，则在获取到待识别图像时，获取所述待识别图像的图像特征；
    将所述图像特征输入至所述业务模型；所述业务模型中的优化结构包括目标网络层、和添加在所述目标网络层的输入位置处的量化算子；所述目标网络层是根据量化算子，以及所述初始模型中的所述目标结构的原网络层所得到的；
    基于添加在所述目标网络层的输入位置处的量化算子，将所述优化结构的输入特征的精度由第一精度转换为第二精度，所述第一精度高于所述第二精度；
    通过所述目标网络层，对属于所述第二精度的输入特征进行特征提取处理，由所述目标网络层输出所述输入特征对应的输出特征；
    基于所述输出特征，得到所述待识别图像的预测类别。
  16. 一种数据处理装置，其特征在于，包括：
    搜索模块，用于在获取到初始模型时，对所述初始模型进行结构搜索，得到所述初始模型的模型结构图；
    量化模块，用于根据所述模型结构图中的目标结构的种类，采用用于进行量化的伪造算子，对所述目标结构进行量化，所述伪造算子包括量化算子和反量化算子；
    业务模型确定模块，用于基于量化后的初始模型，得到用于业务处理的业务模型。
  17. 一种计算机设备，其特征在于，包括：处理器和存储器以及网络接口；
    所述处理器与所述存储器、所述网络接口相连，其中，所述网络接口用于提供数据通信功能，所述存储器用于存储计算机程序，所述处理器用于调用所述计算机程序，以使得所述计算机设备执行权利要求1至15任一项所述的数据处理方法。
  18. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机程序，所述计算机程序适于由处理器加载并执行，以使得具有所述处理器的计算机设备 执行权利要求1至15任一项所述的数据处理方法。
  19. 一种计算机程序产品，其特征在于，所述计算机程序产品包括计算机程序，所述计算机程序存储在计算机可读存储介质中，所述计算机程序适于由处理器读取并执行，以使得具有所述处理器的计算机设备执行权利要求1至15任一项所述的数据处理方法。
PCT/CN2023/128556 2023-06-19 2023-10-31 一种数据处理方法、装置、计算机设备及存储介质 Ceased WO2024259853A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US19/292,293 US20250363103A1 (en) 2023-06-19 2025-08-06 Data processing method, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310731911.0 2023-06-19
CN202310731911.0A CN116976418A (zh) 2023-06-19 2023-06-19 一种数据处理方法、装置、计算机设备及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/292,293 Continuation US20250363103A1 (en) 2023-06-19 2025-08-06 Data processing method, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2024259853A1 true WO2024259853A1 (zh) 2024-12-26

Family

ID=88477506

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/128556 Ceased WO2024259853A1 (zh) 2023-06-19 2023-10-31 一种数据处理方法、装置、计算机设备及存储介质

Country Status (3)

Country Link
US (1) US20250363103A1 (zh)
CN (1) CN116976418A (zh)
WO (1) WO2024259853A1 (zh)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113610232A (zh) * 2021-09-28 2021-11-05 苏州浪潮智能科技有限公司 网络模型量化方法、装置、计算机设备以及存储介质
CN113780551A (zh) * 2021-09-03 2021-12-10 北京市商汤科技开发有限公司 模型量化方法、装置、设备、存储介质及计算机程序产品
CN115983365A (zh) * 2023-01-16 2023-04-18 Oppo广东移动通信有限公司 模型训练方法、装置、计算机设备及存储介质

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113780551A (zh) * 2021-09-03 2021-12-10 北京市商汤科技开发有限公司 模型量化方法、装置、设备、存储介质及计算机程序产品
CN113610232A (zh) * 2021-09-28 2021-11-05 苏州浪潮智能科技有限公司 网络模型量化方法、装置、计算机设备以及存储介质
CN115983365A (zh) * 2023-01-16 2023-04-18 Oppo广东移动通信有限公司 模型训练方法、装置、计算机设备及存储介质

Also Published As

Publication number Publication date
US20250363103A1 (en) 2025-11-27
CN116976418A (zh) 2023-10-31

Similar Documents

Publication Publication Date Title
WO2020228376A1 (zh) 文本处理方法、模型训练方法和装置
CN117764373A (zh) 风险预测的方法、装置、设备及存储介质
CN109783624A (zh) 基于知识库的答案生成方法、装置和智能会话系统
CN109255369A (zh) 利用神经网络识别图片的方法及装置、介质和计算设备
CN114580443B (zh) 文本翻译方法、装置、核函数组合方法、服务器和介质
CN113782022B (zh) 基于意图识别模型的通信方法、装置、设备及存储介质
WO2024174583A9 (zh) 一种模型训练方法、装置、设备、存储介质及产品
CN121170319A (zh) 数据处理方法、装置、设备及可读存储介质
CN116152938A (zh) 身份识别模型训练和电子资源转移方法、装置及设备
WO2024179575A1 (zh) 一种数据处理方法、设备以及计算机可读存储介质
CN113362852A (zh) 一种用户属性识别方法和装置
CN120125972A (zh) 基于多模态大模型的电力图文交互方法、系统及相关设备
US20250285631A1 (en) Audio processing method, electronic device, and storage medium
CN113192201B (zh) 点云数据的数据拟合方法、装置及介质
CN119046442B (zh) 长文档视觉问答方法、装置及电子设备
CN119048121B (zh) 一种电网违规行为监督方法、终端设备及存储介质
WO2026066297A1 (zh) 基于意图识别的数据处理方法、装置、计算机设备和存储介质
WO2024259853A1 (zh) 一种数据处理方法、装置、计算机设备及存储介质
CN114756425B (zh) 智能监控方法、装置、电子设备及计算机可读存储介质
CN117032517A (zh) 基于电力工程项目管理的信息检索方法、设备及存储介质
CN113836915B (zh) 数据处理方法、装置、设备及可读存储介质
KR102873605B1 (ko) 인공지능 에이전트를 이용한 법률사건 시뮬레이션 방법
CN119941584B (zh) 盲图像修复方法及系统
CN118468957A (zh) 多模态敏感信息识别模型的训练方法、装置和电子设备
HK40053948A (zh) 数据处理方法、装置、设备以及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23942120

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE