EP4497086A1

EP4497086A1 - Systems and methods for machine learning model compression

Info

Publication number: EP4497086A1
Application number: EP23822796.1A
Authority: EP
Inventors: Jun Yin; Jianqiang HAN; Boyang CHEN; Li Wu; Xiangming ZHOU
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2022-06-15
Filing date: 2023-04-24
Publication date: 2025-01-29
Also published as: EP4497086A4; CN115829056B; CN115829056A; WO2023241225A1; CN114757353A

Abstract

The present disclosure relates to systems and methods for machine learning model compression. The method may include obtaining a plurality of weights of the machine learning model. The method may also include determining a target quantization interval set of the plurality of weights. The target quantization interval set may include a plurality of target quantization intervals. The method may further include determining a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively. The plurality of target shared weights may be used to compress the machine learning model.

Description

SYSTEMS AND METHODS FOR MACHINE LEARNING MODEL COMPRESSION

CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims priority to Chinese Patent Application No. 202210675517.5, filed on June 15, 2022, the contents of which are hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure generally relates to machine learning, and in particular, to systems and methods for machine learning model compression.

BACKGROUND

With the development of information technologies and communication technologies, the application of machine learning models becomes more and more popular. However, due to relatively large storage space, some machine learning models are not suitable or even unable to be used for devices (e.g., embedded devices) with relatively small storage capacity. For example, a deep learning model includes convolutional layers and fully-connected layers, and these layers include a large amount of weight data. If the deep learning model is deployed on an embedded device (e.g. a camera) with a relatively small storage capacity, the weight data may occupy a large amount of storage space, thereby affecting the normal use of the embedded device. Therefore, it is desirable to provide systems and methods for machine learning model compression.
SUMMARY
According to an aspect of the present disclosure, a method for compressing a machine learning model is provided. The method may include obtaining a plurality of weights of the machine learning model. The method may also include determining a target quantization interval set of the plurality of weights. The target quantization interval set may include a plurality of target quantization intervals. The method may further include determining a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively. The plurality of target shared weights may be used to compress the machine learning model.
In some embodiments, the determining a target quantization interval set of the plurality of weights may include determining a plurality of initial division intervals of the plurality of weights, determining a distribution of the plurality of weights in the plurality of initial division intervals, and determining the plurality of target quantization intervals in the target quantization interval set by adjusting the plurality of initial division intervals based on the distribution. A difference of distribution values of weights among at least two of the plurality of target quantization intervals may be less than a preset difference threshold.
In some embodiments, the determining a distribution of the plurality of weights in the plurality of initial division intervals may include for each of the plurality of initial division intervals, determining an initial count of weights in the initial division interval, and determining a distribution value of weights corresponding to the initial division interval by performing a statistical operation on the initial count of weights.
In some embodiments, the determining a distribution value of weights corresponding to the initial division interval by performing a statistical operation on the initial count of weights may include determining an adjusted value of the initial count of weights by performing an adjustment operation on the initial count of weights, and determining the distribution value of weights corresponding to the initial division interval by performing a normalization operation on the adjusted value.
In some embodiments, the determining an adjusted value of the initial count of weights by performing an adjustment operation on the initial count of weights may include at least one of determining the adjusted value by performing a numerical limit operation on the initial count of weights, determining the adjusted value by performing a transformation operation on the initial count of weights, determining a first intermediate value by performing the numerical limit operation on the initial count of weights, and determining the adjusted value by performing the transformation operation on the first intermediate value, or determining a second intermediate value by performing the transformation operation on the initial count of weights, and determining the adjusted value by performing the numerical limit operation on the second intermediate value.
In some embodiments, the determining the adjusted value by performing a numerical limit operation on the initial count of weights may include in response to the determining that the initial count of weights is equal to zero, determining the adjusted value as zero. The determining the adjusted value by performing a numerical limit operation on the initial count of weights may also include in response to the determining that the initial count of weights is greater than 0 and less than a minimum value of a preset numerical limit range, determining the adjusted value as the minimum value. The determining the adjusted value by performing a numerical limit operation on the initial count of weights may also include in response to the determining that the initial count of weights is within the preset numerical limit range, determining the adjusted value as the initial count of weights. The determining the adjusted value by performing a numerical limit operation on the initial count of weights may further include in response to the determining that the initial count of weights is greater than a maximum value of the preset numerical limit range, determining the adjusted value as the maximum value.
In some embodiments, the determining the adjusted value by performing a transformation operation on the initial count of weights may include performing the transformation operation on the initial count of weights based on a preset transformation function. The preset transformation function may include at least one of a constant function, an identity function, or a square root function.
In some embodiments, the determining the plurality of target quantization intervals in the target quantization interval set by adjusting the plurality of initial division intervals based on the distribution may include determining an accumulation array by accumulating the plurality of distribution values. The accumulation array may include a plurality of accumulated values. The determining the plurality of target quantization intervals in the target quantization interval set by adjusting the plurality of initial division intervals based on the distribution may also include determining the plurality of target quantization intervals based on the plurality of accumulated values, the plurality of weights, and a count of the plurality of initial division intervals.
In some embodiments, the determining the plurality of target quantization intervals based on the plurality of accumulated values, the plurality of weights, and a count of the plurality of initial division intervals may include obtaining a target count of the plurality of target quantization intervals, selecting candidate accumulated values satisfying a preset segmentation condition from the accumulation array based on the target count, determining a plurality of interval segmentation points based on the candidate accumulated values, a maximum weight of the plurality of weights, and a minimum weight of the plurality of weights, and determining the plurality of target quantization intervals based on the plurality of interval segmentation points.
In some embodiments, the determining a target quantization interval set of the plurality of weights may include obtain a target count of the plurality of target quantization intervals in the target quantization interval set, and obtaining a plurality of candidate division interval sets by performing a plurality of division operations on the plurality of weights based on the target count. Each of the plurality of candidate division interval sets may include a plurality of interval position parameters. The determining a target quantization interval set of the plurality of weights may also include for each of the plurality of candidate division interval sets, performing at least one iteration on the candidate division interval set based on an incremental set corresponding to the candidate division interval set. The incremental set may include a plurality of incremental parameters. The plurality of incremental parameters may correspond to the plurality of interval position parameters respectively, and each of the plurality of incremental parameters may indicate an updating magnitude of a corresponding interval position parameter. The determining a target quantization interval set of the plurality of weights may further include in response to determining that a preset condition is satisfied, determining the target quantization interval set based on a plurality of iteratively updated candidate division interval sets.
In some embodiments, the performing at least one iteration on the candidate division interval set based on an incremental set corresponding to the candidate division interval set may include determining an evaluation value of the candidate division interval set, updating the incremental set corresponding to the candidate division interval set based on the evaluation value, adjusting the plurality of interval position parameters in the candidate division interval set respectively based on updated incremental parameters in the updated incremental set, and determining the updated incremental set as an incremental set of a next iteration.
In some embodiments, the updating the incremental set corresponding to the candidate division interval set based on the evaluation value may include determining a first difference between the candidate division interval set and an individual optimal solution of the candidate division interval set, determining a second difference between the candidate division interval set and a group optimal solution of the plurality of candidate division interval sets, and updating the incremental set by adjusting the plurality of incremental parameters based on the first difference and the second difference.
In some embodiments, the determining a target quantization interval set of the plurality of weights may include obtaining a target count of the plurality of target quantization intervals in the target quantization interval set, obtaining a plurality of candidate division interval sets by performing a plurality of division operations on the plurality of weights based on the target count, and determining the target quantization interval set by performing at least one iteration on the plurality of candidate division interval sets based on a plurality of evaluation values corresponding to the plurality of candidate division interval sets respectively.
In some embodiments, the determining the target quantization interval set by performing at least one iteration on the plurality of candidate division interval sets based on a plurality of evaluation values corresponding to the plurality of candidate division interval sets respectively may include determining a plurality of first candidate interval sets from the plurality of candidate division interval sets based on the plurality of evaluation values corresponding to the plurality of candidate division interval sets respectively, determining a plurality of second candidate interval sets by performing a first transformation operation and/or a second transformation operation on the plurality of first candidate interval sets, determining a plurality of third candidate interval sets from the plurality of first candidate interval sets and/or the plurality of second candidate interval sets based on evaluation values of the plurality of first candidate interval sets and/or evaluation values of the plurality of second candidate interval sets, and determining the plurality of third candidate interval sets as candidate division interval sets of a next iteration.
In some embodiments, the determining a target quantization interval set of the plurality of weights may include determining a plurality of candidate quantization interval sets according to a plurality of preset division manners. Each of the plurality of candidate quantization interval sets may include a plurality of candidate quantization intervals. The determining a target quantization interval set of the plurality of weights may also include for the each of the plurality of candidate quantization interval sets, obtaining a plurality of candidate shared weights corresponding to the plurality of candidate quantization intervals respectively in the candidate quantization interval set, obtaining test data of the plurality of candidate shared weights, and selecting the target quantization interval set from the plurality of candidate quantization interval sets based on the test data.
In some embodiments, the determining a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively may include for each of the plurality of target quantization intervals, determining an average value of a maximum weight and a minimum weight in the target quantization interval as a target shared weight corresponding to the target quantization interval, determining an average value of weights in the target quantization interval as a target shared weight corresponding to the target quantization interval, or determining a median value of weights in the target quantization interval as a target shared weight corresponding to the target quantization interval.
In some embodiments, the method further may include obtaining test data of the plurality of target shared weights, and determining whether the test data satisfies a preset condition. The method may also include in response to the determining that the test data does not satisfy the preset condition, adjusting the plurality of target quantization intervals. The method may further include updating the plurality of target shared weights based on the plurality of adjusted target quantization intervals.
In some embodiments, the method further may include determining a target model by compressing the machine learning model based on the plurality of target shared weights, and transmitting the target model to a storage device of an acquisition device and storing the target model in the storage device. The target model may be used to achieve an inference operation on data collected by the acquisition device.
According to another aspect of the present disclosure, a system for compressing a machine learning model is provided. The system may include a weight obtaining module configured to obtain a plurality of weights of the machine learning model. The system may also include a quantization interval determination module configured to determine at least one quantization interval set based on the plurality of weights through at least one preset division manner. Each of the at least one quantization interval set may include a plurality of quantization intervals. The system may further include a shared weight determination module configured to determine a shared weight of each of the plurality of quantization intervals. The plurality of shared weights may be used to compress the machine learning model.
According to another aspect of the present disclosure, a model transformation tool is provided. The model transformation tool may include at least one storage device including a set of instructions, and at least one processor configured to communicate with the at least one storage device. When executing the set of instructions, the at least one processor may be configured to direct the system to perform operations including obtaining a machine learning model and performing a quantitative operation on the machine learning model. The operations may also include in response to determining that a result of the quantitative operation does not satisfy a preset condition, triggering a model compression process. The model compression process may include obtaining a plurality of weights of the machine learning model, and determining a target quantization interval set of the plurality of weights. The target quantization interval set may include a plurality of target quantization intervals. The operations may further include determining a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively. The plurality of target shared weights may be used to compress the machine learning model.
According to yet another aspect of the present disclosure, a non-transitory computer readable medium including executable instructions is provided. The instructions, when executed by at least one processor, may cause the at least one processor to effectuate a method comprising obtaining a plurality of weights of the machine learning model, and determining a target quantization interval set of the plurality of weights. The target quantization interval set may include a plurality of target quantization intervals. The method may also include determining a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively. The plurality of target shared weights may be used to compress the machine learning model.
Additional features will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the accompanying drawings or may be learned by production or operation of the examples. The features of the present disclosure may be realized and attained by practice or use of various aspects of the methodologies, instrumentalities, and combinations set forth in the detailed examples discussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is further described in terms of exemplary embodiments. These exemplary embodiments are described in detail with reference to the drawings. These embodiments are non-limiting exemplary embodiments, in which like reference numerals represent similar structures throughout the several views of the drawings, and wherein:
FIG. 1 is a schematic diagram illustrating an exemplary model processing system according to some embodiments of the present disclosure;
FIG. 2 is a schematic diagram illustrating exemplary hardware and/or software components of an exemplary computing device according to some embodiments of the present disclosure;
FIG. 3 is a block diagram illustrating an exemplary processing device according to some embodiments of the present disclosure;
FIG. 4 is a flowchart illustrating an exemplary process for determining target shared weights used for compressing a machine learning model according to some embodiments of the present disclosure;
FIG. 5 is a flowchart illustrating an exemplary process for determining a plurality of target quantization intervals by adjusting a plurality of initial division intervals according to some embodiments of the present disclosure;
FIG. 6A is a flowchart illustrating an exemplary process for determining a distribution value of weights corresponding to an initial division interval according to some embodiments of the present disclosure;
FIG. 6B is a flowchart illustrating an exemplary process for determining a plurality of target quantization intervals based on distribution values corresponding to a plurality of initial division intervals according to some embodiments of the present

disclosure;

FIG. 7 is a flowchart illustrating an exemplary process for determining a target quantization interval set based on a plurality of candidate division interval sets according to some embodiments of the present disclosure;
FIG. 8 is a flowchart illustrating an exemplary process for determining a target quantization interval set based on a plurality of candidate division interval sets according to some embodiments of the present disclosure; and
FIG. 9 is a flowchart illustrating an exemplary process for determining a target quantization interval set based on a plurality of candidate quantization interval sets according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant disclosure. However, it should be apparent to those skilled in the art that the present disclosure may be practiced without such details. In other instances, well- known methods, procedures, systems, components, and/or circuitry have been described at a relatively high level, without detail, in order to avoid unnecessarily obscuring aspects of the present disclosure. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments shown, but to be accorded the widest scope consistent with the claims.
The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms “a, ” “an, ” and “the” may be intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprise, ” “comprises, ” and/or “comprising, ” “include, ” “includes, ” and/or “including, ” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It will be understood that the terms “system, ” “engine, ” “unit, ” “module, ” and/or “block” used herein are one method to distinguish different components, elements, parts, sections, or assemblies of different levels in ascending order. However, the terms may be displaced by other expressions if they may achieve the same purpose.
It will be understood that when a unit, engine, module or block is referred to as being “on, ” “connected to, ” or “coupled to, ” another unit, engine, module, or block, it may be directly on, connected or coupled to, or communicate with the other unit, engine, module, or block, or an intervening unit, engine, module, or block may be present, unless the context clearly indicates otherwise. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.
These and other features, and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structure and the combination of parts and economies of manufacture, may become more apparent upon consideration of the following description with reference to the accompanying drawings, all of which form a part of this disclosure. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended to limit the scope of the present disclosure. It is understood that the drawings are not to scale.
The present disclosure provides systems and methods for machine learning model compression. The system may determine a target quantization interval set of a plurality of weights of a machine learning model. The target quantization interval set may include a plurality of target quantization intervals. The system may also determine a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively, thereby compressing the machine learning model based on the plurality of target shared weights. In some embodiments, the system may determine a plurality of candidate division interval sets of the plurality of weights, and perform at least one iteration on each of the plurality of candidate division interval sets based on an incremental set corresponding to each of the plurality of candidate division interval sets to determine the target quantization interval set including the plurality of target quantization intervals. In some embodiments, the system may determine a plurality of candidate division interval sets of the plurality of weights, and determine the target quantization interval set including the plurality of target quantization intervals based on evaluation values corresponding to the plurality of candidate division interval sets respectively.
In some embodiments, the system may also determine a plurality of candidate quantization interval sets based on a plurality of preset division manners. For each of the plurality of candidate quantization interval sets, the system may obtain a plurality of candidate shared weights corresponding to the plurality of candidate quantization intervals respectively in the candidate quantization interval set, and obtain test data of the plurality of candidate shared weights. The system may further select a candidate quantization interval set with optimal test data from the plurality of candidate quantization interval sets based on the test data, and determine the candidate quantization interval set with the optimal test data as the target quantization interval set.
According to the embodiments of the present disclosure, the plurality of target quantization intervals and the corresponding target shared weights can be determined efficiently and accurately. Accordingly, a machine learning model can be efficiently compressed and a storage space occupied by the machine learning model can be efficiently reduced. In addition, a division manner with the best compression effect is selected based on test data and the target quantization interval set (which includes the plurality of target quantization intervals) is determined based on the division manner with the best compression effect. Accordingly, a good compression effect can be ensured.
FIG. 1 is a schematic diagram illustrating an exemplary model processing system according to some embodiments of the present disclosure. In some embodiments, as illustrated in FIG. 1, the model processing system 100 may include a processing device 110, a terminal device 120, and a network 130. In some embodiments, two or more components of the model processing system 100 may be connected to and/or communicate with each other via a wireless connection, a wired connection, or a combination thereof. The connection between the components of the model processing system 100 may be various. Merely by way of example, the processing device 110 may be connected to the terminal device 120 through the network 130 or directly as indicated by a dotted arrow linking the processing device 110 and the terminal device 120.
. The processing device 110 may process data of a machine learning model and/or data related to the machine learning model. In some embodiments, the processing device 110 may compress a trained machine learning model. For example, the processing device 110 may obtain a plurality of weights of the machine learning model; determine a target quantization interval set (which includes a plurality of target quantization intervals) of the plurality of weights; and determine a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively. Further, the processing device 110 may compress the machine learning model based on the plurality of target shared weights to determine a target model (i.e., a compressed model) . In some embodiments, the processing device 110 may transmit the target model to a storage device of the terminal device 120. In some embodiments, the target model may be used to perform an inference processing based on data (e.g., characteristic data of a predicted subject) obtained by the terminal device 120.
In some embodiments, the processing device 110 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processing device 110 may be local or remote. In some embodiments, the processing device 110 may be implemented on a cloud platform. Merely by way of example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, or the like, or any combination thereof. In some embodiments, the processing device 110 may be implemented by a computing device 200 having one or more components illustrated in FIG. 2.
The terminal device 120 may receive a (compressed) machine learning model and/or perform related operations based on the (compressed) machine learning model. In some embodiments, the terminal device 120 may store the compressed machine learning model and perform a prediction task (also referred to as a “model inference” ) based on the compressed machine learning model. In some embodiments, during the model inference, the terminal device 120 may read mapping values (e.g., 8 bit numbers) of weights or indexes stored in the terminal device 120, and restore compressed weights to original weights or approximate values of the original weights, which can be collectively referred to as “reduction values. ” Further, the terminal device 120 may perform the model inference based on the reduction values of weights.
In some embodiments, the terminal device 120 may include an embedded device with a relatively small storage capacity. In some embodiments, the terminal device 120 may include a smart phone 121, a smart camera 122, a smart audio 123, a smart TV, a smart fridge, a robot, a tablet, a laptop, a wearable, a payment device, a cashier device, or the like, or any combination thereof.
The network 130 may include any suitable network that can facilitate the exchange of information and/or data for the model processing system 100. In some embodiments, one or more components (e.g., the processing device 110, the terminal device 120) of the model processing system 100 may communicate information and/or data with one or more other components of the model processing system 100 via the network 130. For example, the processing device 110 may transmit the compressed machine learning model to the terminal device 120 via the network 130.
In some embodiments, the network 130 may be or include a public network (e.g., the Internet) , a private network (e.g., a local area network (LAN) ) , a wired network, a wireless network (e.g., an 802.11 network, a Wi-Fi network) , a frame relay network, a virtual private network (VPN) , a satellite network, a telephone network, routers, hubs, switches, server computers, and/or any combination thereof. For example, the network 130 may include a cable network, a wireline network, a fiber- optic network, a telecommunications network, an intranet, a wireless local area network (WLAN) , a metropolitan area network (MAN) , a public telephone switched network (PSTN) , a Bluetooth^TM network, a ZigBee^TM network, a near field communication (NFC) network, or the like, or any combination thereof. In some embodiments, the network 130 may include one or more network access points. For example, the network 130 may include wired and/or wireless network access points such as base stations and/or internet exchange points through which one or more components of the model processing system 100 may be connected to the network 130 to exchange data and/or information.
In some embodiments, the model processing system 100 may also include a storage device (not shown) . The storage device may store data, instructions, and/or any other information. In some embodiments, the storage device may store data obtained from the processing device 110 and/or the terminal device 120. In some embodiments, the storage device may store data and/or instructions that the processing device 110 may execute or use to perform exemplary methods described in the present disclosure. In some embodiments, the storage device may include a mass storage, removable storage, a volatile read-and-write memory, a read-only memory (ROM) , or the like, or any combination thereof. In some embodiments, the storage device may be implemented on a cloud platform. In some embodiments, the storage device may integrated into the processing device 110 and/or the terminal device 120.
It should be noted that the above description regarding the model processing system 100 is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations and modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure. In some embodiments, the model processing system 100 may include one or more additional components and/or one or more components of the model processing system 100 described above may be omitted. Additionally or alternatively, two or more components of the model processing system 100 may be integrated into a single component. A component of the model processing system 100 may be implemented on two or more sub-components.
FIG. 2 is a schematic diagram illustrating exemplary hardware and/or software components of an exemplary computing device according to some embodiments of the present disclosure. In some embodiments, the processing device 110 and/or the terminal device 120 may be implemented on the computing device 200. In some embodiments, the computing device 200 may include a model transformation tool. The model transformation tool may receive a machine learning model to be processed, and output the processed (e.g., compression) machine learning model (e.g., the processed machine learning model may be output to the terminal device 120) . As illustrated in FIG. 2, the computing device 200 may include a processor 210, a storage 220, an input/output (I/O) 230, and a communication port 240.
The processor 210 may execute computer instructions (e.g., program code) and perform functions of the processing device 110 in accordance with techniques describable herein. The computer instructions may include, for example, routines, programs, objects, components, data structures, procedures, modules, and functions, which perform particular functions describable herein.
In some embodiments, the processor 210 may include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC) , an application specific integrated circuits (ASICs) , an application-specific instruction-set processor (ASIP) , a central processing unit (CPU) , a graphics processing unit (GPU) , a physics processing unit (PPU) , a microcontroller unit, a digital signal processor (DSP) , a field programmable gate array (FPGA) , an advanced RISC machine (ARM) , a programmable logic device (PLD) , any circuit or processor capable of executing one or more functions, or the like, or any combinations thereof.
Merely for illustration, only one processor is describable in the computing device 200. However, it should be noted that the computing device 200 in the present disclosure may also include multiple processors, thus operations and/or method operations that are performed by one processor as describable in the present disclosure may also be jointly or separately performed by the multiple processors. For example, if in the present disclosure the processor of the computing device 200 executes both operation A and operation B, it should be understood that operation A and operation B may also be performed by two or more different processors jointly or separately in the computing device 200 (e.g., a first processor executes operation A and a second processor executes operation B, or the first and second processors jointly execute operations A and B) .
The storage 220 may store data obtained from one or more components of the model processing system 100. In some embodiments, the storage 220 may include a mass storage device, a removable storage device, a volatile read-and-write memory, a read-only memory (ROM) , or the like, or any combination thereof. In some embodiments, the storage 220 may store one or more programs and/or instructions to perform exemplary methods describable in the present disclosure. For example, the storage 220 may store a program for the processing device 110 to execute to compress the machine learning model.
The I/O 230 may input and/or output signals, data, information, etc. In some embodiments, the I/O 230 may enable a user interaction with the processing device 110. In some embodiments, the I/O 230 may include an input device and an output device. The input device may include a keyboard, a touch screen, a speech input, an eye tracking input, a brain monitoring system, or any other comparable input mechanism. The input information received through the input device may be transmitted to another component (e.g., the processing device 110) via, for example, a bus, for further processing. Other types of the input devices may include a cursor control device, such as a mouse, a trackball, or cursor direction keys, etc. The output device may include a display (e.g., a liquid crystal display (LCD) , a light-emitting diode (LED) -based display, a flat panel display, a curved screen, a television device, a cathode ray tube (CRT) , a touch screen) , a speaker, a printer, or the like, or a combination thereof.
The communication port 240 may be connected to a network (e.g., the network 130) to facilitate data communications. The communication port 240 may establish connections between the processing device 110 and the terminal device 120. The connection may be a wired connection, a wireless connection, any other communication connection that can enable data transmission and/or reception, and/or any combination of these connections. The wired connection may include, for example, an electrical cable, an optical cable, a telephone wire, or the like, or any combination thereof. The wireless connection may include, for example, a Bluetooth^TM link, a Wi-Fi^TM link, a WiMax^TM link, a WLAN link, a ZigBee^TM link, a mobile network link (e.g., 3G, 4G, 5G) , or the like, or a combination thereof. In some embodiments, the communication port 240 may be and/or include a standardized communication port, such as RS232, RS485, etc. In some embodiments, the communication port 240 may be a specially designed communication port.
FIG. 3 is a block diagram illustrating an exemplary processing device according to some embodiments of the present disclosure. As illustrated in FIG. 3, the processing device 110 may include a weight obtaining module 310, a quantization interval determination 320, and a shared weight determination module 330.
The weight obtaining module 310 may be used to obtain a plurality of weights of the machine learning model.
The quantization interval determination 320 may be used to determine a target quantization interval set of the plurality of weights. The target quantization interval set may include a plurality of target quantization intervals.
The shared weight determination module 330 may be used to determine a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively. The plurality of target shared weights may be used to compress the machine learning model.
More detailed descriptions of the modules of the processing device 110 may be found in elsewhere of the present disclosure, for example, FIGs. 4-9 and the descriptions thereof.
It should be noted that the above descriptions of the processing device 110 are provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, various modifications and changes in the forms and details of the application of the above method and system may occur without departing from the principles of the present disclosure. In some embodiments, the processing device 110 may include one or more other modules and/or one or more modules described above may be omitted. Additionally or alternatively, two or more modules may be integrated into a single module and/or a module may be divided into two or more units. However, those variations and modifications also fall within the scope of the present disclosure.
FIG. 4 is a flowchart illustrating an exemplary process for determining target shared weights used for compressing a machine learning model according to some embodiments of the present disclosure. In some embodiments, process 400 may be executed by the model processing system 100. For example, the process 400 may be implemented as a set of instructions stored in a storage device. In some embodiments, the processing device 110 (e.g., the processor 210 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 400. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 400 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 400 illustrated in FIG. 4 and described below is not intended to be limiting.
In 410, the processing device 110 (e.g., the weight obtaining module 310) may obtain a plurality of weights of the machine learning model.
The machine learning model may be any type of machine learning model. For example, the machine learning model may include a deep learning model, a supervised learning model, an unsupervised learning model, etc. In some embodiments, the machine learning model may be in a form of a multi-layer structure (e.g., a neural network structure) , or in a form of a single-layer structure (e.g., a single-layer neural network structure) . Each layer may include one or more calculation units (e.g., a neuron) , and each calculation unit may have a corresponding weight. In some embodiments, the weight may be in a range of 0 to 1. In some embodiments, after the training of the machine learning model is finished, the weigh may be stored in a form of floating-point number. For example, the weight may be stored in a form of 32 bit floating-point number.
In some embodiments, the processing device 110 may obtain part of weights of the machine learning model, for example, weights of one or more specific layers of the machine learning model.
In some embodiments, the processing device 110 may obtain all weights of the machine learning model.
In 420, the processing device 110 (e.g., the quantization interval determination module 320) may determine a target quantization interval set of the plurality of weights. The target quantization interval set may include a plurality of target quantization intervals.
The target quantization intervals may refer to intervals formed by dividing the plurality of weights to satisfy a weight quantization requirement.
In some embodiments, the processing device 110 may determine a plurality of initial division intervals of the plurality of weights, and determine the plurality of target quantization intervals based on a distribution of the plurality of weights in the plurality of initial division intervals.
In some embodiments, the processing device 110 may determine a plurality of candidate division interval sets of the plurality of weights, and perform at least one iteration on each of the plurality of candidate division interval sets based on an incremental set corresponding to each of the plurality of candidate division interval sets to determine the target quantization interval set including the plurality of target quantization intervals.
In some embodiments, the processing device 110 may determine a plurality of candidate division interval sets of the plurality of weights, and determine the target quantization interval set including the plurality of target quantization intervals based on evaluation values corresponding to the plurality of candidate division interval sets respectively.
In some embodiments, the processing device 110 may also determine a plurality of candidate quantization interval sets based on a plurality of preset division manners. For each of the plurality of candidate quantization interval sets, the processing device 110 may obtain a plurality of candidate shared weights corresponding to the plurality of candidate quantization intervals respectively in the candidate quantization interval set, and obtain test data of the plurality of candidate shared weights. The processing device 110 may further select a candidate quantization interval set with optimal test data from the plurality of candidate quantization interval sets based on the test data, and determine the candidate quantization interval set with the optimal test data as the target quantization interval set.
More details regarding the determination of the target quantization interval set and/or the target quantization intervals thereof may be found elsewhere in the present disclosure, for example, FIGs. 5-9 and related descriptions thereof.
In 430, the processing device 110 (e.g., the shared weight determination module 330) may determine a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively. The plurality of target shared weights may be used to compress the machine learning model.
In some embodiments, for each of the plurality of target quantization intervals, the shared weight determination module 330 may determine an average value of a maximum weight and a minimum weight in the target quantization interval as a target shared weight corresponding to the target quantization interval.
In some embodiments, for each of the plurality of target quantization intervals, the shared weight determination module 330 may determine an average value of weights in the target quantization interval as a target shared weight corresponding to the target quantization interval.
In some embodiments, for each of the plurality of target quantization intervals, the shared weight determination module 330 may determine a median value of weights in the target quantization interval as a target shared weight corresponding to the target quantization interval.
In some embodiments, the determination manner of the target shared weights may be related to the determination manner of the target quantization intervals. For different determination manners of the target quantization intervals, the determination manners of the target shared weights may be accordingly different. More descriptions regarding the determination of the target shared weights may be found elsewhere in the present disclosure, for example, FIG. 6A and related descriptions thereof.
In some embodiments, the quantization interval determination module 320 may obtain test data of the plurality of target shared weights. Specifically, the quantization interval determination module 320 may compress the machine learning model based on the plurality of target shared weights and test a prediction performance of the compressed machine learning model based on a test set to obtain the test data of the plurality of target shared weights. The test data may be quantified by a loss function of the machine learning model. After obtaining the test data, the quantization interval determination module 320 may determine whether the test data satisfies a preset condition. For example, the quantization interval determination module 320 may determine whether a value of the loss function is less than a preset threshold. In response to the determining that the test data does not satisfy the preset condition, the quantization interval determination module 320 may redefine the plurality of target quantization intervals. Further, the quantization interval determination module 320 may determine the plurality of target shared weights based on the plurality of redefined target quantization intervals.
In some embodiments, the quantization interval determination module 320 may redefine the plurality of target quantization intervals by adjusting configuration parameters. For example, as described in connection with FIG. 5, the quantization interval determination module 320 may adjust a count (M) of initial division intervals, a minimum value (thresh_min) and/or a maximum value (thresh_max) in a preset numerical limit range, a preset difference threshold, a count (K) of the target quantization intervals, or the like, or a combination thereof. As another example, as described in connection with FIGs. 7-8, the quantization interval determination module 320 may a maximum number count of iterations, a preset threshold for evaluation values, a ranking threshold for evaluation values, one or more preset coefficients (e.g., α, β₁, β₂) , or the like, or a combination thereof.
In some embodiments, the quantization interval determination module 320 may redefine the target quantization intervals by adjusting the processing flow. For example, as described in connection with FIG. 5, the quantization interval determination module 320 may add, reduce, and/or adjust one or more processing operations (e.g., a numerical limit operation, a transformation operation) . As another example, the quantization interval determination module 320 may adjust an execution order of the processing operations (e.g., exchange the execution order of the numerical limit operation and the transformation operation) . As a further example, as described in connection with FIGs. 7-8, the quantization interval determination module 320 may adjust an iteration termination condition. Specifically, the iteration termination condition may be adjusted from reaching the preset maximum number count of iterations to the evaluation value reaching the preset threshold, or the iteration termination condition may be adjusted from the evaluation value reaching the preset threshold to a convergence of the evaluation value.
In some embodiments, the quantization interval determination module 320 may redefine the target quantization intervals based on another determination manner of the target quantization intervals. For example, if the target quantization intervals determined based on process 500 do not satisfy the preset condition, the quantization interval determination module 320 may redefine the plurality of target quantization intervals based on process 700 or process 900. More detailed descriptions may be found later.
It should be noted that the above description of the process 400 is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations or modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure. For example, one or more other optional operations (e.g., a storing operation, a transmitting operation) may be added elsewhere in the process 400. In the storing operation, the processing device 110 may store information and/or data (e.g., the target shared weights) associated with machine learning model compression in a storage device disclosed elsewhere in the present disclosure. In the transmitting operation, the processing device 110 may transmit the information and/or data to the terminal device 120.
FIG. 5 is a flowchart illustrating an exemplary process for determining a plurality of target quantization intervals by adjusting a plurality of initial division intervals according to some embodiments of the present disclosure. In some embodiments, process 500 may be executed by the model processing system 100. For example, the process 500 may be implemented as a set of instructions stored in a storage device. In some embodiments, the processing device 110 (e.g., the processor 210 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 500. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 500 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 500 illustrated in FIG. 5 and described below is not intended to be limiting. In some embodiments, operation 420 may be performed based on the process 500.
In 510, the processing device 110 (e.g., the quantization interval determination module 320) may determine a plurality of initial division intervals of the plurality of weights.
For convenience, the plurality of weights may be expressed as w₁～w_N, wherein N refers to a total count of the weights. In some embodiments, the quantization interval determination module 320 may determine a minimum weight (can be referred to as w_min) and a maximum weight (can be referred to as w_max) of the plurality of weights, and determine the plurality of initial division intervals based on the minimum weight and the maximum weight. For example, the quantization interval determination module 320 may evenly divide an interval (e.g., [w_min, w_max] ) determined by the minimum weight and the maximum weight into a preset count (e.g., M which may be a system default value or set by a user, such as M=2¹²=4096) of sub-intervals, and determine the sub-intervals as the plurality of initial division intervals. As another example, the quantization interval determination module 320 may arbitrarily select a certain count (e.g., M-1) of weights as segmentation points in the interval (e.g., [w_min, w_max] ) determined by the minimum weight and the maximum weight, and divide the interval into the preset count (e.g., M) of sub-intervals based on the segmentation points.
In 520, the processing device 110 (e.g., the quantization interval determination module 320) may determine a distribution of the plurality of weights in the plurality of initial division intervals.
The distribution may indicate a count of weights and/or a density of weights in the plurality of initial division intervals. In some embodiments, the distribution may be expressed as a distribution value of weights.
In some embodiments, taking a specific initial division interval as an example, a distribution value of weights corresponding to the initial division interval may be expressed as a count of weights in the initial division interval. For example, it is assumed that a count of weights in a specific initial division interval [0.2, 0.3] is 6, the distribution value of the initial division interval may be expressed as 6.
In some embodiments, also taking a specific initial division interval as an example, the quantization interval determination module 320 may determine the distribution value (also referred to as an “corrected count” or a “calibrated count” ) of weights corresponding to the initial division interval by processing the count of weights in the initial division interval. For example, the quantization interval determination module 320 may determine an initial count of weights in the initial division interval, and perform a statistical operation on the initial count of weights to determine the distribution value of weights corresponding to the initial division interval.
In some embodiments, for each of the plurality of initial division intervals, the quantization interval determination module 320 may determine an adjusted value of the initial count of weights by performing an adjustment operation on the initial count of weights. The adjustment operation may refer to a specific processing operation (e.g., a numerical limit operation, a transformation operation) performed on the initial count of weights to reduce a weight quantization error. Further, the processing device 110 may perform a normalization operation on the adjusted value to determine the distribution value of weights corresponding to the initial division interval. More descriptions regarding the determination of the distribution value of weights corresponding to the initial division interval may be found elsewhere in the present disclosure, for example, FIG. 6A and descriptions thereof.
In 530, the processing device 110 (e.g., the quantization interval determination module 320) may determine the plurality of target quantization intervals in the target quantization interval set by adjusting the plurality of initial division intervals based on the distribution.
In some embodiments, after the plurality of initial division intervals are adjusted based on the distribution, a difference of distribution values of weights among at least two of the plurality of target quantization intervals may be less than a preset difference threshold. That is, the adjustment of the intervals aims to equalizing the distribution values of weights among the plurality of target quantization intervals. Merely by way of example, it is assumed that a preset count of target quantization intervals is K (e.g., K=2⁸=256) and a sum of the distribution values of the target quantization intervals is equal to 1, the adjustment of the intervals aims to causing the distribution value of weights corresponding to each target quantization interval to be close to or equal to 1/K.
In some embodiments, a target quantization interval may be formed by an initial division interval, or a target quantization interval may be formed by stitching a plurality of successive initial division intervals. For example, the quantization interval determination module 320 may determine an initial division interval (a, b] or (b, c] as the target quantization interval. As another example, the quantization interval determination module 320 may stitch the initial division intervals (a, b] and (b, c] to determine a target quantization interval (a, c] .
In some embodiments, a target quantization interval may include part of the initial division intervals. For example, the quantization interval determination module 320 may determine a target quantization interval (a, m] based on the initial division intervals (a, b] and (b, c] , wherein a<m<c.
In some embodiments, if a specific initial division interval does not include any one of the plurality of weights, that is, the initial count of weights in the initial division interval is zero, the quantization interval determination module 320 may ignore the initial division interval when determining a target quantization interval.
In some embodiments, as mentioned above, the distribution of the plurality of weights in the plurality of initial division intervals may be represented by a plurality of distribution values corresponding to the plurality of initial division intervals respectively. Accordingly, the quantization interval determination module 320 may determine an accumulation array by accumulating the plurality of distribution values, which includes a plurality of accumulated values. Further, the quantization interval determination module 320 may determine the plurality of target quantization intervals based on the plurality of accumulated values, the plurality of weights, and a count of the plurality of initial division intervals. More descriptions regarding the determination of the plurality of target quantization intervals may be found elsewhere in the present disclosure, for example, FIG. 6B and related descriptions thereof.
It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations or modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure.
FIG. 6A is a flowchart illustrating an exemplary process for determining a distribution value of weights corresponding to an initial division interval according to some embodiments of the present disclosure. In some embodiments, process 600A may be executed by the model processing system 100. For example, the process 600A may be implemented as a set of instructions stored in a storage device. In some embodiments, the processing device 110 (e.g., the processor 210 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 600A. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 600A may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 600A illustrated in FIG. 6A and described below is not intended to be limiting. In some embodiments, operation 520 may be performed based on the process 600A.
In 611, for each of the plurality of initial division intervals, the processing device 110 (e.g., the quantization interval determination module 320) may determine an initial count of weights in the initial division interval.
In 613, the processing device 110 (e.g., the quantization interval determination module 320) may determine an adjusted value of the initial count of weights by performing an adjustment operation on the initial count of weights.
As used herein, the adjustment operation may refer to a specific processing operation (e.g., a numerical limit operation, a transformation operation) performed on the initial count of weights to reduce a weight quantization error.
In some embodiments, the processing device 110 may determine the adjusted value of the initial count of weights by performing the numerical limit operation on the initial count of weights. Specifically, the quantization interval determination module 320 may limit the initial count of weights between a preset lower limit value and a preset upper limit value (also can be referred to as a “preset numerical limit range” ) . In some embodiments, the preset lower limit value may be a non-zero value.
In some embodiments, in response to determining that the initial count of weights is equal to zero, the adjusted value may be determined as zero. In response to the determining that the initial count of weights is greater than 0 and less than a minimum value (i.e., the lower limit value) of the preset numerical limit range, the adjusted value may be determined as the minimum value. In response to the determining that the initial count of weights is within the preset numerical limit range, the adjusted value may be determined as the initial count of weights. In response to the determining that the initial count of weights is greater than a maximum value (i.e., the preset upper limit value) of the preset numerical limit range, the adjusted value may be determined as the maximum value. Merely by way of example, the processing device 110 may perform the numerical limit operation according to formula (1) below:
where h_j refers to an initial count of weights in a jth initial division interval, h_j′ refers to an adjusted value of h_j, thresh_min refers to a minimum value of the preset numerical limit range, and thresh_max refers to a maximum value of the preset numerical limit range. In some embodiments, the minimum value thresh_min and the maximum value thresh_max may be obtained based on experiences or experiments. In some other embodiments, the minimum value thresh_min may be equal to 0, the maximum value thresh_max may be equal to (positive) infinity, and h_j′ may be constantly equal to h_j.
In some embodiments, the processing device 110 may determine the adjusted value by performing a transformation operation on the initial count of weights. Specifically, the quantization interval determination module 320 may perform the transformation operation on the initial count of weights based on a preset transformation function. In some embodiments, the preset transformation function may include at least one of a constant function (e.g., f (x) =c) , an identity function (e.g., f (x) =x) , a square root function (e.g., f (x) =sqrt (x) ) , or the like, or a combination thereof.
In some embodiments, the processing device 110 may determine the adjusted value by performing the numerical limit operation and the transformation operation on the initial count of weights. For example, the quantization interval determination module 320 may determine a first intermediate value by performing the numerical limit operation on the initial count of weights, and determine the adjusted value by performing the transformation operation on the first intermediate value. As another example, the quantization interval determination module 320 may determine a second intermediate value by performing the transformation operation on the initial count of weights, and determine the adjusted value by performing the numerical limit operation on the second intermediate value.
In 615, the processing device 110 (e.g., the quantization interval determination module 320) may perform a normalization operation on the adjusted value to determine the distribution value of weights corresponding to the initial division interval.
In some embodiments, the processing device 110 may perform the normalization operation according to formula (2) below:
whererefers to a sum of adjusted values corresponding to the plurality of initial division intervals respectively, M refers to a count of the plurality of initial division intervals, S_j refers to an adjusted value corresponding to the jth initial division interval, and D_j refers to a distribution value corresponding to the jth initial division interval.
In some embodiments, as described in connection with FIG. 4, after determining the plurality of target quantization intervals, the processing device 110 may determine a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively. The plurality of target shared weights may be used to compress the machine learning model. According to the embodiments of the present disclosure, through the adjustment operation and/or the normalization operation performed on the initial count of weights, the weight quantization error can be efficiently reduced. In some embodiments, the weight quantization error of the plurality of weights may be expressed as a L^p norm of a weight vector which includes a plurality of vector factors corresponding to the plurality of weights respectively. The smaller the L^p norm is, the smaller the weight quantization error may be.
In some embodiments, different adjustment manners may achieve different optimization goals. For example, through the limit of the non-zero lower limit value, the distribution value (s) corresponding to the initial division interval (s) would not be too small, which can avoid a single target shared weight corresponding to a too large target quantization interval, thereby reducing the L^∞ norm of the weight vector. As another example, through the identity function, it is conductive for equalizing the counts of weights in the target quantization intervals respectively, thereby reducing the L¹ norm of the weight vector. As a further example, through the square root function, it is conductive that a count of weights in each target quantization interval would be proportional to a distribution value corresponding to the target quantization interval, thereby reducing the L² norm of the weight vector.
In some embodiments, when the preset transformation function includes the constant function, for each of the plurality of target quantization intervals, the shared weight determination module 330 may determine an average value of a maximum weight and a minimum weight in the target quantization interval as a target shared weight corresponding to the target quantization interval.
In some embodiments, when the preset transformation function includes the identify function, for each of the plurality of target quantization intervals, the shared weight determination module 330 may determine an average value of weights in the target quantization interval as a target shared weight corresponding to the target quantization interval.
In some embodiments, when the preset transformation function includes the square root function, for each of the plurality of target quantization intervals, the shared weight determination module 330 may determine a median value of weights in the target quantization interval as a target shared weight corresponding to the target quantization interval.
It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations or modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure.
FIG. 6B is a flowchart illustrating an exemplary process for determining a plurality of target quantization intervals based on distribution values corresponding to a plurality of initial division intervals according to some embodiments of the present disclosure. In some embodiments, process 600B may be executed by the model processing system 100. For example, the process 600B may be implemented as a set of instructions stored in a storage device. In some embodiments, the processing device 110 (e.g., the processor 210 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 600B. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 600B may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 600B illustrated in FIG. 6B and described below is not intended to be limiting. In some embodiments, operation 530 may be performed based on the process 600B.
In 621, the processing device 110 (e.g., the quantization interval determination module 320) may determine an accumulation array by accumulating a plurality of distribution values corresponding to a plurality of initial division intervals respectively, wherein the accumulation array includes a plurality of accumulated values.
As used herein, the accumulation array may be an array determined by performing an accumulation operation on the plurality of distribution values and combining a plurality of accumulated values obtained through the accumulation operation.
In some embodiments, the quantization interval determination module 320 may determine a preset value (e.g., zero) as a first accumulated value in the accumulation array, and calculate a sum of an (n-1) th accumulated value and an (n-1) th distribution value to determine an nth accumulated value in the accumulation array, wherein n refers to an integer which is greater than or equal to 1, and less than or equal to a count of initial division intervals.
In some embodiments, the quantization interval determination module 320 may determine the accumulation array according to formula (3) below:
where {H_j} refers to the accumulation array, H_j refers to a jth accumulated value, H_j >= H_j-1, H₀ = 0, H_M = 1, D_n-1 refers to an (n-1) th distribution value, and M refers to a count of initial division intervals.
In 623, the processing device 110 (e.g., the quantization interval determination module 320) may obtain a target count of the plurality of target quantization intervals.
In some embodiments, the target count may refer to a count (denoted as K) of target quantization intervals to be determined. That is, the quantization interval determination module 320 would determine K-1 interval segmentation points (can be referred to as x_k, k=1, 2, …, K-1) within the interval [w_min, w_max] and determine K target quantization intervals based on the K-1 interval segmentation points, wherein w_min refers to a minimum weight of the plurality of weights and w_max refers to a maximum weight of the plurality of weights.
In 625, the processing device 110 (e.g., the quantization interval determination module 320) may select candidate accumulated values satisfying a preset segmentation condition from the accumulation array based on the target count.
In some embodiments, the quantization interval determination module 320 may define a preset array and select the candidate accumulated values satisfying the preset segmentation condition based on the preset array and the accumulation array.
In some embodiments, the quantization interval determination module 320 may define the preset array as follows:
T=k/K, k=1, 2, …, K-1 (4)
Further, the quantization interval determination module 320 may define the preset segmentation condition based on the preset array as follows:
H_j<=T<=H_j+1 (5)
Accordingly, the quantization interval determination module 320 may determine accumulated values H_j and H_j+1 as candidate accumulated values.
In 627, the processing device 110 (e.g., the quantization interval determination module 320) may determine a plurality of interval segmentation points based on the candidate accumulated values, a maximum weight of the plurality of weights, and a minimum weight of the plurality of weights.
In some embodiments, the quantization interval determination module 320 may determine the interval segmentation points according to formula (6) below:
where x_k refers to a kth interval segmentation point and M refers to a count of the plurality of initial division intervals. It should be understood that a left endpoint of the first initial division interval is w_min and a right endpoint of the Mth initial division interval is w_max. In some embodiments, when a jth accumulation value H_j equals k/K, the quantization interval determination module 320 may determine a right endpoint of a jth initial division interval corresponding to the jth accumulation value H_j as the interval segmentation point x_k.
In 629, the processing device 110 (e.g., the quantization interval determination module 320) may determine the plurality of target quantization intervals based on the plurality of interval segmentation points.
After determining K-1 interval segmentation points, the quantization interval determination module 320 may divide the interval [w_min, w_max] into K target quantization intervals.
It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations or modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure.
FIG. 7 is a flowchart illustrating an exemplary process for determining a target quantization interval set based on a plurality of candidate division interval sets according to some embodiments of the present disclosure. In some embodiments, process 700 may be executed by the model processing system 100. For example, the process 700 may be implemented as a set of instructions stored in a storage device. In some embodiments, the processing device 110 (e.g., the processor 210 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 700. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 700 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 700 illustrated in FIG. 7 and described below is not intended to be limiting. In some embodiments, operation 420 may be performed based on the process 700.
In 710, the processing device 110 (e.g., the quantization interval determination module 320) may obtain a target count of the plurality of target quantization intervals in the target quantization interval set.
As described in connection with operation 623, the target count may refer to a count (denoted as K) of target quantization intervals to be determined.
In 720, the processing device 110 (e.g., the quantization interval determination module 320) may obtain a plurality of candidate division interval sets by performing a plurality of division operations on the plurality of weights based on the target count. Each of the plurality of candidate division interval sets may include a plurality of candidate division intervals.
In some embodiments, each of the plurality of candidate division interval sets may include a plurality of interval position parameters. Taking a specific candidate division interval set as an example, the candidate division interval set may include a plurality of candidate division intervals, and the plurality of interval position parameters may include endpoints or segmentation points of the plurality of candidate division intervals. For example, it is assumed that a count of the candidate division intervals is K, the plurality of interval position parameters may refer to K+1 endpoints corresponding to the K candidate division intervals or K-1 segmentation points corresponding to the K candidate division intervals.
In some embodiments, the quantization interval determination module 320 may randomly select K-1 segmentation points in the interval [w_min, w_max] to determine the plurality of interval position parameters of the plurality of candidate division intervals respectively.
In 730, for each of the plurality of candidate division interval sets, the processing device 110 (e.g., the quantization interval determination module 320) may perform at least one iteration on the candidate division interval set based on an incremental set corresponding to the candidate division interval set.
In some embodiments, the incremental set may include a plurality of incremental parameters. The plurality of incremental parameters may correspond to the plurality of interval position parameters respectively. Each of the plurality of incremental parameters may indicate an updating magnitude of a corresponding interval position parameter.
For illustrative purposes, an ith candidate division interval set before a first iteration is performed may be denoted asand the incremental set corresponding to the candidate division interval set may be denoted as the ith candidate division interval set after the first iteration may be denoted asand the incremental set corresponding to the candidate division interval set may be denoted asthe ith candidate division interval set after the nth iteration may be denoted as and the incremental set corresponding to the candidate division interval set may be denoted aswherein X refers to the interval position parameter and V refers to the incremental parameter. In some embodiments, an initial value of an incremental parameter may be zero, that is, When an initial value of the incremental parameter is zero, the quantization interval determination module 320 may update the incremental set firstly, and then perform at least one iteration on the candidate division interval set based on the incremental set. In some embodiments, an initial value of an incremental parameter may be determined based on an evaluation value of a corresponding candidate division interval set. When an initial value of the incremental parameter is determined based on an evaluation value of a corresponding candidate division interval set, the quantization interval determination module 320 may perform at least one iteration on the candidate division interval set based on the incremental set directly.
In some embodiments, take a specific candidate division interval set (e.g., an ith candidate division interval set) in a specific iteration (e.g., a current iteration) as an example, the quantization interval determination module 320 may determine an evaluation value of the candidate division interval set before the iteration is performed. Further, the quantization interval determination module 320 may update an incremental set corresponding to the candidate division interval set based on the evaluation value. Furthermore, the quantization interval determination module 320 may adjust the plurality of interval position parameters in the candidate division interval set respectively based on updated incremental parameters in the updated incremental set (which may be used as an incremental set in a next iteration) . For convenience, the specific candidate division interval set is described as an example below.
In some embodiments, the quantization interval determination module 320 may determine the evaluation value of the candidate division interval set based on test data. For example, the quantization interval determination module 320 may determine candidate shared weights (corresponding to the candidate division intervals respectively in the candidate division interval set) based on the candidate division interval set, and compress a machine learning model based on the candidate shared weights. Further, the quantization interval determination module 320 may test a prediction performance of the compressed machine learning model based on a test set to determine the evaluation value of the candidate division interval set. Merely by way of example, the evaluation value may be quantified through a loss function of the machine learning model. The loss function may include prediction errors of a plurality of test samples in the test set. Merely by way of example, the evaluation value may be inversely proportional to a value of the loss function. The smaller the value of the loss function is, the larger the evaluation value of the candidate division interval set may be.
In some embodiments, the quantization interval determination module 320 may determine the evaluation value of the candidate division interval set based on a weight quantization error caused by the candidate division interval set. For example, the smaller the weight quantization error is, the larger the evaluation value of the candidate division interval set may be. More descriptions regarding the quantization error may be found elsewhere in the present disclosure.
In some embodiments, the quantization interval determination module 320 may determine the evaluation value based on the loss function of the compressed machine learning model and the weight quantization error. Merely by way of example, the quantization interval determination module 320 may determine a weighted sum of the value of the loss function and the weight quantization error as the evaluation value.
Further, after determining the evaluation value of the candidate division interval set, the quantization interval determination module 320 may update the incremental set corresponding to the candidate division interval set (e.g., the ith candidate division interval set mentioned above) based on the evaluation value.
In some embodiments, the quantization interval determination module 320 may determine a first difference between the candidate division interval set and an individual optimal solution of the candidate division interval set; determine a second difference between the candidate division interval set and a group optimal solution of the plurality of candidate division interval sets; and update the incremental set corresponding to the candidate division interval set based on the first difference and the second difference.
As used herein, the individual optimal solution of the candidate division interval set may refer to a previous candidate division interval set (e.g., a previous ith candidate division interval set corresponding to the current ith candidate division interval set) with a largest evaluation value during the previous iterations before the current iteration; the group optimal solution of the plurality of candidate division interval sets may refer to a candidate division interval set (among all the previous candidate division interval sets) with a largest evaluation value during the previous iterations before the current iteration.
For example, in the first iteration, for an ith candidate division interval set, the individual optimal solution of the ith candidate division interval set is the ith candidate division interval set itselfbefore the first iteration is performed, and the first difference may be {0, 0, …, 0} ; the group optimal solution of the plurality of candidate division interval sets may be a jth candidate division interval set with a largest evaluation value before the first iteration is performed, and the second difference may be
As another example, in an nth iteration, for an ith candidate division interval set, the individual optimal solution of the ith candidate division interval set may be a previous ith candidate division interval set (e.g., a previous ith candidate division interval setin a ηth iteration) with a largest evaluation value during the previous iterations before the nth iteration; the group optimal solution of the plurality of candidate division interval sets may be a jth candidate division interval set (e.g., a jth candidate division interval set in a ηth iteration) with a largest evaluation value during the previous iterations before the nth iteration.
In some embodiments, the quantization interval determination module 320 may update the incremental set corresponding to the candidate division interval set by adjusting the plurality of incremental parameters in the incremental set based on the first difference and the second difference. For example, in the nth iteration, the quantization interval determination module 320 may adjust the incremental parameters in the incremental set according to formula (7) below:
whererefers to a kth updated (or adjusted) incremental parameter in the ith iteration, refers to a kth incremental parameter in an (n-1) th iteration, δ₁ refers to a kth element in the first difference, δ₂ refers to a kth element in the second difference, and α, β₁, and β₂ refer to preset coefficients, which may be obtained through experience or other manners, such as a regression analysis algorithm.
Furthermore, after updating the incremental set corresponding to the candidate division interval set (e.g., the ith candidate division interval set mentioned above) , the quantization interval determination module 320 may adjust the plurality of interval position parameters in the candidate division interval set respectively based on the updated incremental parameters in the updated incremental set. For example, in the nth iteration, the quantization interval determination module 320 may adjust the plurality of interval position parameters in the candidate division interval set according to formula (8) :
whererefers to a kth updated (or adjusted) incremental parameter in the ith iteration, andrefers to a kth interval position parameter in an ith iteration.
In 740, in response to determining that a preset condition is satisfied, the processing device 110 (e.g., the quantization interval determination module 320) may determine the target quantization interval set based on the plurality of iteratively updated candidate division interval sets.
The preset condition may refer to an iteration termination condition. In some embodiments, the preset condition may include a count of iterations reaching a preset count. In some embodiments, the preset condition may include a convergence of the evaluation values of the candidate division interval sets. In some embodiments, “convergence” may refer to that a variation of the evaluation values (e.g., largest evaluation values in the iterations) in two or more consecutive iterations is equal to or smaller than a predetermined threshold. In some embodiments, the preset condition may include the evaluation value (e.g., a largest evaluation value in an iteration) of the candidate division interval set reaching a preset threshold.
In some embodiments, the quantization interval determination module 320 may determine a candidate division interval set with the largest evaluation value during the previous iterations as the target quantization interval set. In some embodiments, the quantization interval determination module 320 may determine a candidate division interval set with the largest evaluation value in the last iteration) as the target quantization interval set.
It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations or modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure.
FIG. 8 is a flowchart illustrating an exemplary process for determining a target quantization interval set based on a plurality of candidate division interval sets according to some embodiments of the present disclosure. In some embodiments, process 800 may be executed by the model processing system 100. For example, the process 800 may be implemented as a set of instructions stored in a storage device. In some embodiments, the processing device 110 (e.g., the processor 210 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 800. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 800 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 800 illustrated in FIG. 8 and described below is not intended to be limiting. In some embodiments, operation 420 may be performed based on the process 800.
In 810, the processing device 110 (e.g., the quantization interval determination module 320) may obtain a target count of the plurality of target quantization intervals in the target quantization interval set.
As described in connection with operation 623, the target count may refer to a count (denoted as K) of target quantization intervals to be determined.
In 820, the processing device 110 (e.g., the quantization interval determination module 320) may obtain a plurality of candidate division interval sets by performing a plurality of division operations on the plurality of weights based on the target count. More descriptions can be found in operation 720 which are not repeated here.
In 830, the processing device 110 (e.g., the quantization interval determination module 320) may determine the target quantization interval set by performing at least one iteration on the plurality of candidate division interval sets based on a plurality of evaluation values corresponding to the plurality of candidate division interval sets respectively.
More descriptions can be found in operation 730 which are not repeated here.
In some embodiments, the quantization interval determination module 320 may determine a plurality of first candidate interval sets from the plurality of candidate division interval sets based on the plurality of evaluation values corresponding to the plurality of candidate division interval sets respectively.
In some embodiments, the quantization interval determination module 320 may determine candidate division interval sets with evaluation values greater than a preset threshold as the first candidate interval sets. In some embodiments, the quantization interval determination module 320 may determine a preset count (e.g., top 3, top 5, top 10) or a preset ratio (e.g., top 1%, top 2%, top 5%, top 10%) of candidate division interval sets based on the evaluation values as the first candidate interval sets.
Further, the quantization interval determination module 320 may determine a plurality of second candidate interval sets by performing a first transformation operation and/or a second transformation operation on the plurality of first candidate interval sets.
In some embodiments, take a specific first candidate interval set as an example, the first transformation operation may include adjusting interval position parameters (e.g., endpoints or segmentation points) of the first candidate interval set. The second transformation operation may include interchanging corresponding interval position parameters or parts thereof in a pair of first candidate interval sets.
Furthermore, the quantization interval determination module 320 may determine a plurality of third candidate interval sets from the plurality of first candidate interval sets and/or the plurality of second candidate interval sets based on evaluation values of the plurality of first candidate interval sets and/or evaluation values of the plurality of second candidate interval sets.
In some embodiments, the quantization interval determination module 320 may determine the plurality of third candidate interval sets from the plurality of first candidate interval sets based on evaluation values of the plurality of first candidate interval sets. In some embodiments, the quantization interval determination module 320 may determine the plurality of third candidate interval sets from the plurality of second candidate interval sets based on evaluation values of the plurality of second candidate interval sets. In some embodiments, the quantization interval determination module 320 may determine the plurality of third candidate interval sets from the plurality of first candidate interval sets and the plurality of second candidate interval sets based on evaluation values of the plurality of first candidate interval sets and evaluation values of the plurality of second candidate interval sets.
In some embodiments, the quantization interval determination module 320 may determine the candidate division interval sets (e.g., the first candidate interval sets and/or the second candidate interval sets) with evaluation values greater than a preset threshold as the third candidate interval sets. In some embodiments, the quantization interval determination module 320 may determine a preset count (e.g., top 3, top 5, top 10) or a preset ratio (e.g., top 1%, top 2%, top 5%, top 10%) of candidate division interval sets (e.g., the first candidate interval sets and/or the second candidate interval sets) based on the evaluation values as the third candidate interval sets.
In some embodiments, the quantization interval determination module 320 may determine the plurality of third candidate interval sets as candidate division interval sets of a next iteration.
As described in connection with operation 740, in response to determining that the preset condition is satisfied, the quantization interval determination module 320 may terminate the iteration, and determine a candidate division interval set with the largest evaluation value during the previous iterations or a candidate division interval set with the largest evaluation value in the last iteration as the target quantization interval set.
It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations or modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure.
FIG. 9 is a flowchart illustrating an exemplary process for determining a target quantization interval set based on a plurality of candidate quantization interval sets according to some embodiments of the present disclosure. In some embodiments, process 900 may be executed by the model processing system 100. For example, the process 900 may be implemented as a set of instructions stored in a storage device. In some embodiments, the processing device 110 (e.g., the processor 210 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 900. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 900 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 900 illustrated in FIG. 9 and described below is not intended to be limiting. In some embodiments, operation 420 may be performed based on the process 900.
In 910, the processing device 110 (e.g., the quantization interval determination module 320) may determine a plurality of candidate quantization interval sets according to a plurality of preset division manners. Each of the plurality of candidate quantization interval sets may include a plurality of candidate quantization intervals.
In some embodiments, the plurality of preset division manners may include the division manners provided in the present disclosure, such as the division manners described in the process 500, the process 700, and the process 800.
In 920, for each of the plurality of candidate quantization interval sets, the processing device 110 (e.g., the quantization interval determination module 320) may obtain a plurality of candidate shared weights corresponding to the plurality of candidate quantization intervals respectively in the candidate quantization interval set, and obtain test data of the plurality of candidate shared weights.
In some embodiments, as described elsewhere in the present disclosure, the quantization interval determination module 320 may compress a machine learning model based on the plurality of candidate shared weights and test a prediction performance of the compressed machine learning model based on a test set to obtain the test data of the plurality of candidate shared weights.
In some embodiments, after the compressed machine learning model is acquired, the quantization interval determination module 320 may transmit the compressed machine learning model to the terminal device 120 (e.g., an image capturing terminal) that is available for testing, and configure the compressed machine learning model for the terminal device 120. The terminal device 120 may test an output result (e.g., an image may be output) of the compressed machine learning model based on the test set. The quantization interval determination module 320 may obtain the output result from the terminal device 120, evaluate the effect (e.g., the clarity of an image, etc. ) of the output result, and obtain the test data. In some embodiments, the test set used by the terminal device 120 may be pre-stored or obtained from the processing device 110 (e.g., the quantization interval determination module 320) . In some embodiments, the terminal device 120 may obtain different test sets from the processing device 110 based on different compressed machine learning models.
In 930, the processing device 110 (e.g., the quantization interval determination module 320) may select the target quantization interval set from the plurality of candidate quantization interval sets based on the test data.
In some embodiments, the quantization interval determination module 320 may determine a candidate quantization interval set with an optimal test performance (e.g., a loss function with a smallest value) of the plurality of candidate quantization interval sets as the target quantization interval set.
In some embodiments, the quantization interval determination module 320 may compress the machine learning model based on the target quantization interval set after the target quantization interval set is determined, so as to obtain a target machine learning model. The quantization interval determination module 320 may further transmit the target machine learning model to the terminal device 120 to which the target machine learning model needs to be applied, and configure the target machine learning model for the terminal device 120. It should be noted that the terminal device configured with the target machine learning model may be the same terminal device as the terminal device for testing or a different terminal device from the terminal device for testing, which is not limited herein.
It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations or modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure.
Having thus described the basic concepts, it may be rather apparent to those skilled in the art after reading this detailed disclosure that the foregoing detailed disclosure is intended to be presented by way of example only and is not limiting. Various alterations, improvements, and modifications may occur and are intended to those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested by this disclosure, and are within the spirit and scope of the exemplary embodiments of this disclosure.
Moreover, certain terminology has been used to describe embodiments of the present disclosure. For example, the terms “one embodiment, ” “an embodiment, ” and/or “some embodiments” mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, it is emphasized and should be appreciated that two or more references to “an embodiment” or “one embodiment” or “an alternative embodiment” in various portions of this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined as suitable in one or more embodiments of the present disclosure.
Further, it will be appreciated by one skilled in the art, aspects of the present disclosure may be illustrated and described herein in any of a number of patentable classes or context including any new and useful process, machine, manufacture, or collocation of matter, or any new and useful improvement thereof. Accordingly, aspects of the present disclosure may be implemented entirely hardware, entirely software (including firmware, resident software, micro-code, etc. ) or combining software and hardware implementation that may all generally be referred to herein as a “unit, ” “module, ” or “system. ” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable media having computer-readable program code embodied thereon.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including electro-magnetic, optical, or the like, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that may communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable signal medium may be transmitted using any appropriate medium, including wireless, wireline, optical fiber cable, RF, or the like, or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB. NET, Python or the like, conventional procedural programming languages, such as the “C” programming language, Visual Basic, Fortran 2103, Perl, COBOL 2102, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code may execute entirely on the user’s computer, partly on the user’s computer, as a stand-alone software package, partly on the user’s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user’s computer through any type of network, including a local area network (LAN) or a wide area network (WAN) , or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) or in a cloud computing environment or offered as a service such as a Software as a Service (SaaS) .
Furthermore, the recited order of processing elements or sequences, or the use of numbers, letters, or other designations therefore, is not intended to limit the claimed processes and methods to any order except as may be specified in the claims. Although the above disclosure discusses through various examples what is currently considered to be a variety of useful embodiments of the disclosure, it is to be understood that such detail is solely for that purpose, and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover modifications and equivalent arrangements that are within the spirit and scope of the disclosed embodiments. For example, although the implementation of various components described above may be embodied in a hardware device, it may also be implemented as a software only solution, for example, an installation on an existing server or mobile device.
Similarly, it should be appreciated that in the foregoing description of embodiments of the present disclosure, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the various embodiments. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed subject matter requires more features than are expressly recited in each claim. Rather, claimed subject matter may lie in less than all features of a single foregoing disclosed embodiment.
In some embodiments, the numbers expressing quantities or properties used to describe and claim certain embodiments of the application are to be understood as being modified in some instances by the term “about, ” “approximate, ” or “substantially. ” For example, “about, ” “approximate, ” or “substantially” may indicate ±1%, ±5%, ±10%, or ±20%variation of the value it describes, unless otherwise stated. Accordingly, in some embodiments, the numerical parameters set forth in the written description and attached claims are approximations that may vary depending upon the desired properties sought to be obtained by a particular embodiment. In some embodiments, the numerical parameters should be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of some embodiments of the application are approximations, the numerical values set forth in the specific examples are reported as precisely as practicable.
Each of the patents, patent applications, publications of patent applications, and other material, such as articles, books, specifications, publications, documents, things, and/or the like, referenced herein is hereby incorporated herein by this reference in its entirety for all purposes, excepting any prosecution file history associated with same, any of same that is inconsistent with or in conflict with the present document, or any of same that may have a limiting effect as to the broadest scope of the claims now or later associated with the present document. By way of example, should there be any inconsistency or conflict between the description, definition, and/or the use of a term associated with any of the incorporated material and that associated with the present document, the description, definition, and/or the use of the term in the present document shall prevail.
In closing, it is to be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of the application. Other modifications that may be employed may be within the scope of the application. Thus, by way of example, but not of limitation, alternative configurations of the embodiments of the application may be utilized in accordance with the teachings herein. Accordingly, embodiments of the present application are not limited to that precisely as shown and described.

Claims

A method for compressing a machine learning model, comprising:

obtaining a plurality of weights of the machine learning model;

determining a target quantization interval set of the plurality of weights, the target quantization interval set including a plurality of target quantization intervals; and

determining a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively, the plurality of target shared weights being used to compress the machine learning model.
The method of claim 1, wherein the determining a target quantization interval set of the plurality of weights includes:

determining a plurality of initial division intervals of the plurality of weights;

determining a distribution of the plurality of weights in the plurality of initial division intervals; and

determining the plurality of target quantization intervals in the target quantization interval set by adjusting the plurality of initial division intervals based on the distribution, wherein a difference of distribution values of weights among at least two of the plurality of target quantization intervals is less than a preset difference threshold.
The method of claim 2, wherein the determining a distribution of the plurality of weights in the plurality of initial division intervals includes:

for each of the plurality of initial division intervals,

determining an initial count of weights in the initial division interval; and

determining a distribution value of weights corresponding to the initial division interval by performing a statistical operation on the initial count of weights.
The method of claim 3, wherein the determining a distribution value of weights corresponding to the initial division interval by performing a statistical operation on the initial count of weights includes:

determining an adjusted value of the initial count of weights by performing an adjustment operation on the initial count of weights; and

determining the distribution value of weights corresponding to the initial division interval by performing a normalization operation on the adjusted value.
The method of claim 4, wherein the determining an adjusted value of the initial count of weights by performing an adjustment operation on the initial count of weights including at least one of:

determining the adjusted value by performing a numerical limit operation on the initial count of weights;

determining the adjusted value by performing a transformation operation on the initial count of weights;

determining a first intermediate value by performing the numerical limit operation on the initial count of weights, and determining the adjusted value by performing the transformation operation on the first intermediate value; or

determining a second intermediate value by performing the transformation operation on the initial count of weights, and determining the adjusted value by performing the numerical limit operation on the second intermediate value.
The method of claim 5, wherein the determining the adjusted value by performing a numerical limit operation on the initial count of weights includes:

in response to the determining that the initial count of weights is equal to zero, determining the adjusted value as zero;

in response to the determining that the initial count of weights is greater than 0 and less than a minimum value of a preset numerical limit range, determining the adjusted value as the minimum value;

in response to the determining that the initial count of weights is within the preset numerical limit range, determining the adjusted value as the initial count of weights; or

in response to the determining that the initial count of weights is greater than a maximum value of the preset numerical limit range, determining the adjusted value as the maximum value.
The method of claim 5, wherein the determining the adjusted value by performing a transformation operation on the initial count of weights includes:

performing the transformation operation on the initial count of weights based on a preset transformation function, the preset transformation function including at least one of a constant function, an identity function, or a square root function.
The method of claim 3, wherein the determining the plurality of target quantization intervals in the target quantization interval set by adjusting the plurality of initial division intervals based on the distribution includes:

determining an accumulation array by accumulating the plurality of distribution values, the accumulation array including a plurality of accumulated values; and

determining the plurality of target quantization intervals based on the plurality of accumulated values, the plurality of weights, and a count of the plurality of initial division intervals.
The method of claim 8, wherein the determining the plurality of target quantization intervals based on the plurality of accumulated values, the plurality of weights, and a count of the plurality of initial division intervals includes:

obtaining a target count of the plurality of target quantization intervals;

selecting, candidate accumulated values satisfying a preset segmentation condition from the accumulation array based on the target count;

determining a plurality of interval segmentation points based on the candidate accumulated values, a maximum weight of the plurality of weights, and a minimum weight of the plurality of weights; and

determining the plurality of target quantization intervals based on the plurality of interval segmentation points.
The method of claim 1, wherein the determining a target quantization interval set of the plurality of weights includes:

obtain a target count of the plurality of target quantization intervals in the target quantization interval set;

obtaining a plurality of candidate division interval sets by performing a plurality of division operations on the plurality of weights based on the target count, each of the plurality of candidate division interval sets including a plurality of interval position parameters;

for each of the plurality of candidate division interval sets, performing at least one iteration on the candidate division interval set based on an incremental set corresponding to the candidate division interval set, wherein the incremental set includes a plurality of incremental parameters, the plurality of incremental parameters corresponding to the plurality of interval position parameters respectively, and each of the plurality of incremental parameters indicating an updating magnitude of a corresponding interval position parameter; and

in response to determining that a preset condition is satisfied, determining the target quantization interval set based on a plurality of iteratively updated candidate division interval sets.
The method of claim 10, wherein the performing at least one iteration on the candidate division interval set based on an incremental set corresponding to the candidate division interval set includes:

determining an evaluation value of the candidate division interval set;

updating the incremental set corresponding to the candidate division interval set based on the evaluation value;

adjusting the plurality of interval position parameters in the candidate division interval set respectively based on updated incremental parameters in the updated incremental set; and

determining the updated incremental set as an incremental set of a next iteration.
The method of claim 11, wherein the updating the incremental set corresponding to the candidate division interval set based on the evaluation value includes:

determining a first difference between the candidate division interval set and an individual optimal solution of the candidate division interval set;

determining a second difference between the candidate division interval set and a group optimal solution of the plurality of candidate division interval sets; and

updating the incremental set by adjusting the plurality of incremental parameters based on the first difference and the second difference.
The method of claim 1, wherein the determining a target quantization interval set of the plurality of weights includes:

obtaining a target count of the plurality of target quantization intervals in the target quantization interval set;

obtaining a plurality of candidate division interval sets by performing a plurality of division operations on the plurality of weights based on the target count; and

determining the target quantization interval set by performing at least one iteration on the plurality of candidate division interval sets based on a plurality of evaluation values corresponding to the plurality of candidate division interval sets respectively.
The method of claim 13, wherein the determining the target quantization interval set by performing at least one iteration on the plurality of candidate division interval sets based on a plurality of evaluation values corresponding to the plurality of candidate division interval sets respectively includes:

determining a plurality of first candidate interval sets from the plurality of candidate division interval sets based on the plurality of evaluation values corresponding to the plurality of candidate division interval sets respectively;

determining a plurality of second candidate interval sets by performing a first transformation operation and/or a second transformation operation on the plurality of first candidate interval sets;

determining a plurality of third candidate interval sets from the plurality of first candidate interval sets and/or the plurality of second candidate interval sets based on evaluation values of the plurality of first candidate interval sets and/or evaluation values of the plurality of second candidate interval sets; and

determining the plurality of third candidate interval sets as candidate division interval sets of a next iteration.
The method of claim 1, wherein the determining a target quantization interval set of the plurality of weights includes:

determining a plurality of candidate quantization interval sets according to a plurality of preset division manners, each of the plurality of candidate quantization interval sets including a plurality of candidate quantization intervals;

for the each of the plurality of candidate quantization interval sets,

obtaining a plurality of candidate shared weights corresponding to the plurality of candidate quantization intervals respectively in the candidate quantization interval set; and

obtaining test data of the plurality of candidate shared weights; and

selecting the target quantization interval set from the plurality of candidate quantization interval sets based on the test data.
The method of claim 1, wherein the determining a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively includes:

for each of the plurality of target quantization intervals,

determining an average value of a maximum weight and a minimum weight in the target quantization interval as a target shared weight corresponding to the target quantization interval; or

determining an average value of weights in the target quantization interval as a target shared weight corresponding to the target quantization interval; or

determining a median value of weights in the target quantization interval as a target shared weight corresponding to the target quantization interval.
The method of claim 1, further including:

obtaining test data of the plurality of target shared weights;

determining whether the test data satisfies a preset condition;

in response to the determining that the test data does not satisfy the preset condition, adjusting the plurality of target quantization intervals; and

updating the plurality of target shared weights based on the plurality of adjusted target quantization intervals.
The method of claim 1, further including:

determining a target model by compressing the machine learning model based on the plurality of target shared weights; and

transmitting the target model to a storage device of an acquisition device and storing the target model in the storage device, the target model being used to achieve an inference operation on data collected by the acquisition device.
A system for compressing a machine learning model, comprising:

a weight obtaining module, configured to obtain a plurality of weights of the machine learning model;

a quantization interval determination module, configured to determine at least one quantization interval set based on the plurality of weights through at least one preset division manner, each of the at least one quantization interval set including a plurality of quantization intervals; and

a shared weight determination module, configured to determine a shared weight of each of the plurality of quantization intervals, the plurality of shared weights being used to compress the machine learning model.
A model transformation tool, comprising:

at least one storage device including a set of instructions; and

at least one processor configured to communicate with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to direct the system to perform operations including:

obtaining a machine learning model and performing a quantitative operation on the machine learning model;

in response to determining that a result of the quantitative operation does not satisfy a preset condition, triggering a model compression process, the model compression process including:

obtaining a plurality of weights of the machine learning model;

determining a target quantization interval set of the plurality of weights, the target quantization interval set including a plurality of target quantization intervals; and

determining a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively, the plurality of target shared weights being used to compress the machine learning model.
A non-transitory computer readable medium including executable instructions, the instructions, when executed by at least one processor, causing the at least one processor to effectuate a method comprising:

obtaining a plurality of weights of the machine learning model;

determining a target quantization interval set of the plurality of weights, the target quantization interval set including a plurality of target quantization intervals; and

determining a plurality of target shared weights corresponding to the plurality of target quantization intervals respectively, the plurality of target shared weights being used to compress the machine learning model.