KR20230024950A - Method and system for determining optimal parameter - Google Patents

Abstract

Disclosed are a method and system for determining an optimal parameter. The method for determining the optimal parameter according to one embodiment may comprise: a first step of selecting a parameter combination for an inference model; a second step of compressing the inference model using the selected parameter combination; a third step of transmitting the compressed inference model to a target device; a fourth step of receiving a performance of the compressed inference model from the target device; and a fifth step of determining whether or not to perform repetition of the first step to the fourth step. Here, the optimal parameter combination may be determined according to the received performance.

Description

Optimal parameter determination method and system {METHOD AND SYSTEM FOR DETERMINING OPTIMAL PARAMETER}

The following description relates to an optimal parameter determination method and system.

Lightweighting of deep learning models (or artificial intelligence models) refers to functions, modules and/or features that make a given deep learning model into a smaller deep learning model. Here, 'small' may mean reducing the number of weights/bias constituting the deep learning model, reducing capacity, or speeding up inference. At this time, it is very important not to degrade the performance while reducing the weight.

There are various types of lightweighting techniques. Large classifications include pruning, quantization, knowledge distillation, model search (Neural Architecture Search), and filter decomposition, and within each classification, there are many different types of lightweighting techniques. exist.

At this time, each weight reduction technique cannot be simply used. There are parameters for using each lightweight technique. For example, in the case of pruning, the parameters for how many parameters to pruning for each layer must be adjusted in advance, and how the parameters are set greatly affects the lightweighting performance.

[Prior document number]

Korean Patent Publication No. 10-2020-0104201

Provided is an optimal parameter determination method and system capable of compressing an inference model and providing it to a target device, and determining an optimal parameter combination based on the performance of the compressed inference model provided from the target device.

A method for determining an optimal parameter performed by a computer device including at least one processor, comprising: a first step of selecting, by the at least one processor, a parameter combination for an inference model; a second step of compressing, by the at least one processor, the inference model using the selected parameter combination; a third step of transmitting the compressed inference model to a target device by the at least one processor; a fourth step of receiving, by the at least one processor, performance of the compressed inference model from the target device; and a fifth step of determining, by the at least one processor, whether to repeat the first to fourth steps.

According to one aspect, the first step may include selecting at least one compression method for compressing the reasoning model; and selecting a combination of values of at least some of the parameters of the selected at least one compression method as the parameter combination.

According to another aspect, the second step may include (1) at least one of pruning, quantization, knowledge distillation, model search (Neural Architecture Search), and filter decomposition. and (2) compressing the inference model using the based compression method and (2) the parameter combination.

According to another aspect, the target device may be implemented to measure performance including at least one of latency and accuracy for the compressed inference model.

According to another aspect, when the first step is repeatedly performed by the fifth step, a parameter combination including a different value from a previously selected parameter combination may be selected.

According to another aspect, the fifth step is characterized by determining whether the performance satisfies a preset constraint or whether to repeat the first to fourth steps based on a preset number of times. can do.

According to another aspect, the optimal parameter determination method may further include a sixth step of determining an optimal parameter combination according to the received performance.

A computer program stored in a computer readable recording medium is provided in combination with a computer device to execute the method on the computer device.

A computer readable recording medium having a program for executing the method in a computer device is recorded.

A first process comprising at least one processor implemented to execute instructions readable by a computer device, and selecting a parameter combination for an inference model by the at least one processor; a second step of compressing, by the at least one processor, the inference model using the selected parameter combination; a third process of transmitting the compressed inference model to a target device by the at least one processor; a fourth step of receiving, by the at least one processor, performance of the compressed inference model from the target device; and a fifth process of determining, by the at least one processor, whether to repeat the first to fourth processes according to the received performance.

The inference model is compressed and provided to the target device, and an optimal parameter combination may be determined based on performance of the compressed inference model provided from the target device.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.
2 is a block diagram illustrating an example of a computer device according to one embodiment of the present invention.
3 is a diagram showing an example of a lightweight system according to an embodiment of the present invention.
4 is a diagram illustrating an example of an optimal parameter determination process according to an embodiment of the present invention.
5 is a block diagram illustrating an example of an internal configuration of a HPO according to an embodiment of the present invention.
6 is a flowchart illustrating an example of a method for determining an optimal parameter of HPO according to an embodiment of the present invention.
7 is a block diagram illustrating an example of an internal configuration of a target device according to an embodiment of the present invention.
8 is a flowchart illustrating an example of a method for determining optimal parameters of a target device according to an embodiment of the present invention.

Hereinafter, an embodiment will be described in detail with reference to the accompanying drawings.

An optimum parameter determination system according to embodiments of the present invention may be implemented by at least one computer device. At this time, a computer program according to an embodiment of the present invention may be installed and driven in the computer device, and the computer device may perform the optimal parameter determination method according to the embodiments of the present invention under the control of the driven computer program. there is. The above-described computer program may be combined with a computer device and stored in a computer readable recording medium to execute the optimal parameter determination method on a computer.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. The network environment of FIG. 1 shows an example including a plurality of electronic devices 110 , 120 , 130 , and 140 , a plurality of servers 150 and 160 , and a network 170 . 1 is an example for explanation of the invention, and the number of electronic devices or servers is not limited as shown in FIG. 1 . In addition, the network environment of FIG. 1 only describes one example of environments applicable to the present embodiments, and the environment applicable to the present embodiments is not limited to the network environment of FIG. 1 .

The plurality of electronic devices 110, 120, 130, and 140 may be fixed terminals implemented as computer devices or mobile terminals. Examples of the plurality of electronic devices 110, 120, 130, and 140 include a smart phone, a mobile phone, a navigation device, a computer, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), and a portable multimedia player (PMP). ), and tablet PCs. As an example, FIG. 1 shows the shape of a smartphone as an example of the electronic device 110, but in the embodiments of the present invention, the electronic device 110 substantially uses a wireless or wired communication method to transmit other information via the network 170. It may refer to one of various physical computer devices capable of communicating with the electronic devices 120 , 130 , and 140 and/or the servers 150 and 160 .

The communication method is not limited, and short-distance wireless communication between devices as well as a communication method utilizing a communication network (eg, a mobile communication network, a wired Internet, a wireless Internet, and a broadcasting network) that the network 170 may include may also be included. For example, the network 170 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , one or more arbitrary networks such as the Internet. In addition, the network 170 may include any one or more of network topologies including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, and the like. Not limited.

Each of the servers 150 and 160 communicates with the plurality of electronic devices 110, 120, 130, and 140 through the network 170 to provide commands, codes, files, contents, services, and the like, or a computer device or a plurality of computers. Can be implemented in devices. For example, the server 150 provides a service (eg, an instant messaging service, a social network service, a payment service, a virtual exchange) to a plurality of electronic devices 110, 120, 130, and 140 connected through the network 170. service, risk monitoring service, game service, group call service (or voice conference service), messaging service, mail service, map service, translation service, financial service, search service, content provision service, etc.).

2 is a block diagram illustrating an example of a computer device according to one embodiment of the present invention. Each of the plurality of electronic devices 110 , 120 , 130 , and 140 or each of the servers 150 and 160 described above may be implemented by the computer device 200 shown in FIG. 2 .

As shown in FIG. 2 , the computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input/output interface 240. The memory 210 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, a non-perishable mass storage device such as a ROM and a disk drive may be included in the computer device 200 as a separate permanent storage device distinct from the memory 210 . Also, an operating system and at least one program code may be stored in the memory 210 . These software components may be loaded into the memory 210 from a computer-readable recording medium separate from the memory 210 . The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, software components may be loaded into the memory 210 through the communication interface 230 rather than a computer-readable recording medium. For example, software components may be loaded into memory 210 of computer device 200 based on a computer program installed by files received over network 170 .

The processor 220 may be configured to process commands of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processor 220 by memory 210 or communication interface 230 . For example, processor 220 may be configured to execute received instructions according to program codes stored in a recording device such as memory 210 .

The communication interface 230 may provide a function for the computer device 200 to communicate with other devices (eg, storage devices described above) through the network 170 . For example, a request, command, data, file, etc. generated according to a program code stored in a recording device such as the memory 210 by the processor 220 of the computer device 200 is controlled by the communication interface 230 to the network ( 170) to other devices. Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device 200 through the communication interface 230 of the computer device 200 via the network 170 . Signals, commands, data, etc. received through the communication interface 230 may be transferred to the processor 220 or the memory 210, and files, etc. may be stored as storage media that the computer device 200 may further include (described above). permanent storage).

The input/output interface 240 may be a means for interface with the input/output device 250 . For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input/output interface 240 may be a means for interface with a device in which functions for input and output are integrated into one, such as a touch screen. At least one of the input/output devices 250 may be configured as one device with the computer device 200 . For example, like a smart phone, a touch screen, a microphone, a speaker, and the like may be implemented in a form included in the computer device 200 .

Also, in other embodiments, computer device 200 may include fewer or more elements than those of FIG. 2 . However, there is no need to clearly show most of the prior art components. For example, the computer device 200 may be implemented to include at least some of the aforementioned input/output devices 250 or may further include other components such as a transceiver and a database.

3 is a diagram showing an example of a lightweight system according to an embodiment of the present invention. The lightweight system 300 according to this embodiment includes a hyperparameter optimizer 310 (hereinafter referred to as Hyperparameter Optimization (HPO)), a target device pool (320), a compression method pool (330), and A compression pipeline 340 may be included. The optimal parameter determination system may be implemented in a form included in the lightweight system 300 and may include at least the HPO 310 and the target device 350 . The optimal parameter determination method will be described in more detail later.

Since lightweighting techniques are highly dependent on parameters, when a plurality of lightweighting techniques are used, performance may be greatly influenced by how the parameters of each lightweighting technique are set. To solve this problem, the lightweight system 300 may include the HPO 310 and the target device pool 320 .

The HPO 310 may be an algorithm for finding an optimal hyperparameter in a given hyperparameter search space, and the processor 220 of the computer device 200 that implements the lightweight system 300 is substantially It can be a functional representation of a function that operates under the control of a program. For example, the HPO 310 learns parameter combination 1, parameter combination 2, ??, and parameter combination N among possible parameter combinations, discards some parameter combinations with low performance, and selects a parameter combination with good upper performance. Based on the parameter combinations, new parameter combinations can be explored. Examples of hyperparameters include batch size, learning rate, and momentum. When the category of the hyperparameter is set to the number of layers, the number of neurons, and the type of layer, the HPO 310 may include Neural Architecture Search (NAS).

The HPO 310 according to the present embodiment may process a search in a different search space. Parameters of a number of lightweight techniques may be a search space. For example, the search space of the HPO 310 may be a pruning ratio, a quantization threshold, a temperature in KD in knowledge distillation, and the like. For example, the HPO 310 may utilize algorithms such as Hyperband and Bayesian Optimization.

Meanwhile, the target device pool 320 and the compression method pool 330 may be implemented in the form of a database, for example. The target device pool 320 may include information on various devices, and the compression method pool 330 may include codes for each of the various compression methods. The HPO 310 may select a compression method from the compression method pool 330 and may lighten the inference model using the selected compression method. Devices included in the target device pool 320 and compression methods included in the compression method pool 330 may utilize well-known devices and compression methods.

At this time, in lightening the inference model, the HPO 310 may select two or more compression methods from the compression method pool 330 and sequentially place the selected two or more compression methods in the compression pipeline 340. After that, the HPO 310 may input the inference model to the compression pipeline 340 to process weight reduction of the inference model so that the inference model is sequentially compressed by two or more compression methods. Depending on embodiments, the compression pipeline 340 may be implemented in a form included in the HPO 310 .

Also, the HPO 310 may generate a lightweight model for each of various combinations of compression methods. Depending on the embodiment, the HPO 310 may generate a plurality of lightweight models in parallel by applying different combinations of compression methods to one inference model by operating a plurality of compression pipelines. For example, when multiple target devices exist, the HPO 310 can operate multiple compression pipelines to simultaneously generate multiple lightweight models for multiple target devices.

In addition, the HPO 310 may transmit the lightweight inference model to the selected target device 350 through the target device pool 320 . The target device 350 may execute code of the lightweight inference model to measure performance such as latency and accuracy, and then return the measured performance to the HPO 310 . The HPO 310 may determine superiority or inferiority among parameter combinations based on the returned performance, and may find a parameter combination optimized for the target device 350 according to the superiority or inferiority.

For this process, the HPO 310 may receive, for example, an inference model for lightweighting, a dataset (including data and labels), and constraints. Here, the constraint may include a value for at least one of device, accuracy, model size, latency, compression time, and energy consumption.

Device constraints may include information for selecting the target device 350 . The lightweight system 300 may select a target device 350 from the target device pool 320 according to device limitations.

Also, the restriction of accuracy may be a minimum threshold value of accuracy that a lightweight inference model should have. In other words, the HPO 310 may lighten the inference model so that the lightweight inference model has accuracy equal to or greater than a minimum threshold value according to an accuracy restriction. For example, the HPO 310 may select a parameter combination whose accuracy as performance returned by the target device 350 is greater than or equal to a minimum threshold value according to accuracy restrictions.

The restriction on the size of the model may be a restriction on the size of the lightweight model. When a model size restriction is set, the HPO 310 may perform a performance test using lightweight inference models having a size less than (or less than) the model size restriction among lightweight inference models.

The delay time restriction may be a restriction on the time required for the lightweight inference model to generate an output value for an input value. The delay time for the lightweight inference model may be included in performance returned to the PHO 310 by the target device. The PHO 310 may select a parameter combination by selecting a lightweight inference model that satisfies the delay time constraint based on the delay time included in the returned performance.

The limitation of compression time may be a limitation of time required to generate a lightweight inference model. For example, the time to generate an inference model that satisfies a desired input condition depends on the performance and resources of a system for compressing an inference model, and it may take several days to compress one inference model. However, when the user sets compression time constraints, the HPO 310 designates the maximum number of learning times (epochs) according to the set compression time constraints or reduces the number of sequentially applied compression methods to reduce the weight of the inference model. The generation time of can be adjusted so that it does not exceed the limit of the delay time set by the user.

The restriction on energy consumption may include a restriction on energy consumption in the target device when measuring performance of a lightweight inference model in the target device. In other words, the HPO 310 may select a parameter combination in which energy consumption in the target device does not exceed the energy consumption limit set by the user. To this end, an energy consumption measurement module may be included in the target device, and energy consumption measured in the target device may be transmitted to the HPO 310 as part of the performance of the lightweight inference model.

Meanwhile, a result (lightweight inference model) that satisfies all constraints may be generated, but may not be. For example, in the performance of lightweight inference models, accuracy may need to be lowered for lower latency. As another example, accuracy may need to be reduced for lower energy consumption. Accordingly, priorities may be assigned to the constraints, and the HPO 310 may first satisfy constraints having a higher priority and perform model optimization that satisfies lower constraints according to the specified priorities.

4 is a diagram illustrating an example of an optimal parameter determination process according to an embodiment of the present invention. 4 shows HPO 310 and target device 350 . The embodiment of FIG. 4 describes an example of a process in which the HPO 310 determines an optimal parameter.

In the parameter selection process 431, the HPO 310 may select parameters for the input inference model 410. As already described, the inference model 410 may be a pretrained model, and the parameter may be a combination of parameters for a combination of multiple compression methods.

In the model compression process 432, the HPO 310 may compress the inference model 410 using the selected parameters. The compressed inference model may be transmitted to the target device 350 . At this time, along with the compressed inference model, a dataset (including data and labels) input for the inference model 410 may be transmitted to the target device 350 together.

In the model receiving process 433, the target device 350 may receive the compressed inference model delivered from the HPO 310. As already described, the target device 350 may receive a dataset together with a compressed inference model.

In the model test process 434, the target device 350 may test the compressed inference model. For example, the target device 350 may test the compressed inference model using the data of the dataset and the correct label to measure the performance (eg, latency, accuracy, etc.) of the compressed inference model, and measure the performance of the compressed inference model. The performance can be delivered to the HPO (310).

In the repetition process 435 , the HPO 310 may determine whether to repeat the parameter selection process 431 to the model test process 434 according to the performance transmitted from the target device 350 . For example, the HPO 310 may determine whether or not the compressed inference model satisfies all constraints or priorities-based constraints based on the delivered performance or more than a predetermined criterion. If satisfied, the HPO 310 may provide the compressed inference model as the final lightweight inference model 420 without repeating the parameter selection process 431 to the model test process 434 . On the other hand, if not satisfied, the HPO 310 may repeat the parameter selection process 431 to the model test process 434 to test the compressed inference model again according to the new parameters.

Depending on the embodiment, the iterative process 435 may simply be a process for testing a predetermined number of compressed inference models compressed through different parameters. In this case, the HPO 310 may provide a compressed inference model with the best performance in terms of constraints as the final lightweight inference model 420 .

In another embodiment, the iterative process 435 may be a process for testing one compressed inference model on different predetermined number of target devices.

5 is a block diagram illustrating an example of an internal configuration of a HPO according to an embodiment of the present invention, and FIG. 6 is a flowchart illustrating an example of a method for determining optimal parameters of an HPO according to an embodiment of the present invention. The HPO 310 described above may be implemented by the computer device 200 . The HPO 310 of FIG. 5 includes a parameter selection unit 510, a model weight reduction unit 520, a model transmission unit 530, a result reception unit 540, an iteration determination unit 550, and an optimal parameter determination unit 560. can do. At this time, the parameter selection unit 510, the model weight reduction unit 520, the model transmission unit 530, the result reception unit 540, the iteration determination unit 550, and the optimal parameter determination unit 560 implement the HPO 310. The processor 220 of the computer device 200 may be a functional expression of a function that operates under the control of a computer program. For example, the processor 220 of the computer device 200 may be implemented to execute a control instruction according to an operating system code or at least one computer program code included in the memory 210 . Here, the processor 220 controls the computer device 200 so that the computer device 200 performs the steps 610 to 650 included in the method of FIG. 6 according to a control command provided by a code stored in the computer device 200. can control. At this time, as a functional expression of the processor 220 for the execution of each step 610 to 650, a parameter selection unit 510, a model weight reduction unit 520, a model transmission unit 530, a result reception unit 540, and an iteration determination Unit 550 and optimal parameter determination unit 560 may be used.

In step 610, the parameter selector 510 may select a parameter combination for an inference model. As an example, the parameter selection unit 510 may select at least one compression method as a combination of compression methods from the compression method pool 330 described with reference to FIG. 3 . At this time, the compression method pool 320 includes pruning, quantization, knowledge distillation, model search (Neural Architecture Search), filter decomposition, and filter decomposition. It may contain more than one compression method based on at least one of the

In this case, the parameter selector 510 may select compression methods according to a certain rule. For example, the parameter selection unit 510 determines that a compression method based on activation change and a first rule that a compression method based on quantization must be located at the end of a combination of compression methods are included in the combination of compression methods. A combination of compression methods may be selected according to at least one of the second rules to be included before the based compression method. For example, since quantization is often implemented in conjunction with a compiler, when quantization is used for compression at the software level, quantization may be placed at the end of a compression pipeline. Also, since the activation transform can be used for the purpose of improving the performance of quantization, a compression method based on the activation transform may be included in a combination prior to a compression method based on quantization.

Thereafter, the parameter selector 510 may select a combination of values of at least some of the parameters of the selected at least one compression method as the parameter combination. That is, the parameter combination may consist of selected values for at least some of the parameters according to the selected compression methods. As an example, a compression method based on pruning may include a pruning ratio (PR) as a parameter. In this case, the parameter selector 520 may select a value of the parameter PR from a set of previously prepared values (eg, select one from a set of values of the parameter PR {0.3, 0.4, 0.5}). In addition, in the compression method based on filter decomposition, the number of ranks is selected from a set of values prepared in advance (for example, one of the set of values of the parameter filter rank (Filter Rank, FR) {2, 3, 4} can be selected). In other words, if the selected compression methods are the first compression method based on pruning and the second compression method based on filter decomposition, the parameter selector 520 sets [PR 0.4, FR 2] as parameter values for the selected compression methods. ] or [PR 0.3, FR 4] can be selected.

Also, techniques such as hyperband or Bayesian optimization may be utilized to select a combination of values of the parameters. These techniques may later be utilized to select an optimal parameter combination according to multiple capabilities.

In addition, as already described, multiple compressed inference models may be generated in parallel by applying different combinations of compression methods to one inference model in parallel. As an example, it has been described that the HPO 310 can simultaneously generate a plurality of lightweight models by operating a plurality of compression pipelines. In this case, a performance value obtained for an inference model compressed through one compression pipeline may be used to select a parameter combination for another compression pipeline. As an example, it is assumed that the HPO 310 obtains a performance value of 1 after generating a first compressed inference model using a combination of compression methods a and a combination of parameter values b in the compression pipeline A. In addition, it is assumed that a performance value of 2 is obtained after the HPO 310 generates a second compressed inference model using a combination of compression methods c and a combination of parameters d in the compression pipeline B operated in parallel. At this time, when the HPO 310 selects a combination of parameter values for a combination of compression methods a in the compression pipeline B, a new parameter for a combination of compression methods a takes into account the combination of parameter values b and the performance value 1. A combination e of the values of can be selected. As a more specific example, when the performance value 1 is relatively high (for example, when the performance value 1 is higher than the performance value for the performance value 2 or the combination of other compression methods), the HPO 310 is a combination of the values of the parameters b. It is possible to select a combination e of new parameter values by changing only a part in . On the other hand, when the performance value 1 is relatively low in the HPO 310 (for example, when the performance value 1 is lower than the performance value for the performance value 2 or the combination of other compression methods), the total value of the combination of parameter values b It is possible to select a combination e of values of new parameters by changing .

In step 620, the model weight reduction unit 520 may compress the input inference model using the selected parameter combination. For example, the model weight reduction unit 520 may compress the input inference model by sequentially applying the plurality of compression methods selected in step 610 . At this time, the model weight reduction unit 520 may compress the inference model by using a value of a parameter selected for an applied compression method. In other words, the model weight reduction unit 520 may generate a compressed inference model by sequentially applying compression methods to which selected parameter values are applied to the inference model.

In step 630, the model transmitter 530 may transmit the compressed inference model to the target device. Here, the target device may correspond to the target device 350 selected from the target device pool 320 . The target device may then measure the performance of the compressed inference model and deliver it to the HPO 310 as described with reference to FIGS. 7 and 8 .

In step 640, the result receiver 540 may receive the performance of the compressed inference model from the target device. For example, the received performance may include latency and accuracy measured by the target device inputting data into a compressed inference model.

In step 650, the repetition determination unit 550 may determine whether to repeatedly perform steps 610 to 640 based on the received performance. In this case, the repetition determination unit 550 may determine whether or not to repeat depending on whether the received performance satisfies an input (or set) constraint for the inference model. For example, if the received performance satisfies the input (or set) constraint, the iteration determiner 550 does not need to repeat steps 610 to 640, and the parameters used for compression of the inference model The combination can be provided as a parameter combination optimized for the target device. In addition, the compressed inference model may be provided as a final lightweight inference model for the target device. Conversely, if the received performance does not satisfy the input (or set) constraint, the iterative determination unit 550 may search for an optimal parameter combination again by repeatedly performing steps 610 to 640. . At this time, in step 610, the same combination of compression methods may be selected or a different combination of compression methods may be selected. If the same combination of compression methods is selected in step 610, different values of parameters corresponding to the compression methods of the corresponding combination may be selected in step 620. Conversely, if another combination of compression methods is selected at step 610, then other parameter values corresponding to the different combination of compression methods may be selected at step 620. In other words, if the combination of compression methods is changed, the combination of parameters may also be changed, and thus the values of the changed parameters may be selected. If the combination of compression methods is maintained, the combination of parameters may also be maintained, in which case only values for the same parameters may be changed and selected.

Meanwhile, according to embodiments, the repetition determination unit 550 may cause steps 610 to 640 to be repeatedly performed a predetermined number of times. To this end, the repetition determination unit 550 may compare the number of repetitions with a predetermined number of repetitions, and perform steps 610 to 640 repeatedly until the number of repetitions exceeds the predetermined number. In this case, a combination of compression methods and/or a combination of parameter values may be changed for each iteration, so that the performance of a lightweight inference model can be measured for each combination of various compression methods and/or parameter values.

In step 660, the optimal parameter determination unit 560 may determine an optimal parameter combination according to the received performance. For example, when a plurality of capabilities are received from the target device as steps 610 to 640 are repeatedly performed, a parameter combination that yields the best performance may be determined as an optimal parameter combination for the target device. As already described, a parameter combination may mean a combination of values of parameters according to a combination of compression methods.

7 is a block diagram illustrating an example of an internal configuration of a target device according to an embodiment of the present invention, and FIG. 8 is a flowchart illustrating an example of a method for determining optimal parameters of a target device according to an embodiment of the present invention. . The target device 350 described above may also be implemented by the computer device 200 . The target device 350 according to this embodiment may include a model receiver 710, a model test unit 720, and a result transmitter 730. At this time, the model receiving unit 710, the model testing unit 720, and the result transmitting unit 730 are functions of the processor 220 of the computer apparatus 200 implementing the target device 350 operating under the control of a computer program. It can be a functional expression. For example, the processor 220 of the computer device 200 may be implemented to execute a control instruction according to an operating system code or at least one computer program code included in the memory 210 . Here, the processor 220 controls the computer device 200 so that the computer device 200 performs steps 810 to 830 included in the method of FIG. 8 according to a control command provided by a code stored in the computer device 200. can control. At this time, the model receiver 710, the model test unit 720, and the result transmitter 730 may be used as functional representations of the processor 220 for performing each of the steps 810 to 830.

In step 810, the model receiver 710 may receive the inference model compressed by the HPO 310. At this time, the model receiver 710 may receive the data set (data, label) input together with the inference model to the HPO 310 .

In step 820, the model testing unit 720 may test the compressed inference model. In this case, the model testing unit 720 may generate at least one of latency and accuracy as a test result of the compressed inference model. For example, the model test unit 720 may input data of a dataset into a compressed inference model, and delays based on the time the data is input and the time the compressed inference model outputs a result for the input data. time can be measured. As another example, the model test unit 720 may measure the accuracy of the compressed inference model by comparing the output result with a label that is a correct answer to the data.

In step 830, the result transmission unit 730 may transmit the test result to the HPO 310 as the performance of the compressed inference model. At this time, the result receiver 540 of the HPO 310 may receive performance of the inference model, and the HPO 310 may determine an optimal parameter combination based on the performance received from the target device.

As described above, according to embodiments of the present invention, an inference model may be compressed and provided to a target device, and an optimal parameter combination may be determined based on performance of the compressed inference model provided from the target device.

The system or device described above may be implemented as a hardware component or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. can be embodied in Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The medium may continuously store programs executable by a computer or temporarily store them for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or combined hardware, but is not limited to a medium directly connected to a certain computer system, and may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, and ROM, RAM, flash memory, etc. configured to store program instructions. In addition, examples of other media include recording media or storage media managed by an app store that distributes applications, a site that supplies or distributes various other software, and a server. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (18)

An optimal parameter determination method performed by a computer device including at least one processor,
Receiving an inference model, data set, and constraints;
selecting a compression method combination to be applied to the inference model and a parameter combination for the compression method combination in consideration of the constraint;
applying a first compression method included in the selected compression method combination and a first parameter related to the first compression method to the inference model through a compression pipeline;
determining whether a compressed inference model is generated from the inference model through the compression pipeline;
If the compressed inference model is not generated, a second compression method included in a combination of a second parameter among the selected parameter combinations and the selected compression method is followed by the first compression method through the compression pipeline, followed by the inference. applying to the model;
If the compressed inference model is generated, transmitting the compressed inference model to the target device; and
Determining the optimal parameter combination based on the performance of the compressed inference model received from the target device,
The performance of the compressed inference model is measured by the target device using the dataset. According to claim 1,
Selecting the compression method combination and the parameter combination comprises:
selecting the first compression method and the first parameter for the first compression method;
determining whether to further select a compression method and parameters based on the constraint and the first compression method; and
if it is determined to further select a compression method and parameters, selecting the second compression method and the second parameter for the second compression method in consideration of the first compression method;
Including, method. According to claim 1,
When performance of the condensed inference model satisfies the constraint, the selected parameter combination is determined as the optimal parameter combination;
When performance of the compressed inference model does not satisfy the constraint, selecting the compression method combination and the parameter combination, applying the first compression method and the first parameter to the inference model, and the compression method. Determining whether the generated inference model is generated, applying the second compression method and the second parameter to the inference model, transmitting the compressed inference model to the target device, and determining the optimal parameter combination. Characterized in that the step of doing is repeatedly performed, the method. According to claim 1,
The constraints are
A method comprising a value for at least one of device, accuracy, model size, latency, compression time, or energy consumption. According to claim 1,
Priorities are assigned to the constraints,
Determining the optimal parameter combination,
Determining whether or not the compressed inference model satisfies the constraint based on the priority based on the performance of the compressed inference model or more than a predetermined criterion
Including, method. According to claim 1,
The compression method combination,
A method, selected from a pool of compression methods including Pruning, Filter Decomposition, Quantization, and Knowledge Distillation. According to claim 1,
The target device is implemented to measure the performance of the compressed inference model,
Wherein the performance of the condensed inference model includes at least one of latency or accuracy. According to claim 1,
The method of claim 1 , wherein the target device is selected from a target device pool based on the constraint. A computer program stored in a computer readable recording medium to be combined with a computer device to execute the method of any one of claims 1 to 8 on the computer device. A computer readable recording medium on which a program for executing the method of any one of claims 1 to 8 is recorded in a computer device. In a computer device,
a memory storing at least one instruction; and
a processor operatively coupled to the memory;
The processor, by executing the at least one instruction,
Inference model, dataset and constraints are input,
Considering the constraints, selecting a compression method combination to be applied to the inference model and a parameter combination for the compression method combination;
Applying a first compression method included in the selected compression method combination and a first parameter related to the first compression method to the inference model through a compression pipeline;
Determining whether a compressed inference model has been generated from the inference model through the compression pipeline;
If the compressed inference model is not generated, a second compression method included in a combination of a second parameter among the selected parameter combinations and the selected compression method is followed by the first compression method through the compression pipeline, followed by the inference. applied to the model,
If the compressed inference model is generated, transmit the compressed inference model to the target device;
Based on the performance of the compressed inference model received from the target device, determining the optimal parameter combination;
Performance of the compressed inference model is measured by the target device using the dataset. According to claim 11,
the processor,
select the first compression method and the first parameter for the first compression method;
determine whether to further select a compression method and parameters based on the constraint and the first compression method;
and selecting the second compression method and the second parameter for the second compression method in view of the first compression method if it is determined to further select a compression method and parameters. According to claim 11,
the processor,
When performance of the condensed inference model satisfies the constraint, determining the selected parameter combination as the optimal parameter combination;
When the performance of the compressed inference model does not satisfy the constraint, selecting the compression method combination and the parameter combination, applying the first compression method and the first parameter to the inference model, and performing the compression An operation of determining whether the generated inference model is generated, an operation of applying the second compression method and the second parameter to the inference model, an operation of transmitting the compressed inference model to the target device, and an operation of determining the optimal parameter combination. A computer device, characterized in that repeatedly performing the operation. According to claim 11,
The constraints are
A computer device including a value for at least one item of a device, accuracy, model size, latency, compression time, or energy consumption. According to claim 11,
Priorities are assigned to the constraints,
the processor,
Based on the performance of the compressed inference model, it is determined whether the compressed inference model satisfies the constraint based on the priority at least a predetermined criterion. According to claim 11,
The compression method combination,
A computational device, selected from a pool of compression methods including Pruning, Filter Decomposition, Quantization, and Knowledge Distillation. According to claim 11,
The target device is implemented to measure the performance of the compressed inference model,
Wherein the performance of the condensed inference model includes at least one of latency or accuracy. According to claim 11,
wherein the target device is selected from a target device pool based on the constraint.

Images