KR102613367B1 - Method and apparatus for automatically reducing model weight for deep learning model serving optimization, and a method for providing cloud inference services usin the same - Google Patents

Abstract

The present invention relates to a method and device for automatically lightweighting a model for optimizing deep learning model serving, and a method for providing a cloud inference service using the same, the device comprising: receiving a deep learning algorithm for building a deep learning model; Dividing the deep learning algorithm into a plurality of operation steps; determining at least one branch point that exists between the plurality of operation steps in a learning process according to the deep learning algorithm; generating at least one intermediate deep learning model that branches off from the direction of the learning process based on the at least one branch point and proceeds to a final operation step of the deep learning algorithm; and completing the deep learning model and the at least one intermediate deep learning model upon completion of the learning process.

Description

Method and device for automatically reducing model weight for optimizing deep learning model serving, and method for providing cloud inference services using the same

The present invention relates to deep learning model generation technology, and more specifically, a method for automatically lightweighting models for optimizing deep learning model serving that can automatically generate deep learning models at various levels to smoothly provide deep learning model inference services, and This relates to devices and methods of providing cloud inference services using them.

Recently, as deep learning techniques have been applied to various applications, the number of cases of using the algorithm in embedded environments is increasing. In general, deep learning models with higher accuracy may require a longer time to produce results. However, the embedded environment has constraints on time and energy, and existing neural networks have the problem of not being able to dynamically cope with constraints.

This situation can equally apply to a cloud computing environment. Currently, due to the nature of deep learning tasks that require large-scale computing resources, much work is being done in cloud computing environments for both learning and inference services. In the case of deep learning inference services, it can be responsible for providing prediction services according to user requests by utilizing the created model.

Unlike learning services, inference services may have the characteristic of providing scalable services depending on the number of user requests. In the case of deep learning inference models, models with complex structures have the characteristic of having high accuracy while taking long inference times, while models with simple structures may have the disadvantage of having short inference times but low accuracy. General deep learning algorithm developers put a lot of effort into creating models with high accuracy.

However, in the case of providing an inference service for an actual deep learning model, when user requests increase rapidly, a model that provides a fast inference time even if there is a slight loss in accuracy is better to apply to the application service than a model that has high accuracy and takes a long time to calculate. may be more effective.

Korean Patent No. 10-0820723 (2008.04.02)

An embodiment of the present invention is a method and device for automatic model lightweighting for optimizing deep learning model serving that can automatically generate deep learning models at various levels to smoothly provide deep learning model inference services, and a method for providing cloud inference services using the same. We would like to provide.

One embodiment of the present invention provides deep learning model serving optimization that allows deep learning model development researchers to generate various deep learning prediction models with different prediction accuracy and model complexity from the model while developing a complex model with high accuracy. We aim to provide a method and device for automatic model lightweighting and a method for providing cloud inference services using the same.

Among embodiments, a method for automatic model lightweighting for deep learning model serving optimization includes receiving a deep learning algorithm for building a deep learning model; Dividing the deep learning algorithm into a plurality of operation steps; determining at least one branch point that exists between the plurality of operation steps in a learning process according to the deep learning algorithm; generating at least one intermediate deep learning model that branches off from the direction of the learning process based on the at least one branch point and proceeds to a final operation step of the deep learning algorithm; and completing the deep learning model and the at least one intermediate deep learning model upon completion of the learning process.

The deep learning algorithm may include a Deep Neural Network (DNN), a Convolution Neural Network (CNN), and a Recurrent Neural Network (RNN).

The deep learning model and the at least one intermediate deep learning model may have different prediction accuracy and calculation speed, respectively.

The dividing into a plurality of operation steps includes dividing the operations of the deep learning algorithm into a plurality of layers; and determining operation steps corresponding to each of the plurality of layers.

The dividing into a plurality of operation steps includes determining a repetition section that is repeatedly performed during the operation of the deep learning algorithm; determining a pre-repetition section and a post-repetition section based on the repeat section; determining at least one unit section for the repetition section; and arranging the pre-repetition section, the at least one unit section, and the post-repetition section in order to determine the plurality of operation steps.

The step of determining at least one branch point may include determining each point at which the repetition section ends as a branch point.

The step of determining the at least one branch point is performed by selecting the end point of the repetition section that sequentially progresses through at least one convolution layer and a pooling layer as the branch point. It may include a step of deciding.

Generating the at least one intermediate deep learning model may include defining a candidate intermediate deep learning model according to the branch; calculating the adequacy of the corresponding branch point based on the number of layers (L) and prediction accuracy (A) of the candidate intermediate deep learning model; and determining the candidate intermediate deep learning model as an intermediate deep learning model when the appropriateness of the corresponding branch point satisfies a preset condition.

The step of calculating the adequacy includes calculating the adequacy through a product operation (A*L) between the number of layers (L) and the prediction accuracy (A), and the step of determining the intermediate deep learning model is If the adequacy is twice that of the intermediate deep learning model generated in the previous step, the step of determining the candidate intermediate deep learning model as the intermediate deep learning model may be included.

Among embodiments, an automatic model lightweighting device for optimizing deep learning model serving includes an algorithm receiver that receives a deep learning algorithm for building a deep learning model; an algorithm analysis unit that divides the deep learning algorithm into a plurality of operation steps; a branch point determination unit that determines at least one branch point that exists between the plurality of operation steps in a learning process according to the deep learning algorithm; an intermediate model generator that generates at least one intermediate deep learning model that branches off from the direction of the learning process based on the at least one branch point and proceeds to a final operation step of the deep learning algorithm; and a deep learning model construction unit that completes the deep learning model and the at least one intermediate deep learning model upon completion of the learning process.

The intermediate model generator defines a candidate intermediate deep learning model according to the branch, calculates the adequacy of the branch point based on the number of layers (L) and prediction accuracy (A) of the candidate intermediate deep learning model, and calculates the adequacy of the branch point. If the adequacy of satisfies preset conditions, the candidate intermediate deep learning model can be determined as the intermediate deep learning model.

The intermediate model generator calculates the adequacy through a product operation (A*L) between the number of layers (L) and the prediction accuracy (A), and when the adequacy is doubled compared to the intermediate deep learning model generated in the previous step, The candidate intermediate deep learning model may be determined as the intermediate deep learning model.

Among embodiments, a method of providing a cloud inference service using an automatic model lightweighting method for optimizing deep learning model serving includes receiving a request for an inference service from a user terminal; determining the status of available cloud resources based on the time of receipt of the request, predicting a response generation time for the request, and determining one of the plurality of deep learning models according to the response generation time; It includes generating a response to the request using the determined deep learning model and providing it to the user terminal.

The plurality of deep learning models may be generated to have different prediction accuracy and calculation speed based on an automatic model lightweighting method for optimizing deep learning model serving.

The disclosed technology can have the following effects. However, since it does not mean that a specific embodiment must include all of the following effects or only the following effects, the scope of rights of the disclosed technology should not be understood as being limited thereby.

The method and device for automatically lightweighting models for optimizing deep learning model serving according to an embodiment of the present invention, and the method for providing cloud inference services using the same, automatically generate deep learning models at various levels to smoothly provide deep learning model inference services. You can.

A method and device for automatically lightweighting a model for optimizing deep learning model serving according to an embodiment of the present invention, and a method for providing a cloud inference service using the same, allow deep learning model development researchers to You can create various deep learning prediction models with different prediction accuracy and model complexity.

1 is a diagram illustrating an automatic model weight reduction system according to the present invention.
FIG. 2 is a diagram explaining the system configuration of the model automatic weight reduction device of FIG. 1.
FIG. 3 is a diagram explaining the functional configuration of the model automatic weight reduction device of FIG. 1.
Figure 4 is a flowchart explaining the automatic model lightweighting method for optimizing deep learning model serving according to the present invention.
Figure 5 is a flowchart explaining a method of providing a cloud inference service according to the present invention.
Figure 6 is a diagram explaining the layer configuration of CNN.
7 and 8 are diagrams illustrating an embodiment of the automatic model weight reduction method according to the present invention.

Since the description of the present invention is only an example for structural or functional explanation, the scope of the present invention should not be construed as limited by the examples described in the text. In other words, since the embodiments can be modified in various ways and can have various forms, the scope of rights of the present invention should be understood to include equivalents that can realize the technical idea. In addition, the purpose or effect presented in the present invention does not mean that a specific embodiment must include all or only such effects, so the scope of the present invention should not be understood as limited thereby.

Meanwhile, the meaning of the terms described in this application should be understood as follows.

Terms such as “first” and “second” are used to distinguish one component from another component, and the scope of rights should not be limited by these terms. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected to the other component, but that other components may exist in between. On the other hand, when a component is referred to as being “directly connected” to another component, it should be understood that there are no other components in between. Meanwhile, other expressions that describe the relationship between components, such as "between" and "immediately between" or "neighboring" and "directly neighboring" should be interpreted similarly.

Singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as “comprise” or “have” refer to implemented features, numbers, steps, operations, components, parts, or them. It is intended to specify the existence of a combination, and should be understood as not excluding in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

For each step, identification codes (e.g., a, b, c, etc.) are used for convenience of explanation. The identification codes do not explain the order of each step, and each step clearly follows a specific order in context. Unless specified, events may occur differently from the specified order. That is, each step may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the opposite order.

The present invention can be implemented as computer-readable code on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. . Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Additionally, the computer-readable recording medium can be distributed across computer systems connected to a network, so that computer-readable code can be stored and executed in a distributed manner.

All terms used herein, unless otherwise defined, have the same meaning as commonly understood by a person of ordinary skill in the field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as consistent with the meaning they have in the context of the related technology, and cannot be interpreted as having an ideal or excessively formal meaning unless clearly defined in the present application.

1 is a diagram illustrating an automatic model weight reduction system according to the present invention.

Referring to FIG. 1, the automatic model weight reduction system 100 may include a user terminal 110, an automatic model weight reduction device 130, and a database 150.

The user terminal 110 may correspond to a computing device operated by the user. That is, the user may request an inference service from the automatic model lightweighting device 130 through the user terminal 110 for applications such as image classification and natural language processing. The user terminal 110 may be implemented as a smartphone, a laptop, or a computer, but is not necessarily limited thereto, and may also be implemented as a variety of devices such as a tablet PC.

Additionally, the user terminal 110 may be connected to the automatic model weight reduction device 130 through a network, and a plurality of user terminals 110 may be connected to the automatic model weight reduction device 130 at the same time. Additionally, the user terminal 110 may install and execute a dedicated program or application for linking with the automatic model weight reduction device 130.

The automatic model lightweighting device 130 may be implemented as a server corresponding to a computer or program that can build a deep learning model by performing learning based on a deep learning algorithm set by the user. The automatic model lightweighting device 130 performs an operation of automatically generating low-accuracy simple models from a high-accuracy complex model without user intervention in the process of building a deep learning model corresponding to the deep learning algorithm requested by the user. You can. The automatic model lightweight device 130 may be connected to the user terminal 110 through a wired network or a wireless network such as Bluetooth, WiFi, or LTE, and may transmit and receive data with the user terminal 110 through the network.

Additionally, the automatic model weight reduction device 130 may operate in conjunction with an external system (not shown in FIG. 1) to collect data or provide additional functions. For example, the external system may include a cloud server that provides cloud services. In this case, the automatic model lightweighting device 130 may process the inference service requested by the user in conjunction with the cloud server. That is, the automatic model lightweighting device 130 may query the cloud server for the user's inference service request, receive a response, and provide the response to the user.

Meanwhile, the automatic model lightweight device 130 may be implemented by being included in a cloud server. In this case, when providing a cloud inference service, the automatic model lightweight device 130 can build multiple deep learning models with various accuracy and complexity based on one deep learning model, and based on this, it can provide information on user requests. Inference can be performed efficiently to provide service responses.

The database 150 may correspond to a storage device that stores various information required during the operation of the automatic model lightweight device 130. The database 150 can store deep learning algorithms and learning data, and can store deep learning models built through learning, but is not necessarily limited thereto, and the automatic model lightweighting device 130 is used to optimize deep learning model serving. In the process of performing the automatic model lightweighting method, information collected or processed can be stored in various forms.

FIG. 2 is a diagram explaining the system configuration of the model automatic weight reduction device of FIG. 1.

Referring to FIG. 2, the automatic model weight reduction device 130 may be implemented including a processor 210, a memory 230, a user input/output unit 250, and a network input/output unit 270.

The processor 210 can execute a procedure that processes each step in the process of operating the automatic model lightweight device 130, and can manage the memory 230 that is read or written throughout the process, and the memory ( 230), the synchronization time between the volatile memory and the non-volatile memory can be scheduled. The processor 210 can control the overall operation of the automatic model lightweight device 130 and is electrically connected to the memory 230, the user input/output unit 250, and the network input/output unit 270 to control data flow between them. can do. The processor 210 may be implemented as a CPU (Central Processing Unit) of the automatic model lightweight device 130.

The memory 230 may be implemented as a non-volatile memory such as a solid state drive (SSD) or a hard disk drive (HDD) and may include an auxiliary memory used to store all data required for the model automatic lightweight device 130. , may include a main memory implemented as volatile memory such as RAM (Random Access Memory).

The user input/output unit 250 may include an environment for receiving user input and an environment for outputting specific information to the user. For example, the user input/output unit 250 may include an input device including an adapter such as a touch pad, touch screen, on-screen keyboard, or pointing device, and an output device including an adapter such as a monitor or touch screen. In one embodiment, the user input/output unit 250 may correspond to a computing device connected through a remote connection, and in such case, the automatic model lightweight device 130 may be performed as an independent server.

The network input/output unit 270 includes an environment for connecting with external devices or systems through a network, for example, Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), and VAN ( It may include an adapter for communication such as a Value Added Network).

FIG. 3 is a diagram explaining the functional configuration of the model automatic weight reduction device of FIG. 1.

Referring to FIG. 3, the automatic model lightweighting device 130 includes an algorithm receiver 310, an algorithm analysis unit 320, a branch point determination unit 330, an intermediate model generation unit 340, and a deep learning model construction unit 350. ) and a control unit 360.

The algorithm receiver 310 may receive a deep learning algorithm for building a deep learning model. At this time, the deep learning algorithm may include Deep Neural Network (DNN), Convolution Neural Network (CNN), and Recurrent Neural Network (RNN). The algorithm receiver 310 can store a deep learning algorithm through the database 150, and can also process reception operations related to the deep learning algorithm by receiving only selection information about a specific deep learning algorithm from the user terminal 110. . Meanwhile, deep learning algorithms exist in numerous modified algorithms based on the representative algorithms above, and the algorithm receiver 310 receives settings related to the deep learning algorithm from the user terminal 110 and performs the operation steps of the deep learning algorithm. And the operation sequence can be specifically specified.

The algorithm analysis unit 320 may divide the deep learning algorithm into a plurality of operation steps. A deep learning algorithm can be defined to perform a learning process by cumulatively updating weights at each stage in the process of sequentially passing learning data through a plurality of operation stages. The algorithm analysis unit 320 may analyze the deep learning algorithm selected or input by the user and define independent operation steps based on the end points of unit operations during the operation process.

In one embodiment, the algorithm analysis unit 320 divides the operations of the deep learning algorithm into a plurality of layers and determines operation steps corresponding to each of the plurality of layers, thereby dividing them into a plurality of operation steps. You can. Deep learning algorithms can be defined as a set of unit operations divided into layers. For example, CNN, one of the image classification deep learning models, consists of an image processing layer (Convolution layer), Fully-Connected layer, Activation layer, and Softmax layer. You can. The algorithm analysis unit 320 can divide the operations of the deep learning algorithm into a plurality of layers based on the basic layer of the deep learning algorithm and specific parameters set by the user, and determine a plurality of operation steps based on this. . At this time, the connection point between independent operation steps can be used as a branching point to create an intermediate deep learning model in a later step.

In one embodiment, the algorithm analysis unit 320 may determine a repetition section that is repeatedly performed during the operation of a deep learning algorithm and determine a pre-repetition section and a post-repetition section based on the repetition section. In addition, the algorithm analysis unit 320 may divide the repetition section into at least one unit section, and arrange a pre-repetition section, at least one unit section, and a post-repetition section according to the operation order of the deep learning algorithm to perform a plurality of operations. It can be decided in steps. In a deep learning model, the performance (e.g., accuracy) of the final built model can be determined according to the learning process of repeatedly stacking various layers. Accordingly, the algorithm analysis unit 320 may determine a plurality of operation steps related to the deep learning algorithm based on the start and end points of the repetition section.

Meanwhile, the repetition section of the deep learning algorithm can be implemented so that the same unit sections proceed continuously depending on the settings. For example, in the case of the VGG model (VGGNet), the first unit section consisting of two convolution operations and one maxpooling operation may be performed repeatedly, and three convolution operations and one maxpooling operation may be performed repeatedly. The second unit section consisting of the max pooling operation may be performed repeatedly. The algorithm analysis unit 320 can divide the repetition section into unit sections and determine independent operation steps for each section.

The branch point determination unit 330 may determine at least one branch point that exists between a plurality of operation steps in a learning process according to a deep learning algorithm. Here, the branch point is a point between the operation steps that constitute the deep learning algorithm and may correspond to a point that independently branches off in the direction of progress of the deep learning model in order to create an intermediate deep learning model. Therefore, the intermediate deep learning model created by branching from the branch point has lower accuracy compared to the original deep learning model, but the computation speed can increase as complexity is reduced. The branching point determination unit 330 may determine the branching point so that the accuracy of the intermediate deep learning models generated together with the original deep learning model can be evenly distributed.

In one embodiment, the branch point determination unit 330 may determine each point where a repetition section ends as a branch point. Each time the repetition section of the deep learning algorithm is repeated, model performance due to cumulative learning increases, while complexity also increases and computation speed may decrease. Accordingly, the branch point determination unit 330 may determine the branch point based on the end point of the repetition section. However, the performance of the intermediate deep learning model may not change linearly at the end point of the repetition section even though the operation steps are equally accumulated.

In one embodiment, when the deep learning algorithm is CNN, the branch point determination unit 330 sets the end point of the repetition section that sequentially progresses through at least one convolution layer and a pooling layer as the branch point. can be decided. CNN may be composed of a convolutional layer, a pooling layer, and a fully-connected layer, and the convolutional layer and the pooling layer can be performed repeatedly a predetermined number of times as a feature extraction step. The fully connected layer can be performed in a single operation as a classification step. In the case of CNN, the branch point decision unit 330 can determine the branch point based on the end point of the pooling layer, and accordingly, the fully connected layer is simply combined at the branch point, so that the difference is only in the number of repetitions based on the same operation structure. Multiple intermediate deep learning models can be created.

The intermediate model generator 340 may branch from the direction of the learning process based on at least one branch point and generate at least one intermediate deep learning model that proceeds to the final operation step of the deep learning algorithm. In other words, the intermediate deep learning model may correspond to a deep learning model created by omitting part of the learning process based on the original deep learning model. Therefore, the complexity of the intermediate deep learning model is reduced compared to the original deep learning model, which increases computational speed, but prediction performance (i.e., accuracy) may decrease. The intermediate model generator 340 may generate an intermediate deep learning model that proceeds directly to the final operation step without performing an additional learning process at the branch point. Meanwhile, the last operation step may be configured identically to the last operation step of the original deep learning model, or may be composed of only a part of the last operation step, if necessary.

In one embodiment, the intermediate model generator 340 defines a candidate intermediate deep learning model according to the branch and determines the appropriateness of the branch point based on the number of layers (L) and prediction accuracy (A) of the candidate intermediate deep learning model. If the appropriateness of the corresponding branch point satisfies preset conditions, the candidate intermediate deep learning model can be determined as the intermediate deep learning model. Here, the number of layers (L) may correspond to the total number of layers constituting the deep learning model, and the prediction accuracy (A) may correspond to the performance index of the generated deep learning model and that of the original deep learning model. Performance can be assumed to be 100% and expressed as a relative value.

That is, the intermediate model generator 340 may generate an intermediate deep learning model by selecting an appropriate branch point instead of generating an intermediate deep learning model for all branch points. If too many intermediate deep learning models are created, problems may arise in having to manage multiple models with similar performance, and if the number of intermediate deep learning models is too small, there may be differences in calculation speed and accuracy between each intermediate deep learning model. As it grows, it may act as a limitation when providing inference services using it.

In one embodiment, the intermediate model generator 340 calculates the adequacy of the branch point through a product operation (A*L) between the number of layers (L) and the prediction accuracy (A), and determines the adequacy of the intermediate dip generated in the previous step. If the number is twice that of the learning model, the candidate intermediate deep learning model can be determined as the intermediate deep learning model. That is, the intermediate model generator 340 can calculate the A*L value as the appropriateness of the branch point and generate an intermediate deep learning model each time the value increases by twice the initial value. As a result, more intermediate deep learning models can be created because the accuracy increases rapidly in the early stages of modeling, and as the layers become deeper in the later stages of modeling, but accuracy does not improve significantly, relatively fewer intermediate deep learning models are created. It can be.

The deep learning model construction unit 350 may complete the deep learning model and at least one intermediate deep learning model upon completion of the learning process. The deep learning model building unit 350 may perform learning on predetermined learning data according to a deep learning algorithm, and when learning on all learning data is completed, the original deep learning model and at least one generated based on it. An intermediate deep learning model can be completed. That is, the deep learning model building unit 350 can automatically build a plurality of deep learning models based on one deep learning algorithm and a learning data set.

At this time, the plurality of deep learning models created may have different prediction accuracy and calculation speed, and the deep learning model can be selectively applied depending on the operating conditions of the system during the performance of the inference service using this. For example, if service requests explode while providing inference services, a faster response can be provided by utilizing a lightweight deep learning model with fast computation. If there are few service requests, a more accurate response can be provided by utilizing a highly complex deep learning model. can do.

The control unit 360 controls the overall operation of the automatic model weight reduction device 130, and includes an algorithm receiver 310, an algorithm analysis unit 320, a branch point determination unit 330, an intermediate model generation unit 340, and a deep learning unit. Control flow or data flow between the model building units 350 can be managed.

Figure 4 is a flowchart explaining the automatic model lightweighting method for optimizing deep learning model serving according to the present invention.

Referring to FIG. 4, the automatic model weight reduction device 130 may receive a deep learning algorithm for building a deep learning model from the user terminal 110 through the algorithm receiver 310 (step S410). The automatic model weight reduction device 130 may divide the deep learning algorithm into a plurality of operation steps through the algorithm analysis unit 320 (step S430). The automatic model weight reduction device 130 may determine at least one branch point that exists between a plurality of operation steps in a learning process according to a deep learning algorithm through the branch point determination unit 330 (step S450).

In addition, the automatic model lightweighting device 130 branches from the direction of the learning process based on at least one branch point through the intermediate model generator 340 and creates at least one intermediate deep learning process that proceeds to the final operation step of the deep learning algorithm. A learning model can be created (step S470). The automatic model lightweighting device 130 may complete a deep learning model and at least one intermediate deep learning model upon completion of the learning process through the deep learning model building unit 350 (step S490).

Figure 5 is a flowchart explaining a method of providing a cloud inference service according to the present invention.

Referring to Figure 5, using the automatic model lightweighting method for optimizing deep learning model serving according to the present invention, multiple deep learning models with different prediction accuracy and calculation speed can be created, and cloud inference services are provided by utilizing this. can do.

More specifically, in step S510, the method for providing a cloud inference service according to the present invention may receive a request for an inference service from the user terminal 110. If the cloud inference service providing device is implemented independently from the automatic model lightening device according to the present invention, the automatic model lightening device may receive the request and forward it to the cloud inference service providing device.

In step S530), the cloud inference service provision method may determine one of a plurality of previously constructed deep learning models. At this time, the cloud inference service provision method determines the status of available cloud resources based on the time of receipt of the request, predicts the response generation time to the request, and determines one of a plurality of deep learning models according to the response generation time. .

For example, if it is predicted that it will take 10 seconds to generate a response to a service request using the original deep learning model based on the current cloud resources available, a deep learning model with low complexity even if the accuracy is somewhat low is used. You can select to generate a response faster than 10 seconds.

In step S550, the cloud inference service providing method may generate a response to the user's service request using a predetermined deep learning model and provide the response to the user through the user terminal 110.

Figure 6 is a diagram explaining the layer configuration of CNN.

Referring to FIG. 6, a Convolution Neural Network (CNN) may be composed of an image processing layer, a fully connected layer, an activation layer, etc.

The automatic model weight reduction device 130 can analyze the deep learning algorithm and divide it into a plurality of operation steps, and divide the operation steps based on the repetition section defined in the deep learning algorithm.

In Figure 6, in the case of CNN, step S630, in which convolutional layers are repeatedly performed, may correspond to a repetition section. The automatic model weight reduction device 130 may determine the pre-repetition section (S610) and the post-repetition section (S630) based on the corresponding repeat section (S630). In the case of CNN, the section before repetition (S610) may correspond to the input reception step, and the section after repetition (S610) may correspond to the output generation step. At this time, the output generation step may include a fully connected layer (Dense Layer) and an activation layer (Activation Layer), and may be repeated as necessary.

Additionally, the automatic model weight reduction device 130 may divide the repetition section S630 into a plurality of unit sections and determine the end points of the unit sections as branch points to generate an intermediate deep learning model. That is, the point between the convolutional layers in FIG. 6 may be determined as the branching point.

7 and 8 are diagrams illustrating an embodiment of the automatic model weight reduction method according to the present invention.

Referring to FIG. 7, the automatic model lightweighting device 130 may receive one deep learning algorithm from the user and build a deep learning model according to the original learning direction 750. The automatic model lightening device 130 may determine at least one branch point 740 in the learning process and define a branched learning direction 760 that branches off from the original learning direction 750 based on the branch point. there is. In other words, the automatic model lightweighting device 130 can generate an intermediate deep learning model that performs the learning end step (S730) immediately at the branch point 740 according to the branched learning direction 760. The intermediate deep learning model created in this way has a simpler structure than the original learning direction 750, so it may have a short inference time but low accuracy.

Meanwhile, in the case of the VGG model, the learning end step (S730) may consist of a flatten layer, a dense layer, a dropout layer, and an activation layer, but is not necessarily limited to this. Of course, it can be applied selectively as needed. Additionally, the dense layer and dropout layer may be performed repeatedly.

Additionally, the automatic model lightweighting device 130 may determine the branching point 740 between operation steps of the original learning process in order to generate an intermediate deep learning model. At this time, the automatic model weight reduction device 130 may determine a repetition section in which predetermined operation steps are repeatedly performed and determine the end point of the repetition section as a branch point. In FIG. 7, steps S710 and S720 are sections in which two convolution operations (Conv1 and Conv2) and one pooling operation (Pool) are sequentially performed, respectively, and may correspond to a repetition section. That is, the automatic model lightweighting device 130 may determine the point between step S710 and step S720 as the branch point 740 and generate an intermediate deep learning model branching from this point.

Referring to FIG. 8, the automatic model lightweighting device 130 may generate an intermediate deep learning model (Model A or Model B) that branches off during the learning process of the original deep learning model. Deep learning models can increase model performance by repeatedly performing the same operation, but as complexity increases, computational overhead may also increase at the same time. In other words, the automatic model lightweighting device 130 generates intermediate deep learning models (Model A and Model B) in which part of the original learning process is omitted, and automatically secures deep learning models with various accuracy and computational overhead. You can.

Accordingly, a plurality of deep learning models built through the automatic model lightweighting method according to the present invention can be applied to a service learning in the cloud, and deep learning model developers can perform intermediate learning tasks while executing the learning task without a separate intermediate deep learning model creation task. Deep learning models can be automatically created. In other words, although intermediate deep learning models may have low accuracy, they can provide a diverse spectrum of model calculation overhead, enabling smooth service provision even in specific service environments. For example, when requests explode while providing an inference service, the service can be provided using a lightweight model with fast computation, thereby increasing user service satisfaction.

Although the present invention has been described above with reference to preferred embodiments, those skilled in the art may make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that you can do it.

100: Model automatic lightweight system
110: User terminal 130: Automatic model lightweight device
150: database
210: Processor 230: Memory
250: user input/output unit 270: network input/output unit
310: Algorithm receiving unit 320: Algorithm analysis unit
330: Branch point determination unit 340: Intermediate model generation unit
350: Deep learning model construction unit 360: Control unit
740: Branch point 750: Original learning direction
760: Branched Learning Directions

Claims (13)

In the automatic model lightweighting method performed in the automatic model lightweighting device,
Receiving a deep learning algorithm for building a deep learning model through an algorithm receiving unit;
Dividing the deep learning algorithm into a plurality of operation steps based on a repetition section that is repeatedly performed through an algorithm analysis unit;
determining at least one branch point existing between the plurality of operation steps in a learning process according to the deep learning algorithm, through a branch point determination unit;
Generating, through an intermediate model generator, at least one intermediate deep learning model that branches from the direction of the learning process based on the at least one branch point and proceeds to the final operation step of the deep learning algorithm; and
Completing the deep learning model and the at least one intermediate deep learning model upon completion of the learning process through a deep learning model building unit,
Generating the at least one intermediate deep learning model may include defining a candidate intermediate deep learning model according to the branch; Based on the number of layers (L) and prediction accuracy (A) of the candidate intermediate deep learning model, the appropriateness of the branch point is determined through the product operation (A*L) between the number of layers (L) and the prediction accuracy (A). calculating step; and determining the candidate intermediate deep learning model as an intermediate deep learning model when the appropriateness of the corresponding branch point satisfies a preset condition. A model automatic model lightweighting method for optimizing deep learning model serving.
The method of claim 1, wherein the deep learning algorithm is
An automatic model lightweighting method for optimizing deep learning model serving, characterized by including Deep Neural Network (DNN), Convolution Neural Network (CNN), and Recurrent Neural Network (RNN).
According to paragraph 1,
An automatic model lightweighting method for deep learning model serving optimization, wherein the deep learning model and the at least one intermediate deep learning model have different prediction accuracy and calculation speed.
According to paragraph 1,
The step of dividing into the plurality of operation steps is
Dividing operations of the deep learning algorithm into a plurality of layers; and
An automatic model lightweighting method for deep learning model serving optimization, comprising the step of determining operation steps corresponding to each of the plurality of layers.
According to paragraph 1,
The step of dividing into the plurality of operation steps is
Determining a repetition section that is repeatedly performed during the operation of the deep learning algorithm;
determining a pre-repetition section and a post-repetition section based on the repeat section;
determining at least one unit section for the repetition section; and
An automatic model lightweight method for optimizing deep learning model serving, comprising arranging the pre-repetition section, the at least one unit section, and the post-repetition section in order to determine the plurality of operation steps.
According to clause 5,
The step of determining the at least one branch point is
An automatic model lightweighting method for deep learning model serving optimization, comprising the step of determining each point where the repetition section ends as a branch point.
According to clause 6,
The step of determining the at least one branch point is
When the deep learning algorithm is CNN, determining the end point of a repetition section that sequentially progresses through at least one convolution layer and a pooling layer as the branch point. Automatic model lightweighting method for optimizing deep learning model serving.
delete According to paragraph 1,
The step of deciding with the intermediate deep learning model is
Automatic model lightweighting for deep learning model serving optimization, comprising the step of determining the candidate intermediate deep learning model as the intermediate deep learning model when the adequacy is twice that of the intermediate deep learning model generated in the previous step. method.
An algorithm receiving unit that receives a deep learning algorithm for building a deep learning model;
an algorithm analysis unit that divides the deep learning algorithm into a plurality of operation steps based on a repetition section that is repeatedly performed;
a branch point determination unit that determines at least one branch point that exists between the plurality of operation steps in a learning process according to the deep learning algorithm;
an intermediate model generator that generates at least one intermediate deep learning model that branches off from the direction of the learning process based on the at least one branch point and proceeds to a final operation step of the deep learning algorithm; and
A deep learning model construction unit that completes the deep learning model and the at least one intermediate deep learning model upon completion of the learning process,
The intermediate model generator defines a candidate intermediate deep learning model according to the branch, and based on the number of layers (L) and prediction accuracy (A) of the candidate intermediate deep learning model, the number of layers (L) and the prediction accuracy ( A) The adequacy of the branch point is calculated through the product operation (A*L) between the branches, and if the adequacy of the branch point satisfies preset conditions, the candidate intermediate deep learning model is determined as the intermediate deep learning model. Features an automatic model lightweight device for optimizing deep learning model serving.
delete The method of claim 10, wherein the intermediate model generator
Automatic model lightweighting device for deep learning model serving optimization, characterized in that when the adequacy is doubled compared to the intermediate deep learning model generated in the previous step, the candidate intermediate deep learning model is determined as the intermediate deep learning model.
Receiving a request for an inference service from a user terminal;
Determining the status of available cloud resources based on the time of receipt of the request, predicting a response generation time for the request, and determining one of a plurality of deep learning models according to the response generation time; and
Generating a response to the request using the determined deep learning model and providing it to the user terminal,
The plurality of deep learning models include a deep learning model generated to have different prediction accuracy and calculation speed based on an automatic model lightweighting method for deep learning model serving optimization and at least one intermediate deep learning model,
When the at least one intermediate deep learning model is a candidate intermediate deep learning model defined based on a branching point that exists between a plurality of operation steps divided based on a repetitive section of the deep learning model, the model Based on the number of layers (L) and prediction accuracy (A), the adequacy of the branch point calculated through the product operation (A*L) between the number of layers (L) and the prediction accuracy (A) satisfies the preset conditions. A method of providing cloud inference services using an automatic model lightweighting method for optimizing deep learning model serving, characterized in that it corresponds to a candidate intermediate deep learning model that satisfies the criteria.