KR20200023660A - Electronic device for controlling performance of at least one processor when providing inference service through deep learning model and operating method thereof - Google Patents

Abstract

The present invention relates to an electronic device for controlling the performance of at least one processor when an inference service is provided through a deep learning model, and an operating method thereof. According to various embodiments of the present invention, the electronic device performing an arithmetic operation through a deep learning model includes: a layer management part performing an arithmetic operation on input data through a deep learning model when receiving the input data, and identifying a milestone identifier pre-allocated to at least one of a plurality of layers included in the deep learning model while performing the arithmetic operation on the input data; a performance confirmation part measuring and storing time consumed from the start of the arithmetic operation for the input data to the moment for the identification of the milestone identifier in response to the identification of the milestone, and then, obtaining a delay value based on the stored consumed time and a preset target time; a delay control part determining whether the obtained delay value is within a threshold range based on a preset value, and then, generating a control command for adjusting the performance of at least one processor based on a result of the determination; a resource management part changing the frequency of the processor in the electronic device in accordance with the generated control command; and an input/output part providing data calculated until a moment for the identification of a termination identifier as output data corresponding to the received input data in response to the identification of the termination identifier when the arithmetic operation for the input data is continued through the processor with the changed frequency.

Description

ELECTRONIC DEVICE FOR CONTROLLING PERFORMANCE OF AT LEAST ONE PROCESSOR WHEN PROVIDING INFERENCE SERVICE THROUGH DEEP LEARNING MODEL AND OPERATING METHOD THEREOF }

The present invention relates to an electronic device that controls the performance of at least one processor when providing an inference service through a deep learning model, and a method of operating the same.

Deep learning is a technique used to cluster or classify objects or data, and enables robust and accurate inference through multi-layer neural networks. In recent years, its use has been expanded. For example, techniques related to understanding visual scenes driven by neural networks in deep learning include cyber-physical, such as wearable devices for augmented reality, home automation devices, camera-based surveillance and autonomous vehicles. system is being actively applied.

As such, the neural network used for deep learning may be composed of a plurality of layers arranged in succession, and the plurality of layers perform calculation for each layer according to the order in which the plurality of layers are arranged, and then transfer the calculation result to the next arranged layer. Can be set.

On the other hand, since inference processes through deep learning involve numerous operations, electronic devices that process data using deep learning have high performance processors, power supplies capable of supplying sufficient power, and memory capable of providing sufficient storage space. Etc., it is necessary to be provided. Therefore, the use of deep learning has been limited in mobile devices with limited resources such as performance, power supply, and storage space.

Recently, various methods for efficiently managing resources have been proposed so that the reasoning process according to deep learning can be smoothly performed in a mobile device. For example, a method of applying a hardware-based accelerator to an electronic device or a method of reducing a computation amount by compressing a deep learning model has been proposed. Through the proposed methods, the inference process of deep learning in mobile and embedded devices is proposed. It became possible to perform more smoothly.

The inference process may mean a process of obtaining an output corresponding to an arbitrary input through deep learning. In addition, the deep learning model is an algorithm that can be used in an inference process, and can be used to define a structure of a plurality of layers constituting a neural network or define a parameter or weight of an operation performed for each layer.

According to an embodiment of the present disclosure, a parameter or weight defined based on the deep learning model may be learned, that is, updated through a training process. Since the training process requires a lot of computational processing, it can be performed through a server such as a cloud, and the deep learning model learned through this can be provided to various mobile devices such as smart phones, surveillance cameras, smart speakers, and autonomous vehicles. have. In this way, deep learning models provided to various mobile devices can be used in the inference process to classify the data input from the application and predict the results.

On the other hand, the inference process may require relatively less computational throughput than the training process, but the performance and power supply of the processor in the mobile device may be limited, so the latency and power efficiency of the computation may be It can be considered as a major performance indicator. For example, when a result of a speech-to-text (STT) application or a translation application executed on a mobile device is output through an inference process or a result of a judgment used for autonomous driving is output through an inference process, respectively. The result of can be requested to be printed as soon as possible. In this case, the mobile device may increase the performance of the processor and output the result value according to the inference process with a short latency (for example, 200 ms). However, at the same time, it is important to consider that the amount of power that a mobile device can supply is a relatively large amount of power that can be consumed if the performance of the processor is increased to output the result with short latency. Increasing the processor's performance is subject to limitations. As a result, when the calculation according to the inference process is performed in the mobile device, the latency and power consumption may be considered to be in a trade off relationship. Accordingly, in consideration of the optimal latency requested for each output result, The performance of the processor needs to be dynamically adjusted.

Furthermore, in performing the determination using the inference process in the mobile device, the result values are set to be output at regular intervals according to the specified latency at each determination, as much as the correct result is output at each determination. This may be an important factor in terms of increasing the reliability of the value. For example, when performing a judgment required for autonomous driving using a reasoning process in a mobile device, if the result value is output according to a different latency at each determination, it is difficult to predict the time at which the result value is output. As a result, the deep learning model used in the inference process may be difficult to use when performing the judgment required for autonomous driving.

In various embodiments of the present disclosure, in performing an inference process using a deep learning model, a method of dynamically adjusting a processor's performance in order to set a time point at which a result value according to the inference process is output within a predictable range is provided. Suggest. For example, the electronic device according to an embodiment may calculate an optimal latency requested for each output value and adjust the performance of at least one processor in the electronic device to output the result value according to the calculated latency.

According to various embodiments of the present disclosure, when an electronic device performs an operation using a deep learning model, the electronic device performs an operation on the input data using the deep learning model and receives an operation on the input data. A layer manager for identifying a milestone identifier pre-allocated to at least one of a plurality of layers included in the deep learning model, and in response to the milestone identifier being identified, the operation on the input data is started. A performance checking unit for measuring and storing a time required until the time point at which the milestone identifier is identified, and obtaining a delay value using the stored time required and a predetermined target time; and the obtained delay value is based on a preset value. It is determined whether or not within the threshold range, and based on the determination result A delay controller for generating a control command for adjusting the performance of at least one processor, a resource manager for changing a frequency of at least one processor in the electronic device according to the generated control command, and at least the frequency of which is changed In response to the termination identifier being identified when the operation on the input data is continued through one processor, an input / output for providing the calculated data to the time point at which the termination identifier is identified as output data corresponding to the received input data. It may include wealth.

Also, according to various embodiments of the present disclosure, a method of controlling an electronic device that performs an operation using a deep learning model may include performing an operation on the input data using a deep learning model when input data is received. Identifying a milestone identifier previously assigned to at least one of a plurality of layers included in the deep learning model when the operation on the input data is performed; and in response to the milestone identifier being identified, the input Measuring and storing the time required until the milestone identifier is identified after the operation on the data is started, acquiring a delay value using the stored time required and a predetermined target time, and obtaining the delay value Determining whether it is within a threshold range based on a preset value Generating a control command for adjusting the performance of at least one processor based on the determination result, changing a frequency of at least one processor in the electronic device according to the generated control command, and the frequency Responsive to identifying an end identifier when the operation on the input data is continued through the at least one processor having changed, providing output data corresponding to the input data until the point at which the end identifier is identified. It may include a step.

According to various embodiments of the present disclosure, in performing an inference process through a deep learning model, the electronic device may be configured to output a result value according to the inference process within a predictable time range. As such, as the result value according to the inference process is set to be output within a predictable time range, the stability of the electronic device using the deep learning model and the reliability of the operation according to the deep learning can be guaranteed.

In addition, according to various embodiments of the present disclosure, the electronic device may dynamically adjust the performance of at least one processor so that a result value according to an inference process is output according to a specified latency. Accordingly, the case in which the performance of the processor is adjusted upward more than necessary disappears, so that power consumption in the electronic device can be managed efficiently.

1 is a diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure.
2 is a diagram illustrating a configuration of a runtime module included in an electronic device according to an embodiment of the present disclosure.
3 is a diagram illustrating a configuration of a feedback circuit included in an electronic device according to an embodiment of the present disclosure.
4 is a diagram for describing a method of allocating a milestone identifier to a deep learning model according to an exemplary embodiment.
5 is a flowchart illustrating an operation of allocating a milestone identifier to a deep learning model in an electronic device according to an embodiment of the present disclosure.
6 is a flowchart illustrating a method of controlling performance of at least one processor when performing an inference process through a deep learning model in an electronic device according to an embodiment of the present disclosure.

The various embodiments disclosed in this document are not presented to limit the present invention to specific embodiments, and the components introduced through the various embodiments are not limited to all changeable equivalents or substitutes included in the spirit and technical scope of the present invention. It will be readily understood by those skilled in the art that the present invention is presented in the meaning included. In addition, in describing each of the drawings, all terms used in the present specification, including technical or scientific terms, unless otherwise defined, are generally understood by those of ordinary skill in the art. It can be interpreted as having the same meaning. In addition, the objects and effects of the present invention, and the technical configurations for achieving them will be apparent through the embodiments described in detail with the accompanying drawings. In the following description of the present invention, when it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof may be omitted, and the terminology described hereinafter will be described in detail. As terms defined in consideration of roles, functions, and the like, they may be interpreted differently from the meanings used in the past according to intentions or customs of users and operators.

It is to be understood that the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. Various embodiments disclosed in this document are provided to make the disclosure of the present invention complete, and to fully inform the scope of the invention to those skilled in the art to which the present invention pertains. It is only defined by the scope of the claims set forth in the scope.

In this document, when a part is said to "include" a certain component, it means that it can further include other components, without excluding the other components unless otherwise stated. In addition, in various embodiments of the present disclosure, each component, functional block, or means may be composed of one or more subcomponents, and the electrical, electronic, and mechanical functions performed by each component are electronic circuits. It may be implemented by various known elements or mechanical elements such as an integrated circuit, an application specific integrated circuit (ASIC), and may be implemented separately or two or more may be integrated into one.

Meanwhile, the steps of the blocks or flowcharts in the attached block diagrams refer to computer program instructions that are mounted in a processor or memory of a data processing device such as a general purpose computer, a special purpose computer, a portable notebook computer, a network computer, and perform specified functions. Can be interpreted. Since these computer program instructions may be stored in a memory provided in a computer device or in a computer readable memory, the functions described in the blocks of the block diagram or the steps of the flowchart are produced as an article containing an instruction means for performing this. May be In addition, each block or step may represent a portion of a module, segment, or code that includes one or more executable instructions for executing a specified logical function (s). It should also be noted that in some alternative embodiments, the functions recited in blocks or steps may be executed in a different order. For example, two blocks or steps shown in succession may be performed substantially concurrently or in reverse order, and in some cases, some blocks or steps may be omitted.

1 is a diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure. According to various embodiments of the present disclosure, the electronic device 100 may include at least one of the application 110, the runtime module 120, the processor 130, the sensor 140, the storage 150, and the input / output unit 160. Can be. Here, the runtime module 120 is used to mean a module that performs a specific instruction, and it is understood that the instructions performed through the modules may be understood to be performed by the processor 130 of the electronic device.

The application 110 of the electronic device 100 may refer to a program capable of performing a specific task or a module in which the program is executed. According to an embodiment of the present disclosure, the application 110 may call an inference process through the deep learning model or perform an operation of providing a result data obtained through the inference process to the user in a predetermined manner. The application 110 may be configured to invoke an inference process in response to the input data being received, wherein the called inference process is periodic or aperiodic through the runtime module 120 based on the data received at the sensor 140. It can be performed as.

According to an embodiment of the present disclosure, the application 110 may call a specific deep learning model M by referring to preset quality of service (QoS) data. Here, the QoS data is an indicator indicating a quality of service, and the indicator may be used to guarantee a latency or a data loss rate of a certain level or less on a network or a processor. For example, Q representing QoS data may be defined as in Equation 1 below.

In Equation 1, d may be a value representing a first response time for obtaining output data when performing an inference process through the deep learning model M. FIG. For example, the first response time may mean a time required from when the inference process starts to when it ends. In addition, in Equation 1, C may be a value representing a compression bound for the deep learning model M.

According to an embodiment of the present disclosure, when the deep learning model M is called based on preset QoS data, the application 110 may transmit the deep learning model M to the runtime module 120 together with the QoS data.

The runtime module 120 of the electronic device 100 may perform a deep learning operation using input data received through the input / output unit 160 and a deep learning model received from the application 110. For example, the runtime module 120 may include a plurality of pieces included in the deep learning model to generate answer data for the identified question data when the “cat picture” is received as the input data and question data for identifying the input data is identified. An operation on the input data may be performed using an operation method defined in each of one or more layers among the layers of. The runtime module 120 may obtain answer data called "cat" as a result of performing the operation, and deliver the obtained answer data to the application 110 as an inference result of the input data.

According to an embodiment of the present disclosure, when the QoS data and the deep learning model are received from the application 110, the runtime module 120 may check the QoS data and perform compression on the deep learning model. For example, the runtime module 120 may identify a first response time for which acquisition of output data from the QoS data is requested. In addition, the runtime module 120 may check the total time required by summing the time required to perform an operation using each of the plurality of layers in the deep learning model. If it is determined that the total required time exceeds the first response time, the runtime module 120 may reduce the time required for the calculation using the deep learning model by compressing the deep learning model.

According to an embodiment of the present disclosure, the runtime module 120 may check the speed at which the calculation proceeds at regular intervals while performing an operation on the input data using the deep learning model, and operate the processor 130 based on the check result. By controlling the speed, the result data of the operation can be prevented from being output earlier or later than the target time. For example, when the runtime module 120 performs compression on the deep learning model, the runtime module 120 may assign a first milestone identifier to at least one layer of the plurality of layers in the deep learning model. Thereafter, when the pre-allocated first milestone identifier is identified while performing the operation on the input data using the deep learning model, the runtime module 120 determines whether the first module is determined from the time when the operation is started or when the other milestone identifier is previously identified. It is possible to measure the second response time, which is the time actually spent on the operation up to the point of time when the milestone identifier is confirmed. At the same time, the runtime module 120 may identify the first target time that should be used for the calculation from the time when the operation is started or when another milestone identifier is confirmed to the time when the first milestone identifier is confirmed. According to various embodiments of the present disclosure, the runtime module 120 may obtain a ratio of the second response time to the first target time as the delay value at the k th time point, and the delay value at the k th time point may be expressed by Equation 2 below. Can be defined.

If the delay value is larger than 1, it may be determined that the operation through the runtime module 120 is being performed slower than the target time. On the other hand, if the delay value is less than 1, it may be determined that the operation through the runtime module 120 is being performed faster than the target speed.

According to an embodiment of the present disclosure, when it is determined that the operation is being performed slower than the target speed, the runtime module 120 may adjust the operation speed of the processor 130 used for the operation. In addition, when it is determined that the operation is being performed faster than the target time, the runtime module 120 may adjust the operation speed of the processor 130 used for the operation.

The processor 130 of the electronic device 100 may process at least some of the operations performed by the runtime module 120. For example, the processor 130 may perform an operation performed on at least one layer of the plurality of layers in the deep learning model, and then provide the result data to the runtime module 120. According to various embodiments of the present disclosure, the processor 130 may be implemented in a form including a runtime module 120, and in this case, all instructions executed in the runtime module 120 may be understood to be performed in the processor 130. have.

According to an embodiment of the present disclosure, the processor 130 may include one or more processors. For example, the processor 130 may include one or more central processing units (CPUs) and one or more graphics processing units (GPUs). The processor 130 may select a processing apparatus required to perform an operation based on each type of the plurality of layers in the deep learning model, and perform the operation using the selected processing apparatus.

According to an embodiment of the present disclosure, when a control command corresponding to increasing the operation speed is received from the runtime module 120, the processor 130 adjusts the frequency of the processor 130 corresponding to the received control command to increase the frequency. It can speed up the computation process. On the other hand, when a control command corresponding to a lowering of the operation speed is received from the runtime module 120, the processor 130 adjusts the frequency of the processor 130 corresponding to the received control command to lower the processing speed. Can be lowered.

The sensor 140 of the electronic device 100 may generate an electrical signal or data value corresponding to an operating state (eg, power or temperature) inside the electronic device 100 or an external environmental state. For example, the sensor 140 may include at least one of a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, and an illuminance sensor. It may include one. According to an embodiment of the present disclosure, the electronic device 100 may perform an operation through the deep learning model using input data obtained from the sensor 140 after being mounted in the autonomous vehicle, and may perform calculation through the deep learning model. The output data may be generated as a result value and provided to the autonomous vehicle.

The storage unit 150 of the electronic device 100 may store various data used by at least one component of the electronic device 100. Here, the data may include input data or output data for software and commands related thereto. In addition, the storage unit 150 may include a volatile memory or a nonvolatile memory. According to an embodiment of the present disclosure, the storage unit 150 stores information about one or more deep learning models that can be used when performing an operation on input data and a plurality of layers included in the one or more deep learning models. Can be.

The input / output unit 160 of the electronic device 100 may receive or provide a command or data to be used for a component of the electronic device 100 from an external (eg, a user) of the electronic device 100. The input / output unit 160 may be divided into an input unit and an output unit, the input unit may include a mouse, a keyboard, and a touch pad, and the output unit may include a display and a speaker.

2 is a diagram illustrating a configuration of a runtime module included in an electronic device according to an embodiment of the present disclosure. According to various embodiments of the present disclosure, the electronic device 100 may include at least one of the runtime module 120, the processor 130, the storage 150, and the input / output unit 160. In addition, the runtime module 120 may include a QoS manager 200 and an execution manager 210. In addition, the QoS manager 200 may include a model compressor 201, a milestone identifier allocator 203, a performance checker 205, a delay controller 207, and a resource manager 209. In addition, the execution manager 210 may include a data preprocessor 211, a model manager 213, and a layer manager 215. In addition, the processor 130 may include a central processing unit 220 and a graphics processing unit 225.

According to an embodiment of the present disclosure, the QoS manager 200 may compress the deep learning model or control the operation speed of the processor 130 based on the QoS data. The execution manager 210 may perform an operation on the input data through the deep learning model or identify a milestone identifier assigned to at least some layers in the deep learning model.

The model compressing unit 201 of the QoS managing unit 200 checks d (first response time for obtaining output data in performing an inference process through a deep learning model) and C (confirmed through preset QoS data). Compression range for the deep learning model) to determine whether compression for the deep learning model is required. For example, when there is a call for the inference process, the model compressor 201 may check the memory usage of the deep learning model selected in accordance with the call and the amount of available memory of the electronic device 100. Accordingly, the amount of compression for the deep learning model can be determined. Meanwhile, the model compressor 201 may use a stepwise compression method in order to prevent the accuracy of the inference process from being unexpectedly lost when performing the compression on the deep learning model. For example, when compressing the deep learning model, the model compressor 201 may use an approximation method through a singular value decomposition technique that can control the level of compression by selecting an appropriate rank. Alternatively, the model compressor 201 may determine the level of compression of the deep learning model through a comparison between the total time required for the calculation through the deep learning model and the first response time. For example, when it is determined that the total time required for the calculation through the deep learning model is longer than the first response time, the model compression unit 201 may calculate a difference between the total time required and the first response time. Accordingly, the compression level for the deep learning model may be determined. Meanwhile, when the compression level of the deep learning model is determined, the model compressor 201 may generate a control command corresponding to the deep learning model, and transmit the generated control command to the model manager 213.

The milestone identifier allocator 203 of the QoS manager 200 may perform profiling to analyze an execution state or a communication state with respect to the deep learning model on which compression is performed. According to an embodiment, the milestone identifier allocator 203 may measure an operation time of each of the plurality of layers in the deep learning model on which compression is performed. Subsequently, the milestone identifier allocator 203 may generate a control command for allocating milestone identifiers to layers located at target time intervals using the measured operating time. For example, the milestone identifier allocator 203 may identify layers that are operated at intervals of 100 ms from the time point at which the calculation through the deep learning model with compression is started, and assigns a milestone identifier to each of the identified layers. You can create a command. More specifically, the milestone identifier allocator 203 assigns a first milestone identifier to a first layer operating at a time when 100 ms has elapsed since the operation is started, and then operates at a time when 100 ms has elapsed again. A control command for assigning the second milestone identifier to the two layers can be generated. Meanwhile, the milestone identifier allocator 203 may transmit a control command generated for allocating the milestone identifier to the model manager 213.

The data preprocessor 211 of the execution manager 210 is used for calculation through the deep learning model in response to the input data being received from the outside after the operation of assigning the milestone identifier to the deep learning model in which the compression is performed is completed. The format or content of the received input data can be modified to make it possible.

The model manager 213 of the execution manager 210 performs a deep learning model or compresses the deep learning model based on the control command received from the model compressor 201 or the control command received from the milestone identifier allocator 203. A milestone identifier may be assigned to at least one layer in the running model.

The layer manager 215 of the execution manager 210 receives an input data whose format or contents are modified from the data preprocessor 211, and infers a process using the deep learning model on which the compression is performed and the modified input data. Operation can be started. For example, the layer manager 215 may perform an operation using an operation formula corresponding to each type of a plurality of layers in the deep learning model on which compression is performed. In this case, the order of the calculation may be performed according to the order in which the plurality of layers in the deep learning model are arranged.

According to an embodiment of the present disclosure, the layer manager 215 may determine whether a milestone identifier is assigned to the layer in operation. If the pre-allocated first milestone identifier is confirmed, the layer manager 215 measures the time taken from the time point at which the operation is started to the time point at which the first milestone identifier is confirmed, and then the first data on the measured time required. To the performance verification unit 205 of the QoS management unit 200. Subsequently, if the second milestone identifier assigned in advance is confirmed by continuing the operation, the layer manager 215 measures the time taken from when the first milestone identifier is confirmed to when the second milestone identifier is verified, As described above, the second data about the measured time required may be transferred to the performance checker 205 of the QoS manager 200. The layer manager 215 may repeat the operation-identifier identification-time measurement-measurement value transfer until the termination identifier is confirmed, and may terminate the operation in response to the termination identifier being confirmed. When the calculation is completed, the layer manager 215 may provide the user with the calculated data as output data corresponding to the input data to the user through the input / output unit 160.

When the performance checker 205 of the QoS manager 200 receives data about the time required from the layer manager 215 to the time point when the milestone identifier is checked, the performance manager 205 uses at least one of the received data in the electronic device 100. You can see the performance of one component. According to an embodiment of the present disclosure, in response to the pre-allocated first milestone identifier being confirmed, first data regarding the time required from the time point at which the operation is started to the time point at which the first milestone identifier is confirmed is received from the layer manager 215. In this case, the performance checking unit 205 may obtain a delay value by comparing the required time according to the received first data with a preset target time. In this case, the delay value may be obtained according to Equation 2. For example, when the time required according to the first data is measured as 120 ms and the target time is set to 100 ms, the delay value may be calculated as 1.2 (= 120 ms / 100 ms). Meanwhile, the delay value calculated in this manner may be transferred from the performance checking unit 205 to the delay control unit 207.

The delay controller 207 of the QoS manager 200 may determine whether the actual operation is progressing faster or slower than the target speed based on the delay value received from the performance checker 205. . For example, when 1.2 is received as the delay value (that is, when the delay value is greater than 1), the delay controller 207 may determine that the actual operation is progressing slower than the target speed. On the other hand, if the delay value is less than 1, the delay controller 207 may determine that the actual operation is proceeding faster than the target speed.

According to an embodiment of the present disclosure, if it is determined that the actual operation is progressing slower than the target speed (when the delay value is greater than 1), the delay control unit 207 instructs to increase the performance of the processor 130. You can create a command. On the other hand, if it is determined that the actual operation is proceeding faster than the target speed (when the delay value is less than 1), the delay control unit 207 generates a control command for instructing to increase the performance of the processor 130. can do. Meanwhile, the delay controller 207 may refer to a preset threshold range as shown in Table 1 below in generating a control command for instructing the processor to increase or decrease the performance of the processor.

Upward adjustment 1st level up 2nd level up 3rd level up Downward adjustment 1st level down adjustment 2nd level down adjustment 3rd level down adjustment

In Table 1, the first level, the second level, and the third level are values related to a degree of adjusting the performance of the processor, and may be preset by the user. For example, the first level upward adjustment (or downward adjustment) may refer to an upward adjustment (or downward adjustment) of the frequency of the processor by 5%. Alternatively, the first level up adjustment (or down adjustment) may mean that the frequency of the processor is adjusted up (or down) by 50 MHz. For example, as described above, when the delay value is measured as 1.2, since the delay value is larger than 1, it corresponds to a case where upward adjustment is required, and the difference between 1 and the delay value may be calculated as 0.2. In this case, the delay controller 207 may determine that the performance of the processor needs to be adjusted to the second level by referring to the definition in Table 1 above, and may generate a control command corresponding thereto.

According to one embodiment, the delay control unit 207 generates a control command for instructing to increase (or down) the performance of the processor 130, the following equation (3) approximated through a proportional integral (PI) controller Can be used.

In Equation 3, speed (k) may mean the performance of the processor 130 required at the k time point based on the data obtained before the k time point. For example, if the performance speed (k) of the processor at k is 10% greater than the processor performance speed (k-1) at k-1, the frequency of the processor should be increased by 10%. Conversely, if speed (k) is less than speed (k-1), the frequency of the processor must be reduced. Here, K _p and K _I are gains of an experimentally obtained PI controller, and e (k) may mean a difference between a delay value obtained at time k and a reference value as an error value. For example, if a delay value obtained at time k is defined as a delay value k and a reference value is defined as 1, e (k) may mean 1-delay value k.

According to various embodiments, K _p and K _I of Equation 3 may be experimentally obtained as shown in Table 2 below.

K _p K _I CPU -0.28 -0.42 GPU -0.60 -0.60

According to Table 2, different values may be obtained depending on whether the processor 130 that needs adjustment is a CPU or a GPU.

The resource manager 209 of the QoS manager 200 may control the performance of at least one component in the electronic device 100 according to the control command generated by the delay controller 207. For example, the resource manager 209 may adjust the performance of the processor 130 based on the control command received from the delay controller 207. In this case, the resource manager 209 may adjust the performance of the processor by using a tuning knob such as dynamic voltage / frequency scaling (DVFS).

The processor 130 may adjust the performance (eg, frequency) of the CPU 220 or the graphic processor 225 included in the processor 130 under the control of the resource manager 209. When the performance is adjusted, the processor 130 may provide a different calculation speed than before the adjustment when the layer manager 215 performs an operation based on the deep learning model.

3 is a diagram illustrating a configuration of a feedback circuit included in an electronic device according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, when the milestone identifier is identified at the kth time point, the performance checking unit 205 may obtain a delay value tard (k). Subsequently, the delay controller 207 may obtain e (k), which is a difference between 1 set as a delay value and a reference value. The delay control unit 207 may determine whether the performance of the processor 130 needs to be adjusted based on the obtained e (k), and if necessary, to what extent it needs to be adjusted. For example, when the delay value is 1.2, the delay controller 207 may determine that the performance of the processor needs to be adjusted upward because the delay value is greater than one. At this time, e (k) is confirmed as 0.2, the delay control unit 207 can finally determine that the performance of the processor needs to be adjusted to the second level up.

을 생성할 수 있고, 생성된 제어 명령을 DVFS 관리부(300)로 전달할 수 있다. The delay control unit 207 serves as a control command for adjusting the performance of the processor 130 to the second level.

May be generated, and the generated control command may be transmitted to the DVFS manager 300.

In addition, the DVFS management unit 300 controls to adjust the frequency of at least one of the central processing unit 220 and the graphics processing unit 225 in the processor 130 in response to the control command received from the delay control unit 207. As a command, freq (k + 1) may be generated, and the generated control command may be transferred to at least one of the CPU 220 and the graphic processor 225. Thereafter, the calculation through the deep learning model may be continued according to the adjusted performance.

According to an embodiment, when the milestone identifier is identified at the k + 1th time point, the performance checking unit 205 may acquire a delay value tard (k + 1). Subsequently, the delay controller 207 may acquire e (k + 1), which is a difference between 1 set as a delay value and a reference value, and may repeatedly perform an operation of adjusting the performance of the processor 130.

4 is a diagram for describing a method of allocating a milestone identifier to a deep learning model according to an exemplary embodiment.

When the “cat photo” is received as the input data 400, the electronic device 100 may perform an operation on the input data 400 using the deep learning model 420, and may result in operation 410. ) Can be obtained. On the other hand, the deep learning model 420 may be composed of a plurality of layers for performing different operations according to the type, the plurality of layers may be plotted in a form listed in a predetermined order.

According to an embodiment, the milestone identifier allocator 203 may measure the total time required for the calculation through the deep learning model before performing the calculation through the deep learning model. For example, the milestone identifier allocator 203 may confirm that the total time required for the calculation through the deep learning model is 300 ms. In this case, the milestone identifier allocator 203 may allocate an end identifier to the last layer among the plurality of layers in the deep learning model.

In addition, the milestone identifier allocator 203 may allocate the milestone identifier for each preset time interval based on the total time required. For example, the milestone identifier allocator 203 may allocate a milestone identifier to a layer located at a time point corresponding to 1/3 of the total time required. That is, the milestone identifier allocator 203 has a first milestone for each of the first layer 430 located at a time corresponding to 1/3 of the total time required and the second layer 440 at a time corresponding to 2/3. An identifier and a second milestone identifier can be assigned. If the total time required is 300 ms, the first milestone identifier may be assigned to the first layer 430 performing the calculation at a time point 100 ms after the start of the operation, and the second milestone identifier may be assigned to the first milestone identifier. The milestone identifier may be assigned to the second layer 440 that performs the calculation at a time point 100 ms elapsed from the identified time point.

5 is a flowchart illustrating an operation of allocating a milestone identifier to a deep learning model in an electronic device according to an embodiment of the present disclosure.

In operation 500, the preset QoS data may be checked. The QoS data is an indicator indicating the quality of service requested by the user, and may be preset and stored before the inference process through the deep learning model is performed.

In operation 510, compression may be performed on the deep learning model based on the identified QoS data.

In operation 520, the profiling of the deep learning model on which the compression is performed may be performed to measure an operation time of each of the plurality of layers in the deep learning model.

In operation 530, at least one layer from among the plurality of layers may be selected based on the measurement result, and a milestone identifier may be assigned to the selected at least one layer.

According to various embodiments of the present disclosure, some of the operations disclosed in FIG. 5 may be omitted or repeated a plurality of times. In addition, each of the operations disclosed in FIG. 5 may be considered to be an example, and may not be construed as limiting one operation to another operation.

6 is a flowchart illustrating a method of controlling performance of at least one processor when performing an inference process through a deep learning model in an electronic device according to an embodiment of the present disclosure.

In operation 600, when the input data is received, an operation on the received input data may be performed using the deep learning model.

In step 610, it may be determined whether an end identifier is identified. If the end identifier is identified, the operation through the deep learning model may end, and then step 660 may be performed. On the other hand, if the termination identifier is not identified, step 620 may be performed.

In step 620, it may be determined whether the milestone identifier is identified. If the milestone identifier is identified, steps 630 to 650 may be performed. On the other hand, if the milestone identifier is not identified, operation through the deep learning model may continue according to step 600.

In operation 630, the required time from the start of the operation on the input data to the time point when the milestone identifier is identified may be measured and stored.

In operation 640, the delay value may be obtained using the stored time required and the predetermined target time.

In step 650, in response to the delay value not being within a threshold range based on the preset value, the frequency of the at least one processor may be changed.

In operation 660, data calculated until the end identifier is identified may be provided as output data corresponding to the input data.

According to various embodiments of the present disclosure, some of the operations disclosed in FIG. 6 may be omitted or repeated a plurality of times. In addition, each of the operations disclosed in FIG. 6 may be considered to be an example, and may not be construed as limiting one operation to another operation.

According to various embodiments of the present disclosure, when an electronic device performs an operation using a deep learning model, the electronic device performs an operation on the input data using the deep learning model and receives an operation on the input data. A layer manager for identifying a milestone identifier pre-allocated to at least one of a plurality of layers included in the deep learning model, and in response to the milestone identifier being identified, the operation on the input data is started. A performance checking unit for measuring and storing a time required until the time point at which the milestone identifier is identified, and obtaining a delay value using the stored time required and a predetermined target time; and the obtained delay value is based on a preset value. It is determined whether or not within the threshold range, and based on the determination result A delay controller for generating a control command for adjusting the performance of at least one processor, a resource manager for changing a frequency of at least one processor in the electronic device according to the generated control command, and at least the frequency of which is changed In response to the termination identifier being identified when the operation on the input data is continued through one processor, an input / output for providing the calculated data to the time point at which the termination identifier is identified as output data corresponding to the received input data. It may include wealth.

In an electronic device that performs an operation using a deep learning model according to various embodiments of the present disclosure, the layer manager may include a first plurality of layers used to perform an operation on the input data among the plurality of layers. Select layers, identify a type of each of the selected first plurality of layers, and perform an operation on the input data using an operation formula corresponding to each of the identified first plurality of layers. have.

According to various embodiments of the present disclosure, an electronic device that performs an operation using a deep learning model may check preset quality of service (QoS) data and perform an operation on the deep learning model based on the identified QoS data. Profiling the model compression unit performing compression and the deep learning model on which the compression is performed to measure an operation time for each of the plurality of layers in the deep learning model on which the compression is performed, and the measured operation The apparatus may further include a milestone identifier allocator configured to select one or more layers operated at predetermined time intervals from among the plurality of layers and to assign a milestone identifier to each of the selected one or more layers.

In the electronic device performing an operation using a deep learning model according to various embodiments of the present disclosure, the performance checking unit may obtain a ratio of the stored required time to the preset target time as the delay value. have.

In an electronic device performing an operation using a deep learning model according to various embodiments of the present disclosure, the delay controller determines that the obtained delay value is out of the threshold range while being smaller than 1, the preset value. And in response to generating a first control command instructing to downgrade the performance of the at least one processor in the electronic device, wherein the obtained delay value is greater than 1, the preset value, and out of the threshold range. In response to the determination, the second control command may be generated to instruct to increase the performance of the at least one processor in the electronic device.

In the electronic device performing an operation using a deep learning model according to various embodiments of the present disclosure, the delay controller measures the degree to which the obtained delay value is out of the threshold range and measures the measurement. The first control command or the second control command may be determined according to the determined value based on the determined value.

According to various embodiments of the present disclosure, a method of controlling an electronic device that performs an operation using a deep learning model includes performing an operation on the input data using a deep learning model when input data is received. Identifying a milestone identifier previously assigned to at least one of a plurality of layers included in the deep learning model when an operation on the input data is performed, in response to the milestone identifier being identified, Measuring and storing the time required until the milestone identifier is identified after the operation is started; acquiring a delay value using the stored time duration and a preset target time; and obtaining the predetermined delay value Determining whether it is within a threshold range based on a value, wherein Generating a control command for adjusting the performance of at least one processor based on the determination result, changing a frequency of at least one processor in the electronic device according to the generated control command, and changing the frequency In response to the termination identifier being identified when the operation on the input data is continued via at least one processor, providing the calculated data as output data corresponding to the input data up to the point at which the termination identifier is identified. It may include.

According to various embodiments of the present disclosure, a method of controlling an electronic device that performs an operation using a deep learning model, wherein the performing of the operation on the input data may be performed on the input data of the plurality of layers. Selecting a first plurality of layers that are used to perform an operation on the object, identifying a type of each of the selected first plurality of layers, and an operation corresponding to the type of each of the identified first plurality of layers The method may further include performing an operation on the input data using a formula.

According to various embodiments of the present disclosure, a method of controlling an electronic device that performs an operation using a deep learning model may include checking a predetermined quality of service (QoS) data and based on the determined QoS data. Performing compression on a running model, performing profiling on the deep learning model on which the compression is performed, and measuring an operation time for each of the plurality of layers in the deep learning model on which the compression is performed, The method may further include selecting one or more layers operated at predetermined time intervals among the plurality of layers using the measured operating time, and assigning a milestone identifier to each of the selected one or more layers.

According to various embodiments of the present disclosure, a method of controlling an electronic device that performs an operation using a deep learning model, wherein the obtaining of the delay value may include a ratio of the stored time to the preset target time. The method may further include obtaining the delay value as the delay value.

According to various embodiments of the present disclosure, a method of controlling an electronic device that performs an operation using a deep learning model may include generating the control command, wherein the obtained delay value is less than 1, the preset value. In response to being determined to be out of the threshold range, generating a first control command instructing to downgrade the performance of the at least one processor in the electronic device, and wherein the obtained delay value is the preset value. And in response to being determined to be greater than 1 and out of the threshold range, generating a second control command instructing to increase the performance of the at least one processor in the electronic device.

According to various embodiments of the present disclosure, in the method of controlling an electronic device that performs an operation using a deep learning model, the generating of the control command may include the degree to which the obtained delay value is out of the threshold range. degree), determining a value at which the performance of the at least one processor is adjusted down or up based on the measured degree, and according to the determined value, the first control command or the second control. The method may further include generating a command.

According to an embodiment of the present disclosure, a method of controlling an electronic device that performs an operation using a deep learning model may be implemented as a computer program stored in a storage medium for execution in combination with a computer.

In addition, the method for controlling an electronic device performing an operation using the deep learning model according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. have. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

In the present invention as described above has been described by the specific embodiments, such as specific components and limited embodiments and drawings, but this is provided only to help a more general understanding of the present invention, the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations are possible from these descriptions.

Accordingly, the spirit of the present invention should not be limited to the described embodiments, and all of the equivalents or equivalents of the claims as well as the claims to be described later will belong to the scope of the present invention. .

Claims (14)

An electronic device that performs a calculation using a deep learning model,
When input data is received, an operation is performed on the input data using a deep learning model, and when the operation is performed on the input data, the data is pre-assigned to at least one layer among a plurality of layers included in the deep learning model. A layer manager for identifying the milestone identifier;
In response to the milestone identifier being identified, measuring and storing the time required from the start of the operation on the input data to the time point when the milestone identifier is identified, and using the stored time and the predetermined target time delay value Performance verification unit to obtain;
A delay controller configured to determine whether the obtained delay value is within a threshold range based on a preset value, and to generate a control command for adjusting the performance of at least one processor based on the determination result;
A resource manager configured to change a frequency of at least one processor in the electronic device according to the generated control command; And
In response to the termination identifier being identified when the operation on the input data is continued through the at least one processor whose frequency has been changed, the output data corresponding to the received input data until the point at which the termination identifier is identified. An electronic device for performing an operation using a deep learning model, including an input and output unit for providing data.
The method of claim 1,
The layer manager,
Selecting a first plurality of layers used to perform an operation on the input data among the plurality of layers, identifying a type of each of the selected first plurality of layers, and identifying the first plurality of layers The electronic device performs the operation using the deep learning model, characterized in that the operation on the input data is performed using an operation formula corresponding to each type.
The method of claim 1,
A model compressing unit which checks predetermined quality of service (QoS) data and compresses the deep learning model based on the identified QoS data; And
Profiling the deep learning model on which the compression is performed to measure an operation time for each of the plurality of layers in the compression deep learning model, and using the measured operating time, the plurality of layers And a milestone identifier allocator for selecting one or more layers operated at predetermined time intervals and allocating a milestone identifier to each of the selected one or more layers.
The method of claim 1,
The performance check unit,
And calculating, as the delay value, a ratio of the stored required time to the preset target time as the delay value.
The method of claim 4, wherein
The delay control unit,
In response to determining that the obtained delay value is out of the threshold range while being less than the preset value of 1, generate a first control command instructing to downgrade the performance of the at least one processor in the electronic device; ,
In response to determining that the obtained delay value is larger than 1, the predetermined value, and out of the threshold range, generating a second control command instructing to increase the performance of the at least one processor in the electronic device. The electronic device for performing an operation using a deep learning model, characterized in that.
The method of claim 5, wherein
The delay control unit,
Measure a degree to which the obtained delay value is out of the threshold range, and determine a value at which the performance of the at least one processor is adjusted down or up based on the measured amount, and according to the determined value And generating the first control command or the second control command by using a deep learning model.
In the method for controlling an electronic device performing a calculation using a deep learning model,
Performing an operation on the input data using a deep learning model when the input data is received;
Identifying a milestone identifier previously assigned to at least one layer of a plurality of layers included in the deep learning model when the operation on the input data is performed;
In response to the milestone identifier being identified, measuring and storing the time required for the milestone identifier to be identified after the operation on the input data starts;
Obtaining a delay value using the stored time required and a preset target time;
Determining whether the obtained delay value is within a threshold range based on a preset value;
Generating a control command for adjusting the performance of at least one processor based on the determination result;
Changing a frequency of at least one processor in the electronic device according to the generated control command; And
Responsive to identifying an end identifier when the operation on the input data is continued through the at least one processor whose frequency has been changed, outputting data calculated up to a point in time when the end identifier is identified as output data corresponding to the input data. A method of controlling an electronic device performing an operation using a deep learning model, comprising: providing a deep learning model.
The method of claim 7, wherein
Performing an operation on the input data,
Selecting a first plurality of layers used to perform an operation on the input data among the plurality of layers;
Identifying a type of each of the selected first plurality of layers; And
Performing an operation on the input data by using an operation formula corresponding to the type of each of the identified first plurality of layers, wherein the operation is performed using the deep learning model. How to control your device.
The method of claim 7, wherein
Checking predetermined quality of service (QoS) data and performing compression on the deep learning model based on the identified QoS data;
Profiling the deep learning model on which the compression is performed to measure an operation time of each of the plurality of layers in the compression deep learning model;
Selecting one or more layers operated at predetermined time intervals from among the plurality of layers using the measured operating time; And
And allocating a milestone identifier to each of the selected one or more layers.
The method of claim 7, wherein
Acquiring the delay value,
Obtaining a ratio of the stored required time to the preset target time as the delay value. The method of controlling an electronic device using a deep learning model.
The method of claim 10,
Generating the control command,
In response to determining that the obtained delay value is out of the threshold range while being less than the preset value of 1, generating a first control command instructing to downgrade the performance of the at least one processor in the electronic device. step; And
In response to determining that the obtained delay value is larger than 1, the predetermined value, and out of the threshold range, generating a second control command instructing to increase the performance of the at least one processor in the electronic device. And controlling the electronic device performing the operation using the deep learning model.
The method of claim 10,
Generating the control command,
Measuring a degree to which the obtained delay value is outside the threshold range;
Determining a value at which the performance of the at least one processor is down-scaled or up-scaled based on the measured degree; And
Generating the first control command or the second control command according to the determined value. The method of controlling an electronic device using the deep learning model.
A computer-readable recording medium having recorded thereon a program for causing a computer to perform the method of any one of claims 7 to 12.
A computer program stored in a storage medium for executing the method of any one of claims 7 to 12 in combination with a computer.

Images