WO2022030054A1 - Value range determination program, value range determination system, and value range determination method for deep learning model - Google Patents

Value range determination program, value range determination system, and value range determination method for deep learning model Download PDF

Info

Publication number
WO2022030054A1
WO2022030054A1 PCT/JP2021/016965 JP2021016965W WO2022030054A1 WO 2022030054 A1 WO2022030054 A1 WO 2022030054A1 JP 2021016965 W JP2021016965 W JP 2021016965W WO 2022030054 A1 WO2022030054 A1 WO 2022030054A1
Authority
WO
WIPO (PCT)
Prior art keywords
range
deep learning
learning model
layer
procedure
Prior art date
Application number
PCT/JP2021/016965
Other languages
French (fr)
Japanese (ja)
Inventor
希武 田中
ル クオック ヴォン
Original Assignee
コニカミノルタ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by コニカミノルタ株式会社 filed Critical コニカミノルタ株式会社
Priority to JP2022541117A priority Critical patent/JPWO2022030054A1/ja
Publication of WO2022030054A1 publication Critical patent/WO2022030054A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present invention relates to a range determination program, a range determination system, and a range determination method of a deep learning model.
  • edge computing technology has been developed to introduce AI (Artificial Integrity) technology into various familiar devices and the like.
  • Edge computing technology requires low cost and low power consumption, so a quantized deep learning model that can be calculated with a high-speed and small-scale circuit is attracting attention.
  • the quantization deep learning model is lightened by the quantization that converts the parameters expressed by the floating point of 16 to 32 bits in the normal deep learning model to the fixed point of 8 bits or less.
  • the estimation accuracy by the quantized deep learning model may decrease due to the quantization of the parameters, and it changes relatively greatly depending on how the range is set. Therefore, there is a problem that it is difficult to quantize the parameters by setting the range for each layer while suppressing the deterioration of the estimation accuracy.
  • the present invention has been made to solve such a problem. That is, it is an object of the present invention to provide a range determination program, a range determination system, and a range determination method of a deep learning model, which can obtain a highly accurate and lightweight quantization deep learning model by quantizing parameters.
  • the value range is sequentially changed to quantify the parameter, and the loss of the estimation result by the deep learning model based on the input data with respect to the correct answer data is calculated to determine the value range when the loss is minimized.
  • the procedure (b) is a procedure (b1) for provisionally setting the range in the quantization of the parameter of a predetermined target layer of the deep learning model, and the procedure (b1) for setting the parameter of the target layer. ), And the procedure (b2) for learning the target layer or the designated layer by using the learning teacher data after quantization in the range temporarily set in the above procedure, and the deep learning learned in the procedure (b2).
  • the procedure (b3) for calculating the loss for the correct answer data of the estimation result based on the input data by the model and the procedure (b2) and the procedure (b3) are executed by sequentially changing the temporarily set range.
  • the above-mentioned (2) which comprises a procedure (b4) for determining the range when the loss calculated in the procedure (b3) is minimized in the temporarily set range. Range determination program for deep learning models.
  • the procedure (b1) to the procedure (b4) are executed with one lower layer of the target layer as a new target layer, and the range of the parameter of the new target layer is executed.
  • the deep learning model according to (3) above further comprising a procedure (b5) for determining the above procedure (b5) and a procedure (b6) for executing the procedure (b5) until the new target layer becomes the lowest layer. Range determination program.
  • the parameter of each layer of the deep learning model includes a weight and a bias, and the procedure (b2) fixes the bias and the procedure (b21) for learning the weight and the weight.
  • the range determination program for the deep learning model according to any one of (3) to (6) above, which comprises the procedure (b22) for learning the bias.
  • the parameters of each layer of the deep learning model include weight, bias, and output, and in the procedure (b4), the range of the bias is the range of the weight of the same layer as the layer of the bias.
  • the range determination program for the deep learning model according to any one of (3) to (7) above, which is calculated based on the range of the output of the layer above the layer of the bias.
  • the procedure (b4) makes the range of the convolution layer the same as the range of the relu layer.
  • a system for determining the value range in the quantization of the parameters of the deep learning model the acquisition unit for acquiring the input data and the correct answer data corresponding to the input data, and the value range in the quantization of the parameters.
  • a calculation unit that sequentially changes and quantizes the parameters, calculates the loss of the estimation result by the deep learning model based on the input data with respect to the correct answer data, and determines the value range when the loss is minimized. And, the value range determination system of the deep learning model with.
  • the value range in the quantization is sequentially changed to quantify the parameter, and the loss of the estimation result by the deep learning model based on the input data with respect to the correct answer data is calculated to minimize the loss.
  • a method for determining a price range of a deep learning model comprising the step (b) of determining the price range.
  • the range in the quantization of the parameters of the deep learning model is sequentially changed to quantize the parameters, and the loss for the correct answer data of the estimation result by the deep learning model based on the input data is calculated, and the range when the loss is minimized. To determine.
  • a highly accurate and lightweight quantized deep learning model can be obtained by quantizing the parameters.
  • FIG. 1 is a diagram showing a schematic configuration of a range determination system 10 of a deep learning model according to an embodiment.
  • the range determination system 10 includes a range determination device 100, a photographing device 200, and a communication network 300.
  • the range determination device 100 is communicably connected to the photographing device 200 by the communication network 300.
  • the object is an object whose position or partial position is estimated by the estimation unit 112 (see FIGS. 3 and 8) of the range determination device 100.
  • Objects can contain multiple categories.
  • a category is a type of object and includes people, dogs, cats, and the like.
  • the object will be described as being the subject 500 (that is, a “person”).
  • the range determination device 100 estimates the position or partial position of an object in an image input to the range determination device 100 (hereinafter, also simply referred to as “input image 250”), and is a parameter of a deep learning model of the estimation unit 112 (hereinafter, also referred to as “input image 250”). Determines the range in the quantization of (also called "model parameter").
  • the input image 250 may be an image taken by the photographing device 200.
  • the input image 250 constitutes the input data.
  • Quantization means, for example, converting a model parameter expressed by a floating point number of 16 to 32 bits into a fixed point number of 8 bits or less.
  • model parameters include input, weight, bias, and output for each layer of the deep learning model.
  • the estimation unit 112 of the range determination device 100 estimates the position of the joint point 251 of the subject 500, which is an example of the partial position of the object, as the coordinates on the input image 250. Will be described as an example.
  • the positions of the joint points 251 and the like are also simply referred to as the joint points 251 and the like.
  • the range determination device 100 can be included in the photographing device 200 by being integrated with the photographing device 200.
  • the photographing device 200 is arranged, for example, on the ceiling or the upper part of the wall of the living room of the subject 500, or is arranged on the ceiling or the upper part of the wall of a store or the like.
  • the photographing device 200 photographs a predetermined photographing range from a position where the target person 500 can be overlooked, and outputs an input image 250 (image data).
  • the input image 250 includes an image including the subject 500.
  • the photographing apparatus 200 includes a wide-angle camera.
  • the photographing device 200 may photograph a photographing area as an input image 250 of a moving image having a frame rate of, for example, 15 fps to 30 fps.
  • the input image 250 includes a moving image and a still image.
  • the input image 250 is, for example, a black-and-white image and may be an image of 128 pixels ⁇ 128 pixels.
  • the input image 250 may be a color image.
  • the photographing device 200 transmits the input image 250 to the range determining device 100 and the like.
  • a network interface based on a wired communication standard such as Ethernet (registered trademark) can be used.
  • a network interface based on a wireless communication standard such as Bluetooth (registered trademark) or 802.11 may be used.
  • FIG. 2 is a block diagram showing a hardware configuration of the range determination device 100.
  • the range determination device 100 includes a control unit 110, a storage unit 120, a display unit 130, an input unit 140, and a communication unit 150. These components are connected to each other via the bus 160.
  • the range determination device 100 may be configured by a computer.
  • the control unit 110 is configured by a CPU (Central Processing Unit), and controls and performs arithmetic processing of each unit of the range determination device 100 according to a program. The details of the operation of the control unit 110 will be described later.
  • CPU Central Processing Unit
  • the storage unit 120 may be composed of a RAM (Random Access Memory), a ROM (Read Only Memory), and an SSD (Solid State Drive).
  • the RAM temporarily stores programs and data as a work area of the control unit 110.
  • the ROM stores various programs and various data in advance.
  • the SSD stores various programs including an operating system and various data.
  • the display unit 130 is, for example, a liquid crystal display and displays various information.
  • the input unit 140 is composed of, for example, a touch panel and various keys.
  • the input unit 140 is used for various operations and inputs.
  • the communication unit 150 is an interface for communicating with an external device.
  • a network interface according to standards such as Ethernet (registered trademark), SATA, PCI Express, USB, and IEEE 1394 can be used.
  • a wireless communication interface such as Bluetooth (registered trademark), 802.11, or 4G may be used for communication.
  • control unit 110 The details of the operation of the control unit 110 will be described.
  • FIG. 3 is a block diagram showing an example of the function of the control unit 110 of the range determination device 100.
  • the control unit 110 functions as an acquisition unit 111, an estimation unit 112, a range temporary setting unit 113, a loss calculation unit 114, and a range determination unit 115.
  • the estimation unit 112 can be configured by using a deep learning model, which is a neural network.
  • the estimation unit 112, the range temporary setting unit 113, the loss calculation unit 114, and the range determination unit 115 constitute a calculation unit.
  • the acquisition unit 111 acquires the input image 250 and the correct answer data of the joint point 251 corresponding to the input image 250.
  • FIG. 4 is an explanatory diagram showing the input image 250.
  • FIG. 5 is an explanatory diagram showing a joint point 251.
  • the input image 250 includes an image of the target person 500.
  • Joint points 251 include, for example, right ankle, right knee, right waist, left waist, left knee, left ankle, right wrist, right elbow, right shoulder, left wrist, left elbow, left shoulder, neck, and head. Includes joint point 251.
  • the estimation unit 112 quantizes the model parameter expressed by a 32-bit floating point number in the range temporarily set by the range temporary setting unit 113 by making it a value that can be expressed by an 8-bit fixed point number. As will be described later, the range temporary setting unit 113 sequentially temporarily sets different range. The estimation unit 112 sequentially quantizes the model parameters in each range temporarily set by the range temporary setting unit 113.
  • the model parameters expressed in 32-bit floating point before being quantized use a large amount of teacher data (combination of an image and correct answer data corresponding to the image) by a relatively high-performance computer. It can be a model parameter generated by learning in advance.
  • the estimation unit 112 can quantize the model parameters for each layer in the range temporarily set for each layer of the deep learning model by the range temporary setting unit 113.
  • the estimation unit 112 may quantize the model parameters for each model parameter in the range temporarily set for each model parameter of the deep learning model by the range temporary setting unit 113.
  • the estimation unit 112 will be described as quantizing the model parameters for each layer in the range temporarily set for each layer of the deep learning model.
  • the estimation unit 112 quantizes the model parameters for each layer of the deep learning model in the range temporarily set by the range temporary setting unit 113, and estimates the joint point 251 by the deep learning model based on the input image 250.
  • the range temporary setting unit 113 temporarily changes and temporarily sets the range in the quantization of the model parameter.
  • FIG. 6 is an explanatory diagram for explaining the range in the quantization of the model parameters, which are temporarily set sequentially.
  • the horizontal axis of each graph in FIG. 6 shows the values of the model parameters, and the vertical axis shows the frequency.
  • the frequency on the vertical axis is the frequency of the values taken by the model parameters when learning the deep learning model.
  • 2 n , 2 n-1 , and 2 n-2 shown by broken lines represent a range (more specifically, the upper limit of the range).
  • the exponent n is 8
  • the range temporary setting unit 113 can temporarily set the range in the quantization of the model parameter in the order of 2 n , 2 n-1 , and 2 n-2 .
  • the loss calculation unit 114 calculates the loss of the estimation result of the joint point 251 based on the input image 250 by the estimation unit 112 with respect to the correct answer data.
  • the loss can be, for example, the squared average of the difference (distance) between the joint point 251 and the correct answer data calculated for all the joint points 251.
  • the loss calculation unit 114 calculates the loss in the temporarily set range for each layer of the deep learning model.
  • the range determination unit 115 determines the range when the loss calculated by the loss calculation unit 114 is minimized for each layer of the deep learning model, and stores it in the storage unit 120 as a parameter of the range.
  • FIG. 7 is an explanatory diagram showing the transition in which the range in the quantization of the model parameters is determined for each layer of the deep learning model.
  • the figure on the left side of FIG. 7 shows a state in which the range is determined by quantizing the layer 1 which is the uppermost layer of the deep learning model.
  • the figure on the right shows the state in which the range is determined by quantizing the uppermost layer of the deep learning model, Layer1, and then quantizing the uppermost layer of Layer2, which is one layer below. ing.
  • the deep learning model can be quantized layer by layer from the top layer to the bottom layer, and the range can be determined.
  • the quantization of the lower layer is executed in the state where the quantization of the upper layer is executed.
  • the deep learning model may be quantized and the range may be determined only for a predetermined target layer.
  • the predetermined target layer may be a plurality of target layers. Further, the range may be determined by quantizing only the target layer set by inputting from the input unit 140 by the user or the like.
  • the deep learning model may be quantized one layer at a time from a predetermined target layer toward the lowest layer, and the range may be determined.
  • the predetermined target layer may be the uppermost layer of the deep learning model, or may be a layer other than the uppermost layer. That is, the initial value of the predetermined target layer may be the uppermost layer or may be a layer other than the uppermost layer.
  • the range determination unit 115 can calculate (determine) the range of bias, which is a model parameter, based on the range of weight of the same layer as the layer of bias and the range of output of one layer above the layer of bias.
  • the range determination unit 115 can determine the range of the convolution layer to be the same as the range of the relu layer.
  • FIG. 8 is a block diagram showing another example of the function of the control unit 110 of the range determination device 100.
  • the control unit 110 can function as an acquisition unit 111, an estimation unit 112, a range temporary setting unit 113, a loss calculation unit 114, a range determination unit 115, and a learning loss calculation unit 116.
  • the acquisition unit 111 acquires the input image 250 and the correct answer data of the joint point 251 corresponding to the input image 250.
  • the acquisition unit 111 further acquires a learning input image and learning correct answer data of the joint point 251 corresponding to the learning input image.
  • As the input image for learning and the correct answer data for learning the same input image 250 and the correct answer data may be used, respectively.
  • the combination of the input image for learning and the correct answer data for learning constitutes the teacher data for learning.
  • the estimation unit 112 quantizes the model parameter expressed by a 32-bit floating point number in the range temporarily set by the range temporary setting unit 113 by making it a value that can be expressed by an 8-bit fixed point number.
  • the estimation unit 112 sequentially quantizes the model parameters in each range temporarily set for each layer of the deep learning model by the range temporary setting unit 113, and uses the learning teacher data to obtain the deep learning model as follows. To learn.
  • the learning loss calculation unit 116 calculates the learning loss for the learning correct answer data of the estimation result of the joint point 251 based on the learning input image by the estimation unit 112.
  • the learning loss is the difference (distance) between the joint point 251 and the correct answer data for learning, and is calculated for each joint point 251.
  • the learning loss calculation unit 116 calculates the learning loss in the temporarily set range for each layer of the deep learning model.
  • the estimation unit 112 learns the deep learning model by backpropagation so that the learning loss calculated for each joint point 251 by the learning loss calculation unit 116 becomes 0.
  • the estimation unit 112 quantizes the model parameters for each layer of the deep learning model in the value range temporarily set by the value range temporary setting unit 113, and learns the deep learning model using the learning teacher data (that is, the deep). Along with learning all layers of the learning model), the deep learning model after learning estimates the joint points 251 based on the input image 250. As a result, the model parameters are fine-tuned for each layer during the layer-by-layer quantization of the deep learning model. Even if the estimation unit 112 quantizes the model parameters for each layer of the deep learning model in the range temporarily set by the range temporary setting unit 113 and learns for each quantized layer (updates of the model parameters). good. That is, quantization and learning may be performed for each predetermined layer. In this case, the layers other than the predetermined layer are learned without updating the model parameters.
  • the range temporary setting unit 113 temporarily changes and temporarily sets the range in the quantization of the model parameter.
  • the loss calculation unit 114 calculates the loss of the estimation result of the joint point 251 based on the input image 250 by the estimation unit 112 with respect to the correct answer data.
  • the loss can be, for example, the squared average of the difference (distance) between the joint point 251 and the correct answer data calculated for all the joint points 251.
  • the loss calculation unit 114 calculates the loss in the temporarily set range for each layer of the deep learning model.
  • the range determination unit 115 determines the range when the loss calculated by the loss calculation unit 114 is minimized for each layer of the deep learning model.
  • the deep learning model may be quantized and trained to determine the range only for a predetermined target layer.
  • the deep learning model may be quantized and trained layer by layer from a predetermined target layer toward the bottom layer to determine a range.
  • the predetermined target layer may be the uppermost layer of the deep learning model, or may be a layer other than the uppermost layer. That is, the initial value of the predetermined target layer may be the uppermost layer or may be a layer other than the uppermost layer.
  • Learning for each layer of the deep learning model can be performed in the following order.
  • the model parameters of bias are fixed and weight is learned. After that, the weight is fixed and the bias is learned.
  • the range of model parameters is determined in this way, and the learning unit 112 learns that the joint point 251 is detected with high accuracy based on the input image 250.
  • the estimation unit 112 estimates the joint point 251 by reflecting the model parameters after learning stored in the storage unit 120 in the deep learning model. Further, the estimation unit 112 reads out the parameters of the range for each layer stored in the storage unit 120, and quantizes the model parameters for each layer based on the parameters of the range.
  • FIG. 9 is a flowchart showing the operation of the range determination device 100. This flowchart is executed by the control unit 110 according to the program stored in the storage unit 120.
  • the control unit 110 acquires the input image 250 and the correct answer data of the joint point 251 corresponding to the input image 250 (S101).
  • the control unit 110 can acquire the input image 250 and the correct answer data of the joint point 251 by reading from the storage unit 120.
  • the control unit 110 may acquire the input image 250 and the correct answer data of the joint point 251 by receiving the correct answer data by the communication unit 150.
  • the control unit 110 temporarily sets the range in the quantization of the target layer of the deep learning model (S102).
  • the control unit 110 learns the target layer using the teacher data for learning with the model parameters quantized in the temporarily set range (S103).
  • the control unit 110 estimates the joint point 251 from the input image in the deep learning model after learning (S104).
  • the control unit 110 calculates the loss of the estimated joint point 251 with respect to the correct answer data (S105).
  • the control unit 110 determines whether or not the loss has been calculated for all the ranges (S106). If the control unit 110 determines that the loss has been calculated for all the range (S106: YES), the control unit 110 executes step S107. If the control unit 110 determines that the loss has not been calculated for all the range (S106: NO), the control unit 110 executes step S102.
  • the control unit 110 determines the temporarily set range when the loss is minimized as the range of the target layer (S107).
  • the control unit 110 determines whether or not the range has been determined for all layers of the deep learning model (S108). When the control unit 110 determines that the range has been determined for all the layers of the deep learning model (S108: YES), the control unit 110 ends the process. When the control unit 110 determines that the range has not been determined for all the layers of the deep learning model (S108: NO), the target layer is set to one layer lower layer, and steps S102 to S107 are executed.
  • This embodiment has the following effects.
  • the range in the quantization of the parameters of the deep learning model is sequentially changed to quantize the parameters, and the loss for the correct answer data of the estimation result by the deep learning model based on the input data is calculated, and the range when the loss is minimized. To determine.
  • a highly accurate and lightweight quantized deep learning model can be obtained by quantizing the parameters.
  • the value range in the quantization of the model parameters is sequentially changed, the parameters are quantized, the deep learning model is learned using the training teacher data, and the correct answer of the estimation result based on the input data by the deep learning model after learning. Calculate the loss for the data to determine the value range when the loss is minimal. As a result, the estimation accuracy by the quantized deep learning model can be further improved.
  • the range in the quantization of the parameters of the deep learning model is sequentially changed, the parameters are quantized, the target layer or the specified layer is learned, and the estimation result by the deep learning model based on the input data is obtained. Calculate the loss for the correct answer data of, and determine the range when the loss is minimized. As a result, it is possible to obtain a highly accurate and lightweight quantized deep learning model more easily and flexibly.
  • the range in the quantization of the parameters of the deep learning model is sequentially changed for each layer from the predetermined target layer toward the bottom layer to quantize the parameters, and the estimation result by the deep learning model based on the input data is obtained. Calculate the loss for the correct data and determine the range when the loss is minimized. As a result, the quantization error of the upper layer is absorbed in the lower layer of the deep learning model, so that a highly accurate and lightweight quantization deep learning model can be obtained more effectively.
  • the initial value of the target layer is set to the top layer. This makes it possible to obtain a highly accurate and lightweight quantized deep learning model more effectively.
  • the initial value of the target layer is set to a layer other than the top layer. This makes it possible to obtain a highly accurate and lightweight quantized deep learning model effectively and flexibly.
  • learning for each layer of the deep learning model is performed in the following order.
  • the model parameters of bias are fixed and weight is learned. After that, the weight is fixed and the bias is learned. As a result, the estimation accuracy by the quantized deep learning model can be further improved.
  • the range of bias is calculated based on the range of weight of the same layer as the layer of bias and the range of output of one layer above the layer of bias. This makes it possible to obtain a highly accurate and lightweight quantized deep learning model more easily without changing the existing hardware configuration.
  • the range of the convolution layer is made the same as the range of the relu layer. This makes it possible to obtain a highly accurate and lightweight quantized deep learning model more easily in a specific case.
  • the imaging device 200 having a built-in computer may have the function of the range determination device 100.
  • the range determination device 100 and the photographing device 200 may each be configured by a plurality of devices, or any plurality of the devices may be configured as a single device.
  • conversion of a model parameter expressed by a 32-bit floating point number to an 8-bit fixed point number has been described as an example of quantization, but for quantization, a 32-bit floating point number is used. It also includes converting the represented model parameters to floating point numbers less than 32 bits.
  • the means and methods for performing various processes in the range determination system 10 and the range determination device 100 described above can be realized by either a dedicated hardware circuit or a programmed computer.
  • the program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versaille Disc) -ROM, or may be provided online via a network such as the Internet.
  • the program recorded on the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk.
  • the above program may be provided as a single application software, or may be incorporated into the software of a device such as a photographing device as a function.
  • range determination system 100 range determination device, 110 Control unit, 111 acquisition part, 112 Estimator, 113 Range temporary setting section, 114 Loss calculation unit, 115 Range Determination, 116 Learning loss calculation unit, 120 storage, 130 Communication Department, 140 input section, 150 communication department, 200 shooting equipment, 250 input image, 251 Joint Point, 500 Target people.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

[Problem] To provide a value range determination program for a deep learning model with which a lightweight and highly precise quantization deep learning model can be obtained through quantization of parameters. [Solution] A program for determining a value range in the quantization of parameters in a deep learning model, said value range determination program having a procedure (a) for acquiring input data and correct answer data corresponding to the input data, and a procedure (b) for quantizing parameters while sequentially changing the value range in the quantization of the parameters, and determining a value range when the loss is the smallest for an estimation result by a deep learning model based on the input data with respect to the correct answer data.

Description

ディープラーニングモデルの値域決定プログラム、値域決定システム、および値域決定方法Range determination program, range determination system, and range determination method for deep learning models
　本発明は、ディープラーニングモデルの値域決定プログラム、値域決定システム、および値域決定方法に関する。 The present invention relates to a range determination program, a range determination system, and a range determination method of a deep learning model.
　近年、身近な様々な装置等にＡＩ（Ａｒｔｉｆｉｃｉａｌ　Ｉｎｔｅｌｌｉｇｅｎｃｅ）技術を導入しようとするエッジコンピューティング技術が開発されている。 In recent years, edge computing technology has been developed to introduce AI (Artificial Integrity) technology into various familiar devices and the like.
　エッジコンピューティング技術では、低コストおよび低消費電力等が要求されるため、高速かつ小規模な回路で演算可能な量子化ディープラーニングモデルが注目されている。 Edge computing technology requires low cost and low power consumption, so a quantized deep learning model that can be calculated with a high-speed and small-scale circuit is attracting attention.
　量子化ディープラーニングモデルは、通常のディープラーニングモデルにおいて１６～３２ビットの浮動小数点で表現されるパラメーターを、８ビット以下の固定小数点に変換する量子化により軽量化されている。 The quantization deep learning model is lightened by the quantization that converts the parameters expressed by the floating point of 16 to 32 bits in the normal deep learning model to the fixed point of 8 bits or less.
　しかし、量子化ディープラーニングモデルによる推定精度は、パラメーターの量子化により低下する可能性があり、また値域の設定の仕方により比較的大きく変化する。このため、推定精度の低下を抑止しつつ各層ごとの値域を設定してパラメーターを量子化することが困難であるという問題がある。 However, the estimation accuracy by the quantized deep learning model may decrease due to the quantization of the parameters, and it changes relatively greatly depending on how the range is set. Therefore, there is a problem that it is difficult to quantize the parameters by setting the range for each layer while suppressing the deterioration of the estimation accuracy.
　本発明は、このような問題を解決するためになされたものである。すなわち、パラメーターの量子化により、高精度かつ軽量な量子化ディープラーニングモデルが得られる、ディープラーニングモデルの値域決定プログラム、値域決定システム、および値域決定方法を提供することを目的とする。 The present invention has been made to solve such a problem. That is, it is an object of the present invention to provide a range determination program, a range determination system, and a range determination method of a deep learning model, which can obtain a highly accurate and lightweight quantization deep learning model by quantizing parameters.
　本発明の上記課題は、以下の手段によって解決される。 The above problem of the present invention is solved by the following means.
　（１）ディープラーニングモデルのパラメーターの量子化における値域を決定するプログラムであって、入力データと、前記入力データに対応する正解データとを取得する手順（ａ）と、前記パラメーターの量子化における前記値域を順次変えて、前記パラメーターを量子化するとともに、前記入力データに基づく前記ディープラーニングモデルによる推定結果の前記正解データに対するロスを算出して、前記ロスが最小になるときの前記値域を決定する手順（ｂ）と、を有する処理を、コンピューターに実行させるためのディープラーニングモデルの値域決定プログラム。 (1) A program for determining the value range in the quantization of the parameters of the deep learning model, the procedure (a) for acquiring the input data and the correct answer data corresponding to the input data, and the above-mentioned in the quantization of the parameters. The value range is sequentially changed to quantify the parameter, and the loss of the estimation result by the deep learning model based on the input data with respect to the correct answer data is calculated to determine the value range when the loss is minimized. A value range determination program of a deep learning model for causing a computer to execute a process having the procedure (b).
　（２）前記手順（ｂ）は、前記パラメーターの量子化における前記値域を順次変えて、前記パラメーターを量子化して、前記ディープラーニングモデルを学習用教師データを用いて学習するとともに、学習後の前記ディープラーニングモデルによる前記入力データに基づく推定結果の前記正解データに対する前記ロスを算出して、前記ロスが最小になるときの前記値域を決定する、上記（１）に記載のディープラーニングモデルの値域決定プログラム。 (2) In the procedure (b), the value range in the quantization of the parameter is sequentially changed, the parameter is quantized, the deep learning model is learned using the learning teacher data, and the learning is performed. The value range determination of the deep learning model according to (1) above, wherein the loss for the correct answer data of the estimation result based on the input data by the deep learning model is calculated to determine the value range when the loss is minimized. program.
　（３）前記手順（ｂ）は、前記ディープラーニングモデルの所定の対象層の前記パラメーターの量子化における前記値域を仮設定する手順（ｂ１）と、前記対象層の前記パラメーターを、前記手順（ｂ１）において仮設定された前記値域で量子化して、前記学習用教師データを用いて、前記対象層または指定した層を学習する手順（ｂ２）と、前記手順（ｂ２）において学習された前記ディープラーニングモデルによる前記入力データに基づく推定結果の前記正解データに対する前記ロスを算出する手順（ｂ３）と、前記仮設定した前記値域を順次変えて、前記手順（ｂ２）および前記手順（ｂ３）を実行して、前記仮設定した前記値域の中で、前記手順（ｂ３）において算出された前記ロスが最小になるときの前記値域を決定する手順（ｂ４）と、を有する、上記（２）に記載のディープラーニングモデルの値域決定プログラム。 (3) The procedure (b) is a procedure (b1) for provisionally setting the range in the quantization of the parameter of a predetermined target layer of the deep learning model, and the procedure (b1) for setting the parameter of the target layer. ), And the procedure (b2) for learning the target layer or the designated layer by using the learning teacher data after quantization in the range temporarily set in the above procedure, and the deep learning learned in the procedure (b2). The procedure (b3) for calculating the loss for the correct answer data of the estimation result based on the input data by the model and the procedure (b2) and the procedure (b3) are executed by sequentially changing the temporarily set range. The above-mentioned (2), which comprises a procedure (b4) for determining the range when the loss calculated in the procedure (b3) is minimized in the temporarily set range. Range determination program for deep learning models.
　（４）前記手順（ｂ）は、前記対象層の１層下層を新たな対象層として、前記手順（ｂ１）～前記手順（ｂ４）を実行して、前記新たな対象層の前記パラメーターの値域を決定する手順（ｂ５）と、前記手順（ｂ５）を、前記新たな対象層が最下層となるまで実行する手順（ｂ６）と、をさらに有する、上記（３）に記載のディープラーニングモデルの値域決定プログラム。 (4) In the procedure (b), the procedure (b1) to the procedure (b4) are executed with one lower layer of the target layer as a new target layer, and the range of the parameter of the new target layer is executed. The deep learning model according to (3) above, further comprising a procedure (b5) for determining the above procedure (b5) and a procedure (b6) for executing the procedure (b5) until the new target layer becomes the lowest layer. Range determination program.
　（５）前記（ｂ１）における前記対象層の初期値は最上層である、上記（４）に記載のディープラーニングモデルの値域決定プログラム。 (5) The range determination program of the deep learning model according to (4) above, wherein the initial value of the target layer in (b1) is the uppermost layer.
　（６）前記（ｂ１）における前記対象層の初期値は最上層以外の層である、上記（４）に記載のディープラーニングモデルの値域決定プログラム。 (6) The range determination program of the deep learning model according to (4) above, wherein the initial value of the target layer in the above (b1) is a layer other than the uppermost layer.
　（７）前記ディープラーニングモデルの各層の前記パラメーターは、ｗｅｉｇｈｔ、およびｂｉａｓを含み、前記手順（ｂ２）は、前記ｂｉａｓを固定して、前記ｗｅｉｇｈｔを学習する手順（ｂ２１）と、前記ｗｅｉｇｈｔを固定して、前記ｂｉａｓを学習する手順（ｂ２２）と、を有する、上記（３）～（６）のいずれかに記載のディープラーニングモデルの値域決定プログラム。 (7) The parameter of each layer of the deep learning model includes a weight and a bias, and the procedure (b2) fixes the bias and the procedure (b21) for learning the weight and the weight. The range determination program for the deep learning model according to any one of (3) to (6) above, which comprises the procedure (b22) for learning the bias.
　（８）前記ディープラーニングモデルの各層の前記パラメーターは、ｗｅｉｇｈｔ、ｂｉａｓ、およびｏｕｔｐｕｔを含み、前記手順（ｂ４）は、前記ｂｉａｓの前記値域を、前記ｂｉａｓの層と同じ層の前記ｗｅｉｇｈｔの前記値域と、前記ｂｉａｓの層の１層上層の前記ｏｕｔｐｕｔの前記値域とに基づいて算出する、上記（３）～（７）のいずれかに記載のディープラーニングモデルの値域決定プログラム。 (8) The parameters of each layer of the deep learning model include weight, bias, and output, and in the procedure (b4), the range of the bias is the range of the weight of the same layer as the layer of the bias. The range determination program for the deep learning model according to any one of (3) to (7) above, which is calculated based on the range of the output of the layer above the layer of the bias.
　（９）畳み込み層の下層がｒｅｌｕ層である前記ディープラーニングモデルの場合、前記手順（ｂ４）は、前記畳み込み層の前記値域を前記ｒｅｌｕ層の前記値域と同一にする、上記（３）～（８）のいずれかに記載のディープラーニングモデルの値域決定プログラム。 (9) In the case of the deep learning model in which the lower layer of the convolution layer is the relu layer, the procedure (b4) makes the range of the convolution layer the same as the range of the relu layer. The range determination program for the deep learning model described in any of 8).
　（１０）ディープラーニングモデルのパラメーターの量子化における値域を決定するシステムであって、入力データと、前記入力データに対応する正解データとを取得する取得部と、前記パラメーターの量子化における前記値域を順次変えて、前記パラメーターを量子化するとともに、前記入力データに基づく前記ディープラーニングモデルによる推定結果の前記正解データに対するロスを算出して、前記ロスが最小になるときの前記値域を決定する演算部と、を有するディープラーニングモデルの値域決定システム。 (10) A system for determining the value range in the quantization of the parameters of the deep learning model, the acquisition unit for acquiring the input data and the correct answer data corresponding to the input data, and the value range in the quantization of the parameters. A calculation unit that sequentially changes and quantizes the parameters, calculates the loss of the estimation result by the deep learning model based on the input data with respect to the correct answer data, and determines the value range when the loss is minimized. And, the value range determination system of the deep learning model with.
　（１１）ディープラーニングモデルのパラメーターの量子化における値域を、値域決定システムにより決定する方法であって、入力データと、前記入力データに対応する正解データとを取得する段階（ａ）と、前記パラメーターの量子化における前記値域を順次変えて、前記パラメーターを量子化するとともに、前記入力データに基づく前記ディープラーニングモデルによる推定結果の前記正解データに対するロスを算出して、前記ロスが最小になるときの前記値域を決定する段階（ｂ）と、を有するディープラーニングモデルの値域決定方法。 (11) A method of determining the value range in the quantization of the parameters of the deep learning model by the value range determination system, in which the input data and the correct answer data corresponding to the input data are acquired, and the parameter. When the value range in the quantization is sequentially changed to quantify the parameter, and the loss of the estimation result by the deep learning model based on the input data with respect to the correct answer data is calculated to minimize the loss. A method for determining a price range of a deep learning model, comprising the step (b) of determining the price range.
　ディープラーニングモデルのパラメーターの量子化における値域を順次変えて、パラメーターを量子化するとともに、入力データに基づくディープラーニングモデルによる推定結果の正解データに対するロスを算出して、ロスが最小になるときの値域を決定する。これにより、パラメーターの量子化によって、高精度かつ軽量な量子化ディープラーニングモデルを得ることができる。 The range in the quantization of the parameters of the deep learning model is sequentially changed to quantize the parameters, and the loss for the correct answer data of the estimation result by the deep learning model based on the input data is calculated, and the range when the loss is minimized. To determine. As a result, a highly accurate and lightweight quantized deep learning model can be obtained by quantizing the parameters.
ディープラーニングモデルの値域決定システムの概略構成を示す図である。It is a figure which shows the schematic structure of the range determination system of a deep learning model. 値域決定装置のハードウェア構成を示すブロック図である。It is a block diagram which shows the hardware composition of the range determination apparatus. 値域決定装置の制御部の機能の例を示すブロック図である。It is a block diagram which shows the example of the function of the control part of a range determination apparatus. 入力画像を示す説明図である。It is explanatory drawing which shows the input image. 関節点を示す説明図である。It is explanatory drawing which shows the joint point. 順次仮設定される、モデルパラメーターの量子化における値域を説明するための説明図である。It is explanatory drawing for demonstrating the range in the quantization of a model parameter which is tentatively set sequentially. ディープラーニングモデルの層ごとに、モデルパラメーターの量子化における値域が決定される推移を示す説明図である。It is explanatory drawing which shows the transition that the range is determined in the quantization of a model parameter for each layer of a deep learning model. 値域決定装置の制御部の機能の他の例を示すブロック図である。It is a block diagram which shows another example of the function of the control part of a range determination apparatus. 値域決定装置の動作を示すフローチャートである。It is a flowchart which shows the operation of the range determination apparatus.
　以下、図面を参照して、本発明の実施形態に係る、ディープラーニングモデルの値域決定プログラム、値域決定システム、および値域決定方法について説明する。なお、図面において、同一の要素には同一の符号を付し、重複する説明を省略する。また、図面の寸法比率は、説明の都合上誇張されており、実際の比率とは異なる場合がある。 Hereinafter, the range determination program, the range determination system, and the range determination method of the deep learning model according to the embodiment of the present invention will be described with reference to the drawings. In the drawings, the same elements are designated by the same reference numerals, and duplicate description will be omitted. In addition, the dimensional ratios in the drawings are exaggerated for convenience of explanation and may differ from the actual ratios.
　図１は、実施形態に係る、ディープラーニングモデルの値域決定システム１０の概略構成を示す図である。 FIG. 1 is a diagram showing a schematic configuration of a range determination system 10 of a deep learning model according to an embodiment.
　値域決定システム１０は、値域決定装置１００、撮影装置２００、および通信ネットワーク３００を備える。値域決定装置１００は、通信ネットワーク３００により撮影装置２００と相互に通信可能に接続される。本明細書において、物体とは、値域決定装置１００の推定部１１２（図３、図８参照）により位置または部分位置が推定される対象の物体である。物体には複数のカテゴリーが含まれ得る。カテゴリーとは物体の種別であり、人、犬、猫等が含まれる。以下、説明を簡単にするために、物体は対象者５００（すなわち、「人」）であるものとして説明する。値域決定装置１００は、値域決定装置１００に入力される画像（以下、単に「入力画像２５０」とも称する）における物体の位置または部分位置を推定する、推定部１１２のディープラーニングモデルのパラメーター（以下、「モデルパラメーター」とも称する）の量子化における値域を決定する。入力画像２５０は、撮影装置２００で撮影された画像であり得る。入力画像２５０は入力データを構成する。量子化とは、例えば、１６～３２ビットの浮動小数点で表現されたモデルパラメーターを、８ビット以下の固定小数点に変換することをいう。以下、説明を簡単にするために、３２ビットの浮動小数点で表現されたモデルパラメーターを、８ビットの固定小数点で表現可能な値に変換することが量子化であるものとして説明する。例えば、量子化後の値は、８ビットの固定小数点で表現可能な値であれば、８ビットより高ビットの浮動小数点で表現された値であってもよい。また、予め学習されることで、３２ビットの浮動小数点で表現されたモデルパラメーターが、量子化により、８ビットの固定小数点で表現可能なモデルパラメーターに変換されることを前提に説明する。モデルパラメーターには、ディープラーニングモデルの各層のｉｎｐｕｔ、ｗｅｉｇｈｔ、ｂｉａｓ、およびｏｕｔｐｕｔが含まれる。また、説明を簡単にするために、値域決定装置１００の推定部１１２は、物体の部分位置の例である、対象者５００の関節点２５１の位置を、入力画像２５０上の座標として推定する場合を例に説明する。以下、関節点２５１等の位置を、単に関節点２５１等とも称する。 The range determination system 10 includes a range determination device 100, a photographing device 200, and a communication network 300. The range determination device 100 is communicably connected to the photographing device 200 by the communication network 300. In the present specification, the object is an object whose position or partial position is estimated by the estimation unit 112 (see FIGS. 3 and 8) of the range determination device 100. Objects can contain multiple categories. A category is a type of object and includes people, dogs, cats, and the like. Hereinafter, for the sake of simplicity, the object will be described as being the subject 500 (that is, a “person”). The range determination device 100 estimates the position or partial position of an object in an image input to the range determination device 100 (hereinafter, also simply referred to as “input image 250”), and is a parameter of a deep learning model of the estimation unit 112 (hereinafter, also referred to as “input image 250”). Determines the range in the quantization of (also called "model parameter"). The input image 250 may be an image taken by the photographing device 200. The input image 250 constitutes the input data. Quantization means, for example, converting a model parameter expressed by a floating point number of 16 to 32 bits into a fixed point number of 8 bits or less. Hereinafter, for the sake of simplicity, the conversion of a model parameter expressed by a 32-bit floating point number into a value that can be expressed by an 8-bit fixed point number will be described as quantization. For example, the value after quantization may be a value expressed by a floating point number higher than 8 bits as long as it is a value that can be expressed by an 8-bit fixed point number. Further, the description will be made on the premise that the model parameters represented by 32-bit floating point numbers are converted into model parameters that can be represented by 8-bit fixed point numbers by quantization in advance. Model parameters include input, weight, bias, and output for each layer of the deep learning model. Further, for simplification of the explanation, the estimation unit 112 of the range determination device 100 estimates the position of the joint point 251 of the subject 500, which is an example of the partial position of the object, as the coordinates on the input image 250. Will be described as an example. Hereinafter, the positions of the joint points 251 and the like are also simply referred to as the joint points 251 and the like.
　値域決定装置１００は、撮影装置２００と一体化されることで、撮影装置２００に包含され得る。 The range determination device 100 can be included in the photographing device 200 by being integrated with the photographing device 200.
　撮影装置２００は、例えば、対象者５００の居室の天井もしくは壁の上部に配置され、または店舗等の天井もしくは壁の上部に配置される。撮影装置２００は、所定の撮影範囲を、対象者５００を俯瞰可能な位置から撮影し、入力画像２５０（画像データ）を出力する。入力画像２５０には対象者５００を含む画像が含まれる。撮影装置２００には、広角カメラが含まれる。撮影装置２００は、例えば１５ｆｐｓ～３０ｆｐｓのフレームレートの動画の入力画像２５０として撮影領域を撮影し得る。入力画像２５０は動画と静止画とを含む。入力画像２５０は、例えば、白黒画像で、１２８ピクセル×１２８ピクセルの画像であり得る。入力画像２５０は、カラー画像であってもよい。撮影装置２００は、入力画像２５０を値域決定装置１００等へ送信する。 The photographing device 200 is arranged, for example, on the ceiling or the upper part of the wall of the living room of the subject 500, or is arranged on the ceiling or the upper part of the wall of a store or the like. The photographing device 200 photographs a predetermined photographing range from a position where the target person 500 can be overlooked, and outputs an input image 250 (image data). The input image 250 includes an image including the subject 500. The photographing apparatus 200 includes a wide-angle camera. The photographing device 200 may photograph a photographing area as an input image 250 of a moving image having a frame rate of, for example, 15 fps to 30 fps. The input image 250 includes a moving image and a still image. The input image 250 is, for example, a black-and-white image and may be an image of 128 pixels × 128 pixels. The input image 250 may be a color image. The photographing device 200 transmits the input image 250 to the range determining device 100 and the like.
　通信ネットワーク３００には、イーサネット（登録商標）などの有線通信規格によるネットワークインターフェースを使用し得る。通信ネットワーク３００には、Ｂｌｕｅｔｏｏｔｈ（登録商標）、ＩＥＥＥ８０２．１１などの無線通信規格によるネットワークインターフェースを使用してもよい。 For the communication network 300, a network interface based on a wired communication standard such as Ethernet (registered trademark) can be used. For the communication network 300, a network interface based on a wireless communication standard such as Bluetooth (registered trademark) or 802.11 may be used.
　図２は、値域決定装置１００のハードウェア構成を示すブロック図である。値域決定装置１００は、制御部１１０、記憶部１２０、表示部１３０、入力部１４０、および通信部１５０を備える。これらの構成要素は、バス１６０を介して相互に接続される。値域決定装置１００は、コンピューターにより構成され得る。 FIG. 2 is a block diagram showing a hardware configuration of the range determination device 100. The range determination device 100 includes a control unit 110, a storage unit 120, a display unit 130, an input unit 140, and a communication unit 150. These components are connected to each other via the bus 160. The range determination device 100 may be configured by a computer.
　制御部１１０は、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）により構成され、プログラムに従って、値域決定装置１００の各部の制御および演算処理を行う。制御部１１０の作用の詳細については後述する。 The control unit 110 is configured by a CPU (Central Processing Unit), and controls and performs arithmetic processing of each unit of the range determination device 100 according to a program. The details of the operation of the control unit 110 will be described later.
　記憶部１２０は、ＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）、ＲＯＭ（Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ）、およびＳＳＤ（Ｓｏｌｉｄ　Ｓｔａｔｅ　Ｄｒｉｖｅ）により構成され得る。ＲＡＭは、制御部１１０の作業領域として一時的にプログラムやデータを記憶する。ＲＯＭは、あらかじめ各種プログラムや各種データを格納する。ＳＳＤは、オペレーションシステムを含む各種プログラムおよび各種データを格納する。 The storage unit 120 may be composed of a RAM (Random Access Memory), a ROM (Read Only Memory), and an SSD (Solid State Drive). The RAM temporarily stores programs and data as a work area of the control unit 110. The ROM stores various programs and various data in advance. The SSD stores various programs including an operating system and various data.
　表示部１３０は、例えば液晶ディスプレイであり、各種情報を表示する。 The display unit 130 is, for example, a liquid crystal display and displays various information.
　入力部１４０は、例えばタッチパネルや各種キーにより構成される。入力部１４０は、各種操作、入力のために使用される。 The input unit 140 is composed of, for example, a touch panel and various keys. The input unit 140 is used for various operations and inputs.
　通信部１５０は、外部機器と通信するためのインターフェースである。通信には、イーサネット（登録商標）、ＳＡＴＡ、ＰＣＩ　Ｅｘｐｒｅｓｓ、ＵＳＢ、ＩＥＥＥ１３９４などの規格によるネットワークインターフェースが用いられ得る。その他、通信には、Ｂｌｕｅｔｏｏｔｈ（登録商標）、ＩＥＥＥ８０２．１１、４Ｇなどの無線通信インターフェースが用いられ得る。 The communication unit 150 is an interface for communicating with an external device. For communication, a network interface according to standards such as Ethernet (registered trademark), SATA, PCI Express, USB, and IEEE 1394 can be used. In addition, a wireless communication interface such as Bluetooth (registered trademark), 802.11, or 4G may be used for communication.
　制御部１１０の作用の詳細について説明する。 The details of the operation of the control unit 110 will be described.
　図３は、値域決定装置１００の制御部１１０の機能の例を示すブロック図である。制御部１１０は、取得部１１１、推定部１１２、値域仮設定部１１３、ロス算出部１１４、および値域決定部１１５として機能する。推定部１１２は、ニューラルネットワークである、ディープラーニングモデルを用いて構成され得る。推定部１１２、値域仮設定部１１３、ロス算出部１１４、および値域決定部１１５は、演算部を構成する。 FIG. 3 is a block diagram showing an example of the function of the control unit 110 of the range determination device 100. The control unit 110 functions as an acquisition unit 111, an estimation unit 112, a range temporary setting unit 113, a loss calculation unit 114, and a range determination unit 115. The estimation unit 112 can be configured by using a deep learning model, which is a neural network. The estimation unit 112, the range temporary setting unit 113, the loss calculation unit 114, and the range determination unit 115 constitute a calculation unit.
　取得部１１１は、入力画像２５０と、当該入力画像２５０に対応する、関節点２５１の正解データを取得する。 The acquisition unit 111 acquires the input image 250 and the correct answer data of the joint point 251 corresponding to the input image 250.
　図４は、入力画像２５０を示す説明図である。図５は、関節点２５１を示す説明図である。 FIG. 4 is an explanatory diagram showing the input image 250. FIG. 5 is an explanatory diagram showing a joint point 251.
　図４の入力画像２５０の例においては、入力画像２５０に対象者５００の画像が含まれている。 In the example of the input image 250 of FIG. 4, the input image 250 includes an image of the target person 500.
　図５の関節点２５１の例においては、１４個の関節点が示されている。関節点２５１には、例えば、右足首、右膝、右腰、左腰、左膝、左足首、右手首、右肘、右肩、左手首、左肘、左肩、首、および頭部の各関節点２５１が含まれる。 In the example of the joint point 251 in FIG. 5, 14 joint points are shown. Joint points 251 include, for example, right ankle, right knee, right waist, left waist, left knee, left ankle, right wrist, right elbow, right shoulder, left wrist, left elbow, left shoulder, neck, and head. Includes joint point 251.
　推定部１１２は、値域仮設定部１１３により仮設定された値域で、３２ビットの浮動小数点で表現されたモデルパラメーターを、８ビットの固定小数点で表現可能な値にすることで量子化する。後述するように、値域仮設定部１１３は異なる値域を順次仮設定する。推定部１１２は、値域仮設定部１１３により順次仮設定された各値域で、モデルパラメーターを順次量子化する。なお、量子化される前の、３２ビットの浮動小数点で表現されたモデルパラメーターは、比較的高機能なコンピューターにより、大量の教師データ（画像と当該画像に対応した正解データとの組合せ）を用いて、予め学習されることで生成されたモデルパラメーターであり得る。 The estimation unit 112 quantizes the model parameter expressed by a 32-bit floating point number in the range temporarily set by the range temporary setting unit 113 by making it a value that can be expressed by an 8-bit fixed point number. As will be described later, the range temporary setting unit 113 sequentially temporarily sets different range. The estimation unit 112 sequentially quantizes the model parameters in each range temporarily set by the range temporary setting unit 113. The model parameters expressed in 32-bit floating point before being quantized use a large amount of teacher data (combination of an image and correct answer data corresponding to the image) by a relatively high-performance computer. It can be a model parameter generated by learning in advance.
　推定部１１２は、値域仮設定部１１３により、ディープラーニングモデルの層ごとに仮設定された値域で、当該層ごとにモデルパラメーターを量子化し得る。推定部１１２は、値域仮設定部１１３により、ディープラーニングモデルのモデルパラメーターごとに仮設定された値域で、当該モデルパラメーターごとにモデルパラメーターを量子化してもよい。以下、説明を簡単にするために、推定部１１２は、ディープラーニングモデルの層ごとに仮設定された値域で、当該層ごとにモデルパラメーターを量子化するものとして説明する。 The estimation unit 112 can quantize the model parameters for each layer in the range temporarily set for each layer of the deep learning model by the range temporary setting unit 113. The estimation unit 112 may quantize the model parameters for each model parameter in the range temporarily set for each model parameter of the deep learning model by the range temporary setting unit 113. Hereinafter, for the sake of simplicity, the estimation unit 112 will be described as quantizing the model parameters for each layer in the range temporarily set for each layer of the deep learning model.
　推定部１１２は、値域仮設定部１１３により仮設定された値域でモデルパラメーターをディープラーニングモデルの層ごとに量子化するとともに、入力画像２５０に基づいてディープラーニングモデルにより関節点２５１を推定する。 The estimation unit 112 quantizes the model parameters for each layer of the deep learning model in the range temporarily set by the range temporary setting unit 113, and estimates the joint point 251 by the deep learning model based on the input image 250.
　値域仮設定部１１３は、モデルパラメーターの量子化における値域を、順次変えて仮設定する。 The range temporary setting unit 113 temporarily changes and temporarily sets the range in the quantization of the model parameter.
　図６は、順次仮設定される、モデルパラメーターの量子化における値域を説明するための説明図である。図６の各グラフの横軸はモデルパラメーターの値を示しており、縦軸は頻度を示している。縦軸の頻度は、ディープラーニングモデルの学習時にモデルパラメーターがとった値の頻度である。 FIG. 6 is an explanatory diagram for explaining the range in the quantization of the model parameters, which are temporarily set sequentially. The horizontal axis of each graph in FIG. 6 shows the values of the model parameters, and the vertical axis shows the frequency. The frequency on the vertical axis is the frequency of the values taken by the model parameters when learning the deep learning model.
　図６において、破線で示す２ｎ、２ｎ－１、２ｎ－２は値域（より詳細には値域の上限）を表す。例えば、８ビットの固定小数点の表現では指数ｎは８であり、値域２ｎは２５６（１０進数）＝１１１１１１１１（２進数）である。値域２ｎでは、例えば、－１２８～１２７（１０進数）の値を、１（１０進数）＝１（２進数）刻みで表現できる。値域２ｎ－１では、例えば、－６４～６３．５（１０進数）の値を、０．５（１０進数）＝０．１（２進数）刻みで表現できる。従って、値域を大きくするとモデルパラメーターがとり得る値の幅を大きくできるが、量子化誤差が大きくなる。逆に、値域を小さくするとモデルパラメーターがとり得る値の幅が小さくなるが、量子化誤差が小さくなる。 In FIG. 6, 2 n , 2 n-1 , and 2 n-2 shown by broken lines represent a range (more specifically, the upper limit of the range). For example, in the 8-bit fixed-point representation, the exponent n is 8, and the range 2 n is 256 (decimal) = 111111111 (binary). In the range 2 n , for example, a value of −128 to 127 (decimal number) can be expressed in increments of 1 (decimal number) = 1 (binary number). In the range 2 n-1 , for example, a value of −64 to 63.5 (decimal number) can be expressed in increments of 0.5 (decimal number) = 0.1 (binary number). Therefore, if the range is increased, the range of values that the model parameters can take can be increased, but the quantization error becomes large. Conversely, if the range is made smaller, the range of values that the model parameters can take becomes smaller, but the quantization error becomes smaller.
　図６の例に示すように、値域仮設定部１１３は、モデルパラメーターの量子化における値域を、２ｎ、２ｎ－１、２ｎ－２の順で順次仮設定し得る。 As shown in the example of FIG. 6, the range temporary setting unit 113 can temporarily set the range in the quantization of the model parameter in the order of 2 n , 2 n-1 , and 2 n-2 .
　ロス算出部１１４は、推定部１１２による、入力画像２５０に基づく関節点２５１の推定結果の、正解データに対するロスを算出する。ロスは、例えば、すべての関節点２５１について算出した、関節点２５１と正解データとの差分（距離）の２乗平均とし得る。ロス算出部１１４は、ディープラーニングモデルの層ごとに、仮設定された値域におけるロスを算出する。 The loss calculation unit 114 calculates the loss of the estimation result of the joint point 251 based on the input image 250 by the estimation unit 112 with respect to the correct answer data. The loss can be, for example, the squared average of the difference (distance) between the joint point 251 and the correct answer data calculated for all the joint points 251. The loss calculation unit 114 calculates the loss in the temporarily set range for each layer of the deep learning model.
　値域決定部１１５は、ディープラーニングモデルの層ごとに、ロス算出部１１４により算出されたロスが最小になるときの値域を決定し、記憶部１２０に値域のパラメーターとして記憶させる。 The range determination unit 115 determines the range when the loss calculated by the loss calculation unit 114 is minimized for each layer of the deep learning model, and stores it in the storage unit 120 as a parameter of the range.
　図７は、ディープラーニングモデルの層ごとに、モデルパラメーターの量子化における値域が決定される推移を示す説明図である。 FIG. 7 is an explanatory diagram showing the transition in which the range in the quantization of the model parameters is determined for each layer of the deep learning model.
　図７の左側の図は、ディープラーニングモデルの最上層であるＬａｙｅｒ１について量子化され、値域が決定された状態が示されている。右側の図は、ディープラーニングモデルの最上層であるＬａｙｅｒ１について量子化され、値域が決定された後、最上層に対し１層下層であるＬａｙｅｒ２について量子化され、値域が決定された状態が示されている。このように、ディープラーニングモデルは、最上層から最下層に向かって、１層づつ、量子化され、値域が決定され得る。この際、下層の量子化は、上層の量子化が実行された状態で実行される。 The figure on the left side of FIG. 7 shows a state in which the range is determined by quantizing the layer 1 which is the uppermost layer of the deep learning model. The figure on the right shows the state in which the range is determined by quantizing the uppermost layer of the deep learning model, Layer1, and then quantizing the uppermost layer of Layer2, which is one layer below. ing. In this way, the deep learning model can be quantized layer by layer from the top layer to the bottom layer, and the range can be determined. At this time, the quantization of the lower layer is executed in the state where the quantization of the upper layer is executed.
　ディープラーニングモデルは、所定の対象層についてのみ、量子化され、値域が決定されてもよい。所定の対象層は複数の対象層であってもよい。また、入力部１４０からユーザーにより入力されること等により設定された対象層についてのみ、量子化され、値域が決定されてもよい。 The deep learning model may be quantized and the range may be determined only for a predetermined target layer. The predetermined target layer may be a plurality of target layers. Further, the range may be determined by quantizing only the target layer set by inputting from the input unit 140 by the user or the like.
　ディープラーニングモデルは、所定の対象層から最下層に向かって、１層づつ、量子化され、値域が決定されてもよい。この場合、所定の対象層は、ディープラーニングモデルの最上層であってもよく、最上層以外の層であってもよい。すなわち、所定の対象層の初期値は最上層であってもよく、最上層以外の層であってもよい。 The deep learning model may be quantized one layer at a time from a predetermined target layer toward the lowest layer, and the range may be determined. In this case, the predetermined target layer may be the uppermost layer of the deep learning model, or may be a layer other than the uppermost layer. That is, the initial value of the predetermined target layer may be the uppermost layer or may be a layer other than the uppermost layer.
　値域決定部１１５は、モデルパラメーターであるｂｉａｓの値域を、ｂｉａｓの層と同一層のｗｅｉｇｈｔの値域と、当該ｂｉａｓの層の１層上層のｏｕｔｐｕｔの値域とに基づいて算出（決定）し得る。 The range determination unit 115 can calculate (determine) the range of bias, which is a model parameter, based on the range of weight of the same layer as the layer of bias and the range of output of one layer above the layer of bias.
　ディープラーニングモデルが、畳み込み層の下層がｒｅｌｕ層である場合、値域決定部１１５は、畳み込み層の値域をｒｅｌｕ層の値域と同一の値域に決定し得る。 In the deep learning model, when the lower layer of the convolution layer is the relu layer, the range determination unit 115 can determine the range of the convolution layer to be the same as the range of the relu layer.
　図８は、値域決定装置１００の制御部１１０の機能の他の例を示すブロック図である。制御部１１０は、取得部１１１、推定部１１２、値域仮設定部１１３、ロス算出部１１４、値域決定部１１５、および学習用ロス算出部１１６として機能し得る。 FIG. 8 is a block diagram showing another example of the function of the control unit 110 of the range determination device 100. The control unit 110 can function as an acquisition unit 111, an estimation unit 112, a range temporary setting unit 113, a loss calculation unit 114, a range determination unit 115, and a learning loss calculation unit 116.
　取得部１１１は、入力画像２５０と、当該入力画像２５０に対応する、関節点２５１の正解データを取得する。取得部１１１は、さらに、学習用入力画像と、当該学習用入力画像に対応する、関節点２５１の学習用正解データを取得する。なお、ディープラーニングモデルの過学習を防止するため、学習用入力画像および学習用正解データは、それぞれ入力画像２５０および正解データとは異なるものを用いることが好ましい。学習用入力画像および学習用正解データは、それぞれ入力画像２５０および正解データと同じものを用いてもよい。学習用入力画像と学習用正解データの組合せは、学習用教師データを構成する。 The acquisition unit 111 acquires the input image 250 and the correct answer data of the joint point 251 corresponding to the input image 250. The acquisition unit 111 further acquires a learning input image and learning correct answer data of the joint point 251 corresponding to the learning input image. In order to prevent over-learning of the deep learning model, it is preferable to use different learning input images and learning correct answer data from the input image 250 and correct answer data, respectively. As the input image for learning and the correct answer data for learning, the same input image 250 and the correct answer data may be used, respectively. The combination of the input image for learning and the correct answer data for learning constitutes the teacher data for learning.
　推定部１１２は、値域仮設定部１１３により仮設定された値域で、３２ビットの浮動小数点で表現されたモデルパラメーターを、８ビットの固定小数点で表現可能な値にすることで量子化する。推定部１１２は、値域仮設定部１１３によりディープラーニングモデルの層ごとに仮設定された各値域で、モデルパラメーターを順次量子化して、ディープラーニングモデルを、学習用教師データを用いて、以下のように学習する。 The estimation unit 112 quantizes the model parameter expressed by a 32-bit floating point number in the range temporarily set by the range temporary setting unit 113 by making it a value that can be expressed by an 8-bit fixed point number. The estimation unit 112 sequentially quantizes the model parameters in each range temporarily set for each layer of the deep learning model by the range temporary setting unit 113, and uses the learning teacher data to obtain the deep learning model as follows. To learn.
　学習用ロス算出部１１６は、推定部１１２による、学習用入力画像に基づく関節点２５１の推定結果の、学習用正解データに対する学習用ロスを算出する。学習用ロスは、関節点２５１と学習用正解データとの差分（距離）であり、関節点２５１ごとに算出される。学習用ロス算出部１１６は、ディープラーニングモデルの層ごとに、仮設定された値域における学習用ロスを算出する。 The learning loss calculation unit 116 calculates the learning loss for the learning correct answer data of the estimation result of the joint point 251 based on the learning input image by the estimation unit 112. The learning loss is the difference (distance) between the joint point 251 and the correct answer data for learning, and is calculated for each joint point 251. The learning loss calculation unit 116 calculates the learning loss in the temporarily set range for each layer of the deep learning model.
　推定部１１２は、学習用ロス算出部１１６により関節点２５１ごとに算出された学習用ロスがそれぞれ０になるように、バックプロパゲーションによりディープラーニングモデルを学習する。 The estimation unit 112 learns the deep learning model by backpropagation so that the learning loss calculated for each joint point 251 by the learning loss calculation unit 116 becomes 0.
　推定部１１２は、値域仮設定部１１３により仮設定された値域でモデルパラメーターをディープラーニングモデルの層ごとに量子化して、当該ディープラーニングモデルを学習用教師データを用いて学習する（すなわち、当該ディープラーニングモデルのすべての層を学習する）とともに、学習後のディープラーニングモデルにより、入力画像２５０に基づいて関節点２５１を推定する。これにより、ディープラーニングモデルの層ごとの量子化の際に、層ごとにモデルパラメーターがファインチューニングされる。なお、推定部１１２は、値域仮設定部１１３により仮設定された値域でモデルパラメーターをディープラーニングモデルの層ごとに量子化して、量子化した層ごとに学習（モデルパラメーターの更新）をしてもよい。すなわち、所定の層ごとに量子化および学習をしてもよい。この場合、所定の層以外の層は、モデルパラメーターの更新をしないようにして学習する。 The estimation unit 112 quantizes the model parameters for each layer of the deep learning model in the value range temporarily set by the value range temporary setting unit 113, and learns the deep learning model using the learning teacher data (that is, the deep). Along with learning all layers of the learning model), the deep learning model after learning estimates the joint points 251 based on the input image 250. As a result, the model parameters are fine-tuned for each layer during the layer-by-layer quantization of the deep learning model. Even if the estimation unit 112 quantizes the model parameters for each layer of the deep learning model in the range temporarily set by the range temporary setting unit 113 and learns for each quantized layer (updates of the model parameters). good. That is, quantization and learning may be performed for each predetermined layer. In this case, the layers other than the predetermined layer are learned without updating the model parameters.
　値域仮設定部１１３は、モデルパラメーターの量子化における値域を、順次変えて仮設定する。 The range temporary setting unit 113 temporarily changes and temporarily sets the range in the quantization of the model parameter.
　ロス算出部１１４は、推定部１１２による、入力画像２５０に基づく関節点２５１の推定結果の、正解データに対するロスを算出する。ロスは、例えば、すべての関節点２５１について算出した、関節点２５１と正解データとの差分（距離）の２乗平均とし得る。ロス算出部１１４は、ディープラーニングモデルの層ごとに、仮設定された値域におけるロスを算出する。 The loss calculation unit 114 calculates the loss of the estimation result of the joint point 251 based on the input image 250 by the estimation unit 112 with respect to the correct answer data. The loss can be, for example, the squared average of the difference (distance) between the joint point 251 and the correct answer data calculated for all the joint points 251. The loss calculation unit 114 calculates the loss in the temporarily set range for each layer of the deep learning model.
　値域決定部１１５は、ディープラーニングモデルの層ごとに、ロス算出部１１４により算出されたロスが最小になるときの値域を決定する。 The range determination unit 115 determines the range when the loss calculated by the loss calculation unit 114 is minimized for each layer of the deep learning model.
　ディープラーニングモデルは、所定の対象層についてのみ、量子化および学習され、値域が決定されてもよい。 The deep learning model may be quantized and trained to determine the range only for a predetermined target layer.
　ディープラーニングモデルは、所定の対象層から最下層に向かって、１層づつ、量子化および学習され、値域が決定されてもよい。この場合、所定の対象層は、ディープラーニングモデルの最上層であってもよく、最上層以外の層であってもよい。すなわち、所定の対象層の初期値は最上層であってもよく、最上層以外の層であってもよい。 The deep learning model may be quantized and trained layer by layer from a predetermined target layer toward the bottom layer to determine a range. In this case, the predetermined target layer may be the uppermost layer of the deep learning model, or may be a layer other than the uppermost layer. That is, the initial value of the predetermined target layer may be the uppermost layer or may be a layer other than the uppermost layer.
　ディープラーニングモデルの層ごとの学習は、次の順番で行い得る。ｂｉａｓのモデルパラメーターを固定して、ｗｅｉｇｈｔを学習する。その後、ｗｅｉｇｈｔを固定して、ｂｉａｓを学習する。 Learning for each layer of the deep learning model can be performed in the following order. The model parameters of bias are fixed and weight is learned. After that, the weight is fixed and the bias is learned.
　このようにモデルパラメーターの値域が決定されるとともに学習された推定部１１２により、入力画像２５０に基づいて、関節点２５１が高精度に検出される。この際、推定部１１２は、記憶部１２０に記憶された学習後のモデルパラメーターをディープラーニングモデルに反映して関節点２５１を推定する。また、推定部１１２は、記憶部１２０に記憶された、層ごとの値域のパラメーターを読み出して、当該値域のパラメーターに基づいて、層ごとにモデルパラメーターを量子化する。 The range of model parameters is determined in this way, and the learning unit 112 learns that the joint point 251 is detected with high accuracy based on the input image 250. At this time, the estimation unit 112 estimates the joint point 251 by reflecting the model parameters after learning stored in the storage unit 120 in the deep learning model. Further, the estimation unit 112 reads out the parameters of the range for each layer stored in the storage unit 120, and quantizes the model parameters for each layer based on the parameters of the range.
　値域決定装置１００の動作について説明する。 The operation of the range determination device 100 will be described.
　図９は、値域決定装置１００の動作を示すフローチャートである。本フローチャートは、記憶部１２０に記憶されたプログラムに従い、制御部１１０により実行される。 FIG. 9 is a flowchart showing the operation of the range determination device 100. This flowchart is executed by the control unit 110 according to the program stored in the storage unit 120.
　制御部１１０は、入力画像２５０と、当該入力画像２５０に対応する、関節点２５１の正解データを取得する（Ｓ１０１）。制御部１１０は、入力画像２５０と、関節点２５１の正解データを、記憶部１２０から読み出すことにより取得し得る。制御部１１０は、入力画像２５０と、関節点２５１の正解データを、通信部１５０により受信することで取得してもよい。 The control unit 110 acquires the input image 250 and the correct answer data of the joint point 251 corresponding to the input image 250 (S101). The control unit 110 can acquire the input image 250 and the correct answer data of the joint point 251 by reading from the storage unit 120. The control unit 110 may acquire the input image 250 and the correct answer data of the joint point 251 by receiving the correct answer data by the communication unit 150.
　制御部１１０は、ディープラーニングモデルの対象層の量子化における値域を仮設定する（Ｓ１０２）。 The control unit 110 temporarily sets the range in the quantization of the target layer of the deep learning model (S102).
　制御部１１０は、仮設定された値域で量子化したモデルパラメーターで、学習用教師データを用いて対象層を学習する（Ｓ１０３）。 The control unit 110 learns the target layer using the teacher data for learning with the model parameters quantized in the temporarily set range (S103).
　制御部１１０は、学習後のディープラーニングモデルで、入力画像から関節点２５１を推定する（Ｓ１０４）。 The control unit 110 estimates the joint point 251 from the input image in the deep learning model after learning (S104).
　制御部１１０は、推定された関節点２５１の、正解データに対するロスを算出する（Ｓ１０５）。 The control unit 110 calculates the loss of the estimated joint point 251 with respect to the correct answer data (S105).
　制御部１１０は、すべての値域についてロスを算出したかどうか判断する（Ｓ１０６）。制御部１１０は、すべての値域についてロスを算出したと判断した場合は（Ｓ１０６：ＹＥＳ）、ステップＳ１０７を実行する。制御部１１０は、すべての値域についてロスを算出していないと判断した場合は（Ｓ１０６：ＮＯ）、ステップＳ１０２を実行する。 The control unit 110 determines whether or not the loss has been calculated for all the ranges (S106). If the control unit 110 determines that the loss has been calculated for all the range (S106: YES), the control unit 110 executes step S107. If the control unit 110 determines that the loss has not been calculated for all the range (S106: NO), the control unit 110 executes step S102.
　制御部１１０は、ロスが最小となったときの仮設定された値域を、対象層の値域として決定する（Ｓ１０７）。 The control unit 110 determines the temporarily set range when the loss is minimized as the range of the target layer (S107).
　制御部１１０は、ディープラーニングモデルのすべての層について値域を決定したかどうか判断する（Ｓ１０８）。制御部１１０は、ディープラーニングモデルのすべての層について値域を決定したと判断した場合は（Ｓ１０８：ＹＥＳ）、処理を終了する。制御部１１０は、ディープラーニングモデルのすべての層について値域を決定していないと判断した場合は（Ｓ１０８：ＮＯ）、対象層を１層下層にして、ステップＳ１０２～ステップＳ１０７を実行する。 The control unit 110 determines whether or not the range has been determined for all layers of the deep learning model (S108). When the control unit 110 determines that the range has been determined for all the layers of the deep learning model (S108: YES), the control unit 110 ends the process. When the control unit 110 determines that the range has not been determined for all the layers of the deep learning model (S108: NO), the target layer is set to one layer lower layer, and steps S102 to S107 are executed.
　本実施形態は、以下の効果を奏する。 This embodiment has the following effects.
　ディープラーニングモデルのパラメーターの量子化における値域を順次変えて、パラメーターを量子化するとともに、入力データに基づくディープラーニングモデルによる推定結果の正解データに対するロスを算出して、ロスが最小になるときの値域を決定する。これにより、パラメーターの量子化によって、高精度かつ軽量な量子化ディープラーニングモデルを得ることができる。 The range in the quantization of the parameters of the deep learning model is sequentially changed to quantize the parameters, and the loss for the correct answer data of the estimation result by the deep learning model based on the input data is calculated, and the range when the loss is minimized. To determine. As a result, a highly accurate and lightweight quantized deep learning model can be obtained by quantizing the parameters.
　さらに、モデルパラメーターの量子化における値域を順次変えて、パラメーターを量子化して、ディープラーニングモデルを学習用教師データを用いて学習するとともに、学習後のディープラーニングモデルによる入力データに基づく推定結果の正解データに対するロスを算出して、ロスが最小になるときの値域を決定する。これにより、量子化ディープラーニングモデルによる推定精度をより向上できる。 Furthermore, the value range in the quantization of the model parameters is sequentially changed, the parameters are quantized, the deep learning model is learned using the training teacher data, and the correct answer of the estimation result based on the input data by the deep learning model after learning. Calculate the loss for the data to determine the value range when the loss is minimal. As a result, the estimation accuracy by the quantized deep learning model can be further improved.
　さらに、所定の対象層について、ディープラーニングモデルのパラメーターの量子化における値域を順次変えて、パラメーターを量子化して、対象層または指定した層を学習するとともに、入力データに基づくディープラーニングモデルによる推定結果の正解データに対するロスを算出して、ロスが最小になるときの値域を決定する。これにより、より簡単かつ柔軟に、高精度かつ軽量な量子化ディープラーニングモデルを得ることができる。 Furthermore, for a predetermined target layer, the range in the quantization of the parameters of the deep learning model is sequentially changed, the parameters are quantized, the target layer or the specified layer is learned, and the estimation result by the deep learning model based on the input data is obtained. Calculate the loss for the correct answer data of, and determine the range when the loss is minimized. As a result, it is possible to obtain a highly accurate and lightweight quantized deep learning model more easily and flexibly.
　さらに、所定の対象層から最下層に向かって１層ごとに、ディープラーニングモデルのパラメーターの量子化における値域を順次変えて、パラメーターを量子化するとともに、入力データに基づくディープラーニングモデルによる推定結果の正解データに対するロスを算出して、ロスが最小になるときの値域を決定する。これにより、ディープラーニングモデル下層にて上層の量子化誤差が吸収されるため、より効果的に、高精度かつ軽量な量子化ディープラーニングモデルを得ることができる。 Furthermore, the range in the quantization of the parameters of the deep learning model is sequentially changed for each layer from the predetermined target layer toward the bottom layer to quantize the parameters, and the estimation result by the deep learning model based on the input data is obtained. Calculate the loss for the correct data and determine the range when the loss is minimized. As a result, the quantization error of the upper layer is absorbed in the lower layer of the deep learning model, so that a highly accurate and lightweight quantization deep learning model can be obtained more effectively.
　さらに、対象層の初期値を最上層とする。これにより、さらに効果的に、高精度かつ軽量な量子化ディープラーニングモデルを得ることができる。 Furthermore, the initial value of the target layer is set to the top layer. This makes it possible to obtain a highly accurate and lightweight quantized deep learning model more effectively.
　さらに、対象層の初期値を最上層以外の層とする。これにより、効果的かつ柔軟に、高精度かつ軽量な量子化ディープラーニングモデルを得ることができる。 Furthermore, the initial value of the target layer is set to a layer other than the top layer. This makes it possible to obtain a highly accurate and lightweight quantized deep learning model effectively and flexibly.
　さらに、ディープラーニングモデルの層ごとの学習を、次の順番で行う。ｂｉａｓのモデルパラメーターを固定して、ｗｅｉｇｈｔを学習する。その後、ｗｅｉｇｈｔを固定してｂｉａｓを学習する。これにより、量子化ディープラーニングモデルによる推定精度をさらに向上できる。 Furthermore, learning for each layer of the deep learning model is performed in the following order. The model parameters of bias are fixed and weight is learned. After that, the weight is fixed and the bias is learned. As a result, the estimation accuracy by the quantized deep learning model can be further improved.
　さらに、ｂｉａｓの値域を、当該ｂｉａｓの層と同じ層のｗｅｉｇｈｔの値域と、ｂｉａｓの層の１層上層のｏｕｔｐｕｔの値域とに基づいて算出する。これにより、既存のハードウェア構成を変えることなく、より簡単に、高精度かつ軽量な量子化ディープラーニングモデルを得ることができる。 Further, the range of bias is calculated based on the range of weight of the same layer as the layer of bias and the range of output of one layer above the layer of bias. This makes it possible to obtain a highly accurate and lightweight quantized deep learning model more easily without changing the existing hardware configuration.
　さらに、畳み込み層の下層がｒｅｌｕ層であるディープラーニングモデルの場合、畳み込み層の値域を当該ｒｅｌｕ層の値域と同一にする。これにより、特定の場合に、より簡単に、高精度かつ軽量な量子化ディープラーニングモデルを得ることができる。 Furthermore, in the case of a deep learning model in which the lower layer of the convolution layer is the relu layer, the range of the convolution layer is made the same as the range of the relu layer. This makes it possible to obtain a highly accurate and lightweight quantized deep learning model more easily in a specific case.
　以上に説明した、ディープラーニングモデルの値域決定プログラム、値域決定システム１０、および値域決定方法の構成は、上述の実施形態の特徴を説明するにあたって主要構成を説明したのであって、上述の構成に限られず、特許請求の範囲内において、種々改変することができる。また、一般的な位置推定システムおよび値域決定装置が備える構成を排除するものではない。 The configurations of the range determination program, the range determination system 10, and the range determination method of the deep learning model described above have been described as the main configurations in explaining the features of the above-described embodiment, and are limited to the above configurations. However, various modifications can be made within the scope of the claims. Further, it does not exclude the configurations provided in the general position estimation system and the range determination device.
　例えば、値域決定装置１００が有する機能を、コンピューターを内蔵する撮影装置２００が備えるようにしてもよい。 For example, the imaging device 200 having a built-in computer may have the function of the range determination device 100.
　また、値域決定装置１００、および撮影装置２００は、それぞれ複数の装置により構成されてもよく、いずれか複数の装置が単一の装置として構成されてもよい。 Further, the range determination device 100 and the photographing device 200 may each be configured by a plurality of devices, or any plurality of the devices may be configured as a single device.
　また、上述したフローチャートは、一部のステップを省略してもよく、他のステップが追加されてもよい。また各ステップの一部は同時に実行されてもよく、一つのステップが複数のステップに分割されて実行されてもよい。 Further, in the above-mentioned flowchart, some steps may be omitted or other steps may be added. Further, a part of each step may be executed at the same time, or one step may be divided into a plurality of steps and executed.
　また、実施形態においては、３２ビットの浮動小数点で表現されたモデルパラメーターを、８ビットの固定小数点に変換することを量子化の例として説明したが、量子化には、３２ビットの浮動小数点で表現されたモデルパラメーターを、３２ビット未満の浮動小数点に変換すること等も含まれる。 Further, in the embodiment, conversion of a model parameter expressed by a 32-bit floating point number to an 8-bit fixed point number has been described as an example of quantization, but for quantization, a 32-bit floating point number is used. It also includes converting the represented model parameters to floating point numbers less than 32 bits.
　また、上述した値域決定システム１０、値域決定装置１００における各種処理を行う手段および方法は、専用のハードウェア回路、またはプログラムされたコンピューターのいずれによっても実現することが可能である。上記プログラムは、例えば、ＵＳＢメモリやＤＶＤ（Ｄｉｇｉｔａｌ　Ｖｅｒｓａｔｉｌｅ　Ｄｉｓｃ）－ＲＯＭ等のコンピューター読み取り可能な記録媒体によって提供されてもよいし、インターネット等のネットワークを介してオンラインで提供されてもよい。この場合、コンピューター読み取り可能な記録媒体に記録されたプログラムは、通常、ハードディスク等の記憶部に転送され記憶される。また、上記プログラムは、単独のアプリケーションソフトとして提供されてもよいし、一機能としてその撮影装置等の装置のソフトウエアに組み込まれてもよい。 Further, the means and methods for performing various processes in the range determination system 10 and the range determination device 100 described above can be realized by either a dedicated hardware circuit or a programmed computer. The program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versaille Disc) -ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk. Further, the above program may be provided as a single application software, or may be incorporated into the software of a device such as a photographing device as a function.
　本出願は、２０２０年８月６日に出願された日本特許出願（特願２０２０－１３３９７２号）に基づいており、その開示内容は、参照され、全体として、組み入れられている。 This application is based on a Japanese patent application (Japanese Patent Application No. 2020-133972) filed on August 6, 2020, and the disclosure contents are referenced and incorporated as a whole.
　　１０　　値域決定システム、
　　１００　　値域決定装置、
　　１１０　　制御部、
　　１１１　　取得部、
　　１１２　　推定部、
　　１１３　　値域仮設定部、
　　１１４　　ロス算出部、
　　１１５　　値域決定部、
　　１１６　　学習用ロス算出部、
　　１２０　　記憶部、
　　１３０　　通信部、
　　１４０　　入力部、
　　１５０　　通信部、
　　２００　　撮影装置、
　　２５０　　入力画像、
　　２５１　　関節点、
　　５００　　対象者。 10 range determination system,
100 range determination device,
110 Control unit,
111 acquisition part,
112 Estimator,
113 Range temporary setting section,
114 Loss calculation unit,
115 Range Determination,
116 Learning loss calculation unit,
120 storage,
130 Communication Department,
140 input section,
150 communication department,
200 shooting equipment,
250 input image,
251 Joint Point,
500 Target people.

Claims (11)

  1. 　ディープラーニングモデルのパラメーターの量子化における値域を決定するプログラムであって、
    　入力データと、前記入力データに対応する正解データとを取得する手順（ａ）と、
    　前記パラメーターの量子化における前記値域を順次変えて、前記パラメーターを量子化するとともに、前記入力データに基づく前記ディープラーニングモデルによる推定結果の前記正解データに対するロスを算出して、前記ロスが最小になるときの前記値域を決定する手順（ｂ）と、
    　を有する処理を、コンピューターに実行させるためのディープラーニングモデルの値域決定プログラム。     A program that determines the range in the quantization of parameters of a deep learning model.
    The procedure (a) for acquiring the input data and the correct answer data corresponding to the input data, and
    The range in the quantization of the parameter is sequentially changed to quantize the parameter, and the loss of the estimation result by the deep learning model based on the input data with respect to the correct answer data is calculated to minimize the loss. The procedure (b) for determining the above range at the time and
    A range determination program of a deep learning model for letting a computer execute a process having.
  2. 　前記手順（ｂ）は、前記パラメーターの量子化における前記値域を順次変えて、前記パラメーターを量子化して、前記ディープラーニングモデルを学習用教師データを用いて学習するとともに、学習後の前記ディープラーニングモデルによる前記入力データに基づく推定結果の前記正解データに対する前記ロスを算出して、前記ロスが最小になるときの前記値域を決定する、請求項１に記載のディープラーニングモデルの値域決定プログラム。 In the procedure (b), the value range in the quantization of the parameter is sequentially changed, the parameter is quantized, the deep learning model is learned using the learning teacher data, and the deep learning model after learning is used. The value range determination program of the deep learning model according to claim 1, wherein the loss for the correct answer data of the estimation result based on the input data is calculated, and the value range when the loss is minimized is determined.
  3. 　前記手順（ｂ）は、
    　前記ディープラーニングモデルの所定の対象層の前記パラメーターの量子化における前記値域を仮設定する手順（ｂ１）と、
    　前記対象層の前記パラメーターを、前記手順（ｂ１）において仮設定された前記値域で量子化して、前記学習用教師データを用いて、前記対象層または指定した層を学習する手順（ｂ２）と、
    　前記手順（ｂ２）において学習された前記ディープラーニングモデルによる前記入力データに基づく推定結果の前記正解データに対する前記ロスを算出する手順（ｂ３）と、
    　前記仮設定した前記値域を順次変えて、前記手順（ｂ２）および前記手順（ｂ３）を実行して、前記仮設定した前記値域の中で、前記手順（ｂ３）において算出された前記ロスが最小になるときの前記値域を決定する手順（ｂ４）と、
    　を有する、請求項２に記載のディープラーニングモデルの値域決定プログラム。     The procedure (b) is
    The procedure (b1) for provisionally setting the range in the quantization of the parameter of the predetermined target layer of the deep learning model, and
    The procedure (b2) of learning the target layer or the designated layer by quantizing the parameters of the target layer in the range temporarily set in the procedure (b1) and using the teacher data for learning.
    The procedure (b3) for calculating the loss of the estimation result based on the input data by the deep learning model learned in the procedure (b2) with respect to the correct answer data, and the procedure (b3).
    The temporarily set range is sequentially changed, the procedure (b2) and the procedure (b3) are executed, and the loss calculated in the procedure (b3) is the smallest in the temporarily set range. The procedure (b4) for determining the range when becomes
    2. The range determination program for the deep learning model according to claim 2.
  4. 　前記手順（ｂ）は、
    　前記対象層の１層下層を新たな対象層として、前記手順（ｂ１）～前記手順（ｂ４）を実行して、前記新たな対象層の前記パラメーターの値域を決定する手順（ｂ５）と、
    　前記手順（ｂ５）を、前記新たな対象層が最下層となるまで実行する手順（ｂ６）と、
    　をさらに有する、請求項３に記載のディープラーニングモデルの値域決定プログラム。     The procedure (b) is
    A procedure (b5) in which the procedure (b1) to the procedure (b4) are executed with one lower layer of the target layer as a new target layer to determine the range of the parameter of the new target layer.
    The procedure (b6) in which the procedure (b5) is executed until the new target layer becomes the lowest layer, and the procedure (b6).
    The range determination program of the deep learning model according to claim 3, further comprising.
  5. 　前記（ｂ１）における前記対象層の初期値は最上層である、請求項４に記載のディープラーニングモデルの値域決定プログラム。 The range determination program of the deep learning model according to claim 4, wherein the initial value of the target layer in the above (b1) is the uppermost layer.
  6. 　前記（ｂ１）における前記対象層の初期値は最上層以外の層である、請求項４に記載のディープラーニングモデルの値域決定プログラム。 The range determination program of the deep learning model according to claim 4, wherein the initial value of the target layer in the above (b1) is a layer other than the uppermost layer.
  7. 　前記ディープラーニングモデルの各層の前記パラメーターは、ｗｅｉｇｈｔ、およびｂｉａｓを含み、
    　前記手順（ｂ２）は、
    　前記ｂｉａｓを固定して、前記ｗｅｉｇｈｔを学習する手順（ｂ２１）と、
    　前記ｗｅｉｇｈｔを固定して、前記ｂｉａｓを学習する手順（ｂ２２）と、
    　を有する、請求項３～６のいずれか一項に記載のディープラーニングモデルの値域決定プログラム。     The parameters of each layer of the deep learning model include weight, and bias.
    The procedure (b2) is
    The procedure (b21) for fixing the bias and learning the weight, and
    The procedure (b22) for learning the bias by fixing the weight and
    The range determination program of the deep learning model according to any one of claims 3 to 6.
  8. 　前記ディープラーニングモデルの各層の前記パラメーターは、ｗｅｉｇｈｔ、ｂｉａｓ、およびｏｕｔｐｕｔを含み、
    　前記手順（ｂ４）は、前記ｂｉａｓの前記値域を、前記ｂｉａｓの層と同じ層の前記ｗｅｉｇｈｔの前記値域と、前記ｂｉａｓの層の１層上層の前記ｏｕｔｐｕｔの前記値域とに基づいて算出する、請求項３～７のいずれか一項に記載のディープラーニングモデルの値域決定プログラム。     The parameters of each layer of the deep learning model include weight, bias, and output.
    The procedure (b4) calculates the range of the bias based on the range of the weight of the same layer as the layer of the bias and the range of the output of the layer one layer above the layer of the bias. The range determination program for the deep learning model according to any one of claims 3 to 7.
  9. 　畳み込み層の下層がｒｅｌｕ層である前記ディープラーニングモデルの場合、前記手順（ｂ４）は、前記畳み込み層の前記値域を前記ｒｅｌｕ層の前記値域と同一にする、請求項３～８のいずれか一項に記載のディープラーニングモデルの値域決定プログラム。 In the case of the deep learning model in which the lower layer of the convolution layer is the relu layer, the procedure (b4) makes the range of the convolution layer the same as the range of the relu layer, any one of claims 3 to 8. The range determination program for the deep learning model described in Section.
  10. 　ディープラーニングモデルのパラメーターの量子化における値域を決定するシステムであって、
    　入力データと、前記入力データに対応する正解データとを取得する取得部と、
    　前記パラメーターの量子化における前記値域を順次変えて、前記パラメーターを量子化するとともに、前記入力データに基づく前記ディープラーニングモデルによる推定結果の前記正解データに対するロスを算出して、前記ロスが最小になるときの前記値域を決定する演算部と、
    　を有するディープラーニングモデルの値域決定システム。     It is a system that determines the range in the quantization of the parameters of the deep learning model.
    An acquisition unit that acquires input data and correct answer data corresponding to the input data,
    The range in the quantization of the parameter is sequentially changed to quantize the parameter, and the loss of the estimation result by the deep learning model based on the input data with respect to the correct answer data is calculated to minimize the loss. The arithmetic unit that determines the above range at the time,
    Range determination system for deep learning models with.
  11. 　ディープラーニングモデルのパラメーターの量子化における値域を、値域決定システムにより決定する方法であって、
    　入力データと、前記入力データに対応する正解データとを取得する段階（ａ）と、
    　前記パラメーターの量子化における前記値域を順次変えて、前記パラメーターを量子化するとともに、前記入力データに基づく前記ディープラーニングモデルによる推定結果の前記正解データに対するロスを算出して、前記ロスが最小になるときの前記値域を決定する段階（ｂ）と、
    　を有するディープラーニングモデルの値域決定方法。     It is a method to determine the range in the quantization of the parameters of the deep learning model by the range determination system.
    The step (a) of acquiring the input data and the correct answer data corresponding to the input data, and
    The range in the quantization of the parameter is sequentially changed to quantize the parameter, and the loss of the estimation result by the deep learning model based on the input data with respect to the correct answer data is calculated to minimize the loss. At the stage (b) of determining the above range,
    How to determine the range of a deep learning model with.
PCT/JP2021/016965 2020-08-06 2021-04-28 Value range determination program, value range determination system, and value range determination method for deep learning model WO2022030054A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2022541117A JPWO2022030054A1 (en) 2020-08-06 2021-04-28

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020133972 2020-08-06
JP2020-133972 2020-08-06

Publications (1)

Publication Number Publication Date
WO2022030054A1 true WO2022030054A1 (en) 2022-02-10

Family

ID=80117260

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/016965 WO2022030054A1 (en) 2020-08-06 2021-04-28 Value range determination program, value range determination system, and value range determination method for deep learning model

Country Status (2)

Country Link
JP (1) JPWO2022030054A1 (en)
WO (1) WO2022030054A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108734266A (en) * 2017-04-21 2018-11-02 展讯通信（上海）有限公司 Compression method and device, terminal, the storage medium of deep neural network model
JP2020113273A (en) * 2019-01-09 2020-07-27 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． Method and device for neural network quantization

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108734266A (en) * 2017-04-21 2018-11-02 展讯通信（上海）有限公司 Compression method and device, terminal, the storage medium of deep neural network model
JP2020113273A (en) * 2019-01-09 2020-07-27 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． Method and device for neural network quantization

Also Published As

Publication number Publication date
JPWO2022030054A1 (en) 2022-02-10

Similar Documents

Publication Publication Date Title
JP6504590B2 (en) System and computer implemented method for semantic segmentation of images and non-transitory computer readable medium
CN107392255B (en) Generation method and device of minority picture sample, computing equipment and storage medium
WO2021052159A1 (en) Adversarial transfer learning-based face beauty prediction method and device
CN111507343A (en) Training of semantic segmentation network and image processing method and device thereof
WO2019232960A1 (en) Automatic bone age prediction method and system, and computer device and storage medium
JP2020009444A (en) Method and apparatus of processing parameter in neural network
CN111126574A (en) Method and device for training machine learning model based on endoscopic image and storage medium
CN110503680A (en) It is a kind of based on non-supervisory convolutional neural networks monocular scene depth estimation method
CN109063584B (en) Facial feature point positioning method, device, equipment and medium based on cascade regression
US20200089958A1 (en) Image recognition method and apparatus, electronic device, and readable storage medium
CN110838122B (en) Point cloud segmentation method and device and computer storage medium
CN112861659B (en) Image model training method and device, electronic equipment and storage medium
CN111798439A (en) Medical image quality interpretation method and system for online and offline fusion and storage medium
CN113095129A (en) Attitude estimation model training method, attitude estimation device and electronic equipment
CN113129447A (en) Three-dimensional model generation method and device based on single hand-drawn sketch and electronic equipment
CN111814804B (en) Human body three-dimensional size information prediction method and device based on GA-BP-MC neural network
CN115239508A (en) Scene planning adjustment method, device, equipment and medium based on artificial intelligence
CN111382619B (en) Picture recommendation model generation method, picture recommendation method, device, equipment and medium
WO2019188573A1 (en) Computing device, computing method, and program
WO2022030054A1 (en) Value range determination program, value range determination system, and value range determination method for deep learning model
JP7403995B2 (en) Information processing device, control method and program
WO2020052170A1 (en) Target object identification method and device, and storage medium
WO2022127603A1 (en) Model processing method and related device
CN112561050A (en) Neural network model training method and device
CN112598728B (en) Projector attitude estimation, trapezoidal correction method and device, projector and medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21852756

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022541117

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21852756

Country of ref document: EP

Kind code of ref document: A1