JP6986503B2 - Electronic control device, neural network update system - Google Patents

Description

The present invention relates to an electronic control device and a neural network update system.

In recent years, the development of automatic driving systems has become active. In order to drive in a complicated driving environment in an automatic driving system, "recognition" that senses the environment around the vehicle based on information from various sensors such as cameras, laser radars, and millimeter wave radars, and sensors It is necessary to enhance functions such as "cognition" that estimates how the detected objects around the vehicle will behave in the future, and "judgment" that plans future behavior of the vehicle based on the results of recognition and recognition. become. Therefore, it is expected that these functions will be further enhanced by introducing AI (Artificial Intelligence) models such as neural networks and deep learning. When executing the arithmetic processing of the neural network in real time in the processing cycle that can be used in the vehicle running control in the electronic control unit (ECU) for the vehicle, not only the CPU but also the FPGA (Field-Programmable Gate Array) ), ASIC (Application Specific Integrated Circuit), GPU (Graphics Processing Unit) and other hardware devices are often additionally used, and increasing the power consumption of the ECU becomes an issue. Therefore, it is necessary to reduce the amount of calculation of the neural network to reduce the power consumption when executing the neural network. Patent Document 1 describes a process of holding the convolution amount calculated based on the neuron data and parameters of the input image in each first memory area, and the convolution amount held in each first memory area. The first recognition process in which the convolution amount after the thinning is performed in each of the second memory areas is held in the first plurality of layers, and the second recognition process is held in the second memory area. The second plurality of layers perform a process of accumulating the weights held in the third memory area for all the convolution amounts after the thinning and holding the output results in each of the fourth memory areas. For each layer included in the recognition control unit that controls the recognition process and the first plurality of layers and the second plurality of layers, the neuron data size which is the size of each neuron data and the parameter which is the size of each parameter. The memory amount calculation unit that calculates the size and the like, and the gradient of the error of the output result calculated based on the output result held in the fourth memory area are held in the fifth memory area, and the memory amount calculation is performed. Based on the magnitude relationship between the neuron data size and the parameter size of each layer included in the second plurality of layers calculated by the unit, the gradient of the error of the output result held in the fifth memory area or the second plurality of layers. After holding the gradient of the error to the next layer in the second plurality of layers calculated based on the gradient of the error held in the sixth memory area of the previous layer in each sixth memory area, the second The first learning process in the second plurality of layers, which holds the gradient of the parameter error to the next layer in the plurality of layers in each of the third memory areas, and the first learning process calculated by the memory amount calculation unit. Based on the magnitude relationship between the neuron data size and the parameter size of each layer included in the plurality of layers, the sixth memory area of the final layer of the second plurality of layers or the seventh memory of the previous layer in the first plurality of layers. After the gradient of the parameter error to the next layer calculated based on the gradient of the error held in the area is held in each seventh memory area, the gradient of the error to the next layer in the first plurality of layers is determined by each of the above. Disclosed is an image recognition device comprising a learning control unit that controls a second learning process in the first plurality of layers held in a second memory area, respectively.

Japanese Unexamined Patent Publication No. 2018-18350

In the invention described in Patent Document 1, the inference accuracy for data for which high-precision inference is required when the weighting coefficient is deleted may be lower than that before the weighting coefficient is deleted.

The electronic control device according to the first aspect of the present invention is an electronic control device that stores a neural network including a plurality of neurons and a connection information group including a plurality of connection information that links the neurons to each other. includes a first binding information group input values are the binding information group for determining an output value when given, impact for a given data given as the input value exceeds a predetermined value A neural including a third connection information group created by the thinning unit deleting at least one connection information from the first connection information group based on the second connection information group which is the plurality of connection information. It is a network.
The neural network update system according to the second aspect of the present invention includes a server and an electronic control device, and updates a neural network including a plurality of neurons and a coupling information group including a plurality of coupling information linking the neurons to each other. The server is a neural network update system, the server includes a storage unit and a calculation unit, and the storage unit is a combination information group for determining an output value when an input value is given. stores 1 binding information group, the arithmetic unit, when a predetermined data is given as the input value, a second influence on the predetermined data is a plurality of said binding information above a predetermined value At least one binding information is deleted from the first coupling information group based on the coefficient specifying unit for specifying the coupling information group, the first coupling information group, and the second coupling information group, and the third The electronic control device includes a thinning unit for creating the combined information group of the above, and the electronic control device stores a neural network including the third combined information group.

According to the present invention, it is possible to reduce the amount of calculation of the neural network by limiting the decrease in inference accuracy for data that requires high-precision inference.

Diagram showing an example of the structure of the AI model Configuration diagram of the neural network update system 2 in the first embodiment The figure which shows an example of the data set management table 30 Flow chart showing the contraction process of the neural network by the server 21 A flowchart showing the details of S402 in FIG. The figure which shows an example of the data set management table 31 in the modification 1. The figure which shows the detail of the step S402 shown in FIG. 3 in the modification 4 The figure which shows an example of the process of step S602 in the modification 4 The figure which shows the detail of the step S402 shown in FIG. 3 in the modification 5. The figure which shows an example of the process shown in FIG. The figure which shows the detail of the step S402 shown in FIG. 3 in the modification 6. The figure which shows an example of the process of step S902 System configuration diagram in variant 6 Configuration diagram of the neural network update system 2 in the second embodiment The figure which shows an example of the reduction rate correction table 204

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In each of the following embodiments, the same components, the same processing contents, and the like are described with the same numbers to simplify the description. In each embodiment, a neural network will be used as an example of the AI model in the description of the electronic control device, the server, and the entire system provided with the AI model using artificial intelligence processing. However, the AI model may be a model related to machine learning, deep learning, and reinforcement learning. That is, each embodiment described below can be applied to various AI models that can be optimized by approximation and thinning processing. In the embodiment described below, the optimization of the model will be described by taking as an example a weighting coefficient and a method of thinning out neurons, and the processing will be described as “neural network reduction”. Further, in the following, the "neural network" may be referred to as "NN".

-First embodiment-
Hereinafter, the first embodiment of the neural network update system will be described with reference to FIGS. 1 to 5. In the first embodiment, the connection information of the neural network deeply related to the data requiring high-precision inference (hereinafter, also referred to as "important data") is specified, and the connection information is thinned out using the information. conduct. As the thinning process, a method of deleting the join information itself or a method of changing the value to 0 can be considered.

FIG. 1 is a diagram showing an example of the structure of an AI model. As shown in FIG. 1, the neural network model 1 includes an input layer 10, an intermediate layer 11, and an output layer 12, and each layer has I, J, and K arithmetic units 100, respectively. As shown by reference numeral 101 and reference numeral 102 in FIG. 1, each arithmetic unit 100 is connected based on the connection information between the arithmetic units 100. When the information is input to the input layer 10, the coupling information is added and propagated in the intermediate layer 11, and finally the information corresponding to the prediction result is output from the output layer 12. The connection information regarding the connection between the arithmetic units 100 is referred to as "network structure information" in the present embodiment. The arithmetic unit 100 corresponds to the above-mentioned "neuron".

The coupling information is composed of a coupling coefficient and a bias, and the information is propagated from the input layer 10 to the output layer 12 while performing an operation using the coupling coefficient as a coefficient. The coupling coefficient used in the calculation of the AI model is referred to as a "weighting coefficient" in this embodiment. Further, in the present embodiment, for the sake of brevity, bias is excluded and the coupling information is described as being composed of only the weighting coefficients. That is, in the present embodiment, the coupling information and the weighting coefficient are the same. However, the present invention also includes a configuration in which the binding information includes a bias.

Further, in the present embodiment, in order to identify individual weighting factors, an ID is conveniently assigned to each weighting factor. Further, the value of the weighting coefficient is called a "weighting coefficient value" or a "weighting coefficient value". That is, in the following, it is expressed as "the weighting coefficient value of the weighting coefficient whose ID is 11 is xx". The content of the operation is specified by the type of layer included in the join information. The layer included in the binding information is, for example, a convolution layer, a batch normalization, an activation function, a pooling layer, a fully bound layer, an LSTM (Long Short Term Memory) layer, and the like.

Since the number of arithmetic units 100 and the number of layers constituting the intermediate layer 11 are irrelevant to the embodiment, they are set to arbitrary values. Further, the structure of the AI model is not particularly limited, and for example, the connection between the arithmetic units 100 may have recursiveness or bidirectionality. Further, the present embodiment can be applied to any AI model such as a machine learning model with or without a teacher, and a reinforcement learning model. This is because the AI model can be reduced by thinning out the weighting factors and arithmetic units of the AI model without affecting the inference accuracy for high-priority data that requires high-precision inference. However, changing the weighting factor value of the weighting factor to zero is also included in thinning out the weighting factor.

FIG. 2 is a block diagram of the neural network update system 2 according to the first embodiment. The neural network update system 2 includes a storage unit 20, a server 21, and an electronic control device 22. As shown in FIG. 2, the server 21 and the electronic control device 22 are connected via the network 23. Although the storage unit 20 is directly connected to the server 21 in FIG. 2, the storage unit 20 may be connected to the server 21 via the network 23.

In the present embodiment, the server 21 contracts the neural network, and the electronic control device 22 downloads the contracted neural network from the server and updates it. As a result, in an application such as automatic driving, arithmetic processing using a neural network is executed. However, the present embodiment is not limited to such a configuration. That is, as a configuration in which all or part of the above-mentioned functions are mounted on the electronic control device 22, such as when the architecture is such that the neural network is additionally learned and updated by being closed to the vehicle equipped with the electronic control device 22 without going through the server 21. May be good.

The storage unit 20 is a non-volatile storage area, for example, a flash memory. The storage unit 20 stores neural network information and data sets used by the server 21 in the learning, generation, and reduction processing of the neural network. Specifically, the storage unit 20 includes a pre-reduction neural network 200, a post-reduction neural network 201, important data 202, and normal data 203.

The pre-reduction neural network 200 refers to the information of the neural network before the reduction is applied, and is composed of the first connection information group 2000 and the network structure information 2001. The first coupling information group 2000 is a weighting coefficient value of the weighting coefficient of the neural network before the reduction is applied. The network structure information 2001 is the network structure information of the neural network before the reduction is applied. The pre-reduction NN200 is, for example, the entire neural network model 1 shown in FIG. The first coupling information group 2000 is, for example, a weighting coefficient value of all the weighting coefficients extracted from the NN model 1 shown in FIG. Further, the network structure information 2001 is, for example, information obtained by removing the weighting coefficient values of all the weighting coefficients from the NN model 1.

The post-reduction neural network 201 is information on the neural network after the reduction is applied, and is composed of a third coupling information group 2010 and network structure information 2011. The third coupling information group 2010 refers to the weighting coefficient value of the weighting coefficient of the neural network after the reduction is applied. Network structure information 2011 refers to the network structure information of the neural network before the reduction is applied. If there is no difference in the network structure information of the neural network before and after applying the reduction, the network structure information 2001 and the network structure information 2011 are the same.

The important data 202 and the normal data 203 are a group of data created in advance, and are used for constructing a neural network and testing inference. The data included in the important data 202 and the normal data 203 are, for example, the sensor output collected by the moving vehicle, the relative distance to the vehicle traveling ahead, and whether the vehicle traveling ahead is traveling in the same lane. It is a combination of information on whether or not.

The important data 202 is data that requires highly accurate inference, and is, for example, a data group that is directly related to vehicle running control and wants to reduce inference errors as much as possible. The important data 202 is, for example, data when the relative distance is short and the vehicle is traveling in the same lane. The normal data 203 is a group of data other than the important data 202 that is not directly related to the vehicle travel control. The normal data 203 is, for example, data in which the relative distance is very long or data in the case of traveling in different lanes. Whether a certain data is classified into important data 202 or normal data 203 may be determined by the server 21 by arithmetic processing with reference to the data set management table 30 described below, or by the operator of the server 21. It may be decided according to a predetermined rule.

FIG. 3 is a diagram showing an example of the data set management table 30. The data set management table 30 is composed of a plurality of records, and each record is composed of a data type name 300, a priority determination condition 301, a priority 302, and an operation coefficient 303. The data type name 300 is a name provided for convenience in order to identify the data. Therefore, the data type name 300 may be any character string as long as it can be identified. The priority determination condition 301 is a condition for determining the priority, in other words, a condition indicating the applicability to the record.

The priority 302 indicates that the higher the numerical value, the higher the priority, that is, more accurate inference is required. Each data may be classified into important data 202 or normal data 203 according to the value of this priority. The operation coefficient 303 is information used at the time of reduction, and is used, for example, to determine the ID of the weighting coefficient to be thinned out, that is, the weighting coefficient to be thinned out.

The priority determination condition 301 is composed of a relative distance 3010 and a preceding vehicle flag 3011. The relative distance 3010 is information representing the relative distance between the own vehicle and the object, and the smaller the value of the relative distance, the larger the value of the priority 302, that is, the higher the priority of this data tends to be. The preceding vehicle flag 3011 is flag information indicating whether or not the target object corresponds to the preceding vehicle on the same traveling lane when viewed from the own vehicle.

When the preceding vehicle flag 3011 is TRUE, the priority value tends to be large. In the example shown in the first line of FIG. 3, when the preceding vehicle flag 3011 is on and the relative distance 3010 is within 30 meters, it is determined that the data is for object detection 1, the priority 302 is set to 3, and the operation coefficient is set to 2. Will be done. Further, in the example shown in the last row of FIG. 3, when the preceding vehicle flag 3011 is FALSE, it is determined that all objects within 500 meters of the relative distance 3010 are object detection data 4, the priority 302 is set to 1, and the operation coefficient is set to 1. Will be done.

Returning to the description of FIG. The server 21 includes a calculation unit 210, a program storage unit 211, and a RAM 212. The arithmetic unit 210 includes at least a CPU (Central Processing Unit) 2100, which is a central arithmetic unit, as hardware, and the CPU 2100 performs learning, generation, and reduction processing of a neural network according to a program stored in the program storage unit 211. Run. However, the same function as the program stored in the program storage unit 211 may be realized as a hardware circuit. Further, when the functions of all the programs stored in the program storage unit 211 are realized as a hardware circuit, the server 21 does not have to include the program storage unit 211.

Accelerator 2101 may be included in the arithmetic unit 210 in order to speed up learning / generation / reduction of the neural network. The accelerator 2101 may be a hardware device such as FPGA, ASIC, and GPU.

The program storage unit 211 includes a non-volatile semiconductor memory such as a flash memory. In order to realize the calculation coefficient specifying unit 2110, the operation coefficient determining unit 2111, the coefficient group calculation unit 2112, the thinning unit 2113, the NN execution control unit 2114, and the NN learning control unit 2115 in the program storage unit 211. Program is stored. These programs are executed by the arithmetic unit 210. The operation of these programs will be described later with reference to FIGS. 4 to 5. The structure of the program is arbitrary, and the programs may be combined or separated in any way. For example, even if one program has all the functions corresponding to the calculation coefficient specifying unit 2110, the operation coefficient determining unit 2111, the coefficient group calculation unit 2112, the thinning unit 2113, the NN execution control unit 2114, and the NN learning control unit 2115. good.

The RAM 212 is a volatile semiconductor memory. The RAM 212 stores data for the arithmetic unit 210 to execute the program, for example, a second connection information group 2120. The second coupling information group 2120 is used as a determination condition when the neural network is contracted. The second binding information group 2120 is also referred to as a "significantly related WHI". Details will be described later.

The network 23 represents a communication network for a vehicle equipped with the electronic control device 22 to access the server 21. As the access means, for example, a method such as cellular or wireless LAN can be considered. In addition to these wireless means, a wired means may be connected via OBD (On-Board Diagnostics) or OBD2 (On-Board Diagnostics 2).

The electronic control device 22 includes a storage unit 220, a calculation unit 221, a program storage unit 222, and a RAM 223. The storage unit 220 is a non-volatile storage device such as a flash memory. The post-reduction neural network 2200 is stored in the storage unit 220. After reduction, the NN2200 is composed of a third coupling information group 22000 and network structure information 22001. The contracted NN2200 stored in the electronic control device 22 has the same name as the contracted NN201 stored in the storage unit 20, but the data itself is not necessarily the same. However, in the initial state immediately after shipment, the contracted NN2200 and the contracted NN201 are the same.

The arithmetic unit 221 includes a CPU 2210 which is a central arithmetic processing unit. The CPU 2210 executes inference processing using a neural network and learning processing of the neural network according to the program stored in the program storage unit 222. The arithmetic unit 221 may include an accelerator 2101 in order to speed up these inference and learning processes. The accelerator 2101 may be a hardware device such as FPGA, ASIC, or GPU. The program storage unit 222 includes a non-volatile semiconductor memory such as a flash memory. The program storage unit 222 stores a program for realizing the vehicle travel control unit 2220 and the NN execution control unit 2221.

The vehicle travel control unit 2220 has functions necessary for controlling the vehicle travel such as recognition, recognition, and judgment. It should be noted that the vehicle travel control unit 2220 does not have to independently execute recognition, recognition, and determination for controlling the vehicle travel, and is connected via, for example, another device mounted on the vehicle or a network 23. It may be shared or linked with other devices.

The NN execution control unit 2221 executes the inference process by using the reduced NN2200 stored in the storage unit 220. Although not shown, the electronic control device 22 may further have a learning function of the post-reduction neural network 2200 in addition to the inference processing. Since the electronic control device 22 has a learning function, the electronic control device 22 can independently learn and update the neural network without coordinating with the server 21.

(motion)
The reduction of the neural network by the server 21 will be described with reference to FIGS. 4 to 5. FIG. 4 is a flowchart showing the contraction processing of the neural network by the server 21. First, in step S400, the NN learning control unit 2115 generates a neural network having a learned weight coefficient value WLE, that is, a neural network before reduction.

Then, in step S401, the important data 202 is selected by referring to the data set management table 30 for the data stored in the storage unit 20. In this case, for example, data having a priority of 3 or more is classified into important data 202, and data having a priority of 2 or less is classified into normal data 203. In this classification, the operator of the server 21 may select the data, or the server 21 may select the data by calculation. Further, if the data has already been classified, the processing of S401 may be omitted.

Subsequently, in step S402, the calculation coefficient specifying unit 2110 identifies the weighting coefficient involved in the important data 202 in the learned weighting coefficient value WLE, and records the set of IDs as the important related WHI. That is, the important-related WHI is a list of ID numbers of weighting factors involved in the important data 202. The details of S402 will be described with reference to FIG. 5 below.

In the following step S403, the operation coefficient determination unit 2111 calculates the operation coefficient WCOEF based on the data set management table 30 and the important data 202. The operation coefficient WCOEF exists corresponding to each weight coefficient, and the initial values of all the operation coefficients WCOEF are the same, for example, zero. In the example shown in FIG. 3, the operation coefficient WCOEF corresponding to the weighting coefficient specified by using the important data 202 classified in the object detection data 1 in the uppermost stage is the value of the operation coefficient 303 of the data. "Is added. The operation factor WCOEF corresponding to the weighting factor specified by many important data 202 will have a larger value.

In step S404, the coefficient group calculation unit 2112 calculates the evaluation coefficient WNEW used for determining the thinning application location of the weighting coefficient in the neural network by using the important relation WHI and the operation coefficient WCOEF. The evaluation factor WNEW also exists for each weighting factor. The evaluation coefficient WNEW is calculated for each weighting factor, in other words, for the ID of each weighting factor, for example, as in the following equation 1.
WNEW = D (WHI) * Val (ID) * WCOEF ... (Equation 1)

However, D (WHI) is a function that returns "1" when the ID of the weighting factor of the processing target is included in the important related WHI, and returns zero when the ID of the weighting factor of the processing target is not included in the important related WHI. Is. Further, Val (ID) is a function that returns the weighting coefficient value of the weighting coefficient which is the ID of the processing target. As will be described in detail later, the evaluation coefficient WNEW is a value referred to in the thinning determination process, and the larger the value, the more difficult it is for thinning.

In the following step S405, the thinning unit 2113 determines the weighting coefficient to be thinned out based on the value of the evaluation coefficient WNEW. Then, in step S406, the weighting coefficient value of the weighting coefficient to be thinned out is set to zero from the learned weighting coefficient value WLE, and a new neural network is generated. However, if all of the row or column components of the weighting factor are zero, the row or column may be deleted.

Further, it may be deleted in a set unit of all the weighting coefficients connected to each arithmetic unit 100 (neuron). As a method, a method of calculating the sum of squares of each weighting coefficient connected to the arithmetic unit 100 and deleting the set of weighting coefficients in ascending order from the smallest value can be considered. The generated neural network is stored in the storage unit 20 as NN201 after reduction, transmitted to the electronic control device 22, and stored in the storage unit 220. The number of places where thinning is applied is one or more.

Then, in step S407, the weighting coefficient in the new neural network newly generated in step S406 is optimized by re-learning, that is, fine tuning. This fine tuning makes it possible to improve the inference accuracy. However, step S407 is not essential, and the process shown in FIG. 4 may be terminated when the execution of S406 is completed.

FIG. 5 is a flowchart showing the details of S402 in FIG. The execution subject of the process shown in FIG. 5 is the coefficient specifying unit 2110. First, in S500, the coefficient specifying unit 2110 causes the NN execution control unit 2114 to input important data 202 into the neural network generated in step S400 to execute inference, and calculates the output OHNU of each arithmetic unit of the neural network. In the following step S501, the coefficient specifying unit 2110 identifies the arithmetic unit NHI having a high degree of activation based on the output OHNU. It is determined that the arithmetic unit specified in this process is an arithmetic unit that is largely involved in the important data 202.

Here, a method of determining whether or not the degree of activation is high can be considered based on whether or not the output OHNU is equal to or higher than the threshold value ACTTH. As another method, it is conceivable to determine that a certain percentage of all the output OHNUs for each layer of the neural network has a higher degree of activation in order from the one with the largest value.

After that, in step S502, the coefficient specifying unit 2110 specifies the weighting coefficient connected to the calculation unit NHI for the calculation unit NHI specified in step S501, and sets the specified weighting coefficient as the important related WHI. This processing flow ends. To show an example of the processing in step S502, when the arithmetic unit NHI is the arithmetic unit 100 shown in FIG. 1, it is determined as follows. That is, the weighting coefficients connected to the arithmetic unit 100 are the coupling information 101 and the coupling information 102. In this way, the weighting factor connected to each arithmetic unit is uniquely specified from the network structure information of the neural network.

According to the first embodiment described above, the following effects can be obtained.
(1) The neural network update system 2 includes a server 21 and an electronic control device 22. The neural network update system 2 updates a neural network including a plurality of neurons 100 and a coupling information group including a plurality of coupling information shown by reference numerals 101 and 102 of FIG. 1 that associate the neurons with each other. The server 21 includes a storage unit 20 and a calculation unit 210. The storage unit 20 stores a first combination information group 2000 for determining an output value when an input value is given. The calculation unit 210 together with the coefficient specifying unit 2110 that specifies the second combined information group 2120, which is a plurality of combined information whose degree of influence on the predetermined data exceeds the predetermined value when the predetermined data is given as an input value. , A thinning unit that creates a third combined information group 2010 by deleting at least one combined information from the first combined information group 2000 based on the first combined information group 2000 and the second combined information group 2120. 2113 and. The electronic control device 22 stores the reduced NN2200 including the third coupling information group 2010.

Since the combined information is deleted in the third combined information group 2010 as compared with the first combined information group 2000, the amount of calculation, that is, the power consumption can be reduced. Furthermore, since the join information to be deleted is determined in consideration of the degree of influence on important data that requires high-precision inference, it is possible to limit the decrease in inference accuracy for important data.

(2) The predetermined data is data in which the priority regarding vehicle running exceeds the predetermined value. Therefore, it is possible to limit the deterioration of the inference system for data that requires highly accurate inference regarding the running of the vehicle.

(3) The thinning unit 2113 determines an operation coefficient group which is a set of operation coefficients corresponding to each connection information included in the second connection information group 2120, and sets each operation coefficient into the first connection information group 2000. The corresponding coupling information contained in is multiplied by the evaluation coefficient to obtain an evaluation coefficient group, and the corresponding coupling information is specified from the first coupling information group 2000 for the evaluation coefficients exceeding a predetermined value in the evaluation coefficient group. The join information other than the specified join information is deleted from the first join information group 2000, and the third join information group 2010 is created. In this way, since the combined information whose degree of influence on important data that requires high-precision inference exceeds a predetermined value is not deleted, it is possible to limit the decrease in inference accuracy for important data.

(4) The coefficient specifying unit 2110 identifies the neuron 100 involved in the determination of the output value when the output value when the predetermined data is given exceeds the predetermined value among the plurality of neurons 100, and the said The connection information associated with the specified neuron 100 is specified as the connection information constituting the second connection information group 2120.

(Modification 1)
The data stored in the storage unit 20, that is, the important data 202 and the normal data 203 may not include the relative distance to the vehicle traveling ahead. The data stored in the storage unit 20 may include information capable of determining whether or not high-precision inference is required. For example, when the data stored in the storage unit 20 includes the type and height of the object, the data set management table 30 is changed to the data set management table 31 shown in FIG.

The data set management table 31 shown in FIG. 6 has different contents of the priority determination condition 301 as compared with the data set management table 30 in the first embodiment, and the priority determination condition 301 has the identification class 3012 and the object height 3013. It is composed of and. The identification class 3012 indicates the type of the object, and in the example shown in FIG. 3, when the object is a vehicle, a pedestrian, or a bicycle, the priority is set high. The object height 3013 indicates the height of the object, and the higher the object height, the higher the priority is set.

When various information is included in the data stored in the storage unit 20, the priority determination condition 301 may be changed according to the application that uses this data. That is, it suffices if the priority 302 and the operation coefficient 303 can be determined for each data, and other information may be used for the priority determination condition 301. For example, the priority determination condition 301 may be a condition related to other automatic driving logics in which AI is used, such as self-position estimation, sensor fusion, map fusion, free space detection, object movement prediction, and travel track planning.

(Modification 2)
Even if the pre-reduction NN 220 is further stored in the storage unit 220 of the electronic control device 22, and if any problem occurs in the post-reduction NN2200, the operation is switched to the pre-reduction NN 220 and the arithmetic processing using the neural network is executed. good. Some problem is that, for example, by using the output result of NN2200 after reduction, sudden braking must be operated to avoid a collision. By switching to the pre-reduction NN 220 when such a problem occurs, it is possible to eliminate the influence on the vehicle running due to the reduction processing.

(Modification 3)
The method for determining the evaluation coefficient WNEW is not limited to the method using the formula 1 shown in the first embodiment. The value of the evaluation coefficient WNEW may be increased so that the weighting coefficients related to the important data 202 are less likely to be thinned out. For example, the evaluation coefficient WNEW may be calculated by performing a bit shift operation on the weighting coefficient value of the weighting coefficient, which is the ID of the processing target, using a correction value such as the operation coefficient WCOEF. Further, the evaluation coefficient WNEW may be calculated by adding a correction value such as an operation coefficient WCOEF to the weighting coefficient value of the weighting coefficient which is the ID of the processing target.

(Modification example 4)
FIG. 7 is a diagram showing details of step S402 shown in FIG. 3 in the modified example 4. That is, in this modification, the process shown in FIG. 6 is executed instead of the process shown in FIG. However, the same process as that of the first embodiment is assigned the same step number, and the description thereof will be omitted. In FIG. 7, first, step S500 is executed, and the output OHNU is calculated in the same manner as in the first embodiment.

In the following step S601, the NN learning control unit 2115 executes inference with the normal data 203 as an input to the neural network, and calculates the output OLNU of each arithmetic unit of the neural network. Then, in step S602, the calculation coefficient specifying unit 2110 identifies a calculation unit NHI having a high degree of activation based on OHNU and OLNU, for example, based on the difference between OHNU and OLNU. In the subsequent S502, the important related WHI is specified and the process shown in FIG. 7 is terminated as in the first embodiment.

FIG. 8 is a diagram showing an example of the process of step S602. FIG. 8A is a diagram showing OHNU for each arithmetic unit in step S600, FIG. 8B is a diagram showing OLNU for each arithmetic unit in step S601, and FIG. 8C is a diagram showing arithmetic units. It is a figure which shows the difference between OHNU and OLNU for each. However, in FIGS. 8 (a) to 8 (c), if the outputs of different arithmetic units are arranged in the horizontal axis direction and the positions in the horizontal directions of FIGS. 8 (a) to 8 (c) are the same, the same arithmetic unit is shown. ing.

In FIG. 8, an ID is assigned to each arithmetic unit. For example, the arithmetic unit having the ID “n11” is shown at the left end of FIGS. 8 (a) to 8 (c), and the difference between FIGS. 8 (a) and 8 (b) is shown in FIG. 8 (c). .. Then, in FIG. 8C, the threshold value WTH is set, and the calculation unit larger than the threshold value WTH is specified as the calculation unit NHI having a high degree of activation.

(Modification 5)
FIG. 9 is a diagram showing details of step S402 shown in FIG. 3 in the modified example 5. That is, in this modification, the process shown in FIG. 9 is executed instead of the process shown in FIG. However, the same process as that of the first embodiment is assigned the same step number, and the description thereof will be omitted. In this modification, the weighting factor is specified without specifying the arithmetic unit that is largely related to the important data 202.

In FIG. 9, first, in step S800, the NN learning control unit 2115 performs additional learning of the neural network using the important data 202, and calculates the weighting coefficient WHADDL after the additional learning. The data used in the additional learning in step S800 is different from the data used in generating the pre-reduction NN200.

In the following step S801, the coefficient specifying unit 2110 is involved in the important data 202 by calculating, for example, the difference between the weighting coefficient values of the weighting coefficient value WLE before the additional learning and the weighting coefficient value WHADDLE after the additional learning. Identify the weighting function. The weight coefficient value WLE before the additional learning is the weight coefficient value WLE calculated in S400 of FIG. Then, the process shown in FIG. 9 is terminated.

FIG. 10 is a diagram showing an example of the process shown in FIG. FIG. 10A is a diagram showing a weighting coefficient value WLE for each ID of the weighting coefficient of the pre-reduction NN200, and FIG. 10B shows a weighting coefficient value WHADDLE for each weighting factor ID in step S800. 10 (c) is a diagram showing the difference between the weighting coefficient value WHADDLE and the weighting coefficient value WLE for each weighting factor.

For example, it can be determined that a weighting coefficient in which the difference between the weighting coefficient value WHADDLE and the weighting coefficient value WLE is larger than the predetermined threshold value WTH is largely related to the important data 202. At this time, the absolute value of the difference between the weighting coefficient value WHADDLE and the weighting coefficient value WLE may be evaluated. Further, a certain ratio may be determined as a weighting coefficient having a high degree of activation in order from the one with the largest difference between the two.

(Modification 6)
FIG. 11 is a diagram showing details of step S402 shown in FIG. 3 in the modified example 6. That is, in this modification, the process shown in FIG. 11 is executed instead of the process shown in FIG. However, the same process as that of the first embodiment is assigned the same step number, and the description thereof will be omitted. In this modification, the weighting coefficient is specified without specifying the arithmetic unit that is largely related to the important data 202, as in the case of modification 5.

In FIG. 11, first, in step S900, the NN learning control unit 2115 learns the neural network using the important data 202 and calculates the weighting coefficient value WHLE. In the following step S901, the NN learning control unit 2115 learns the neural network using the normal data 203, and calculates the weight coefficient value WLLE. In the subsequent step S902, the coefficient specifying unit 2110 identifies the weighting coefficient involved in the important data 202 based on the learned weighting coefficient value WHLE and the weighting coefficient value WLLE, and names it as the important related WHI.

Note that steps S900 and S901 may be executed in the process of generating the neural network in step S400. Further, additional learning of the neural network may be performed using important data 202 and normal data 203 different from those used in step S400, whereby the processes of steps S900 and S901 may be performed.

An example of the process of step S902 is shown in FIG. FIG. 12A shows the weighting coefficient value WHLE for each ID of the weighting coefficient in step S900. FIG. 12B shows the weighting coefficient value WHADDLE for each ID of the weighting coefficient in step S901. FIG. 12 (c) is a diagram showing the difference between the weight coefficient value WHLE shown in FIG. 12 (a) and the weight coefficient value WHADDL shown in FIG. 12 (b) for each weight coefficient ID.

In FIG. 12 (c), it is determined whether or not each weighting coefficient is significantly related to the important data 202 by determining whether or not the difference value for each ID is equal to or greater than the threshold value WTH. Here, an absolute value may be taken for the difference result between the weight coefficient value WHLE and the weight coefficient value WLLE. As another method, a method of determining a certain ratio from the top in descending order of the difference value to have a high degree of activation can be considered.

(Modification 7)
FIG. 13 is a diagram showing a system configuration in the modified example 7. In this modification, the neural network contracted with respect to the electronic control device 22 is updated in cooperation with the server 21 by utilizing the traveling data acquired by the vehicle. The neural network update system 2 in FIG. 13 has a different configuration of the electronic control device 22 as follows, as compared with the system configuration shown in FIG.

In this modification, important data 2201 is further stored in the storage unit 220. The important data 2201 is the data extracted by the important data selection unit 2222, which will be described later, among the data obtained by sensing the surroundings of the vehicle by the sensor mounted on the vehicle. The program storage unit 222 further stores a program for executing the important data selection unit 2222, the NN update control unit 2223, and the important data upload unit 2224.

The important data selection unit 2222 selects data that is determined to have a high priority in relation to vehicle travel control from the data collected by external sensing or the like in the vehicle. As the selection method, the one described in FIG. 3 can be considered. As another method, although the accident did not occur, it is conceivable to select the sensing data regarding the situation where the sudden braking is operated to avoid the collision as important data. In addition to that, when the driver is manually driving, or when the automatic driving logic or driving support logic that does not use the neural network is being executed, the inference processing by the neural network is executed in the background, and the output result at the time of manual driving is executed. When there is a discrepancy between the output result of the logic that does not use the neural network or the inference processing result by the neural network, it is conceivable to select the input data to the neural network as important data.

The NN update control unit 2223 acquires the neural network newly generated by the server 21 and updates the reduced NN2200 stored in the storage unit 220. The important data upload unit 2224 uploads the important data 2201 selected by the important data selection unit 2222 and buffered in the storage unit 220 to the server 21 via the network 23.

(Modification 8)
In the first embodiment described above, a configuration in which the calculation coefficient is deleted in the reduction process has been described. However, neurons, or arithmetic units, may be deleted. The arithmetic unit can be deleted when all the arithmetic coefficients connected to the arithmetic unit become zero, or when all the arithmetic coefficients connected to the arithmetic unit are deleted.

-Second embodiment-
A second embodiment of the neural network update system will be described with reference to FIGS. 14 to 15. In the following description, the same components as those in the first embodiment are designated by the same reference numerals, and the differences will be mainly described. The points not particularly described are the same as those in the first embodiment. The present embodiment is different from the first embodiment mainly in that the ratio of thinning out the weighting coefficients is determined. In the present embodiment, the ratio of thinning out the weighting factor is referred to as "reduction rate".

FIG. 14 is a block diagram of the neural network update system 2 according to the second embodiment. In the configuration shown in FIG. 14, as compared with the configuration shown in FIG. 2, a program for realizing the reduction rate correction unit 2116 is further stored in the program storage unit 211 of the server 21, and the reduction rate correction is performed in the storage unit 20. Table 204 is further stored. The reduction rate correction unit 2116 corrects the reduction rate used in the thinning unit 2113 by using the arithmetic unit and weight coefficient information specified by the coefficient specifying unit 2110 and the reduction rate correction table 204. The reduction rate correction table 204 is information used for determining the correction value of the reduction rate.

FIG. 15 is a diagram showing an example of the reduction rate correction table 204. The reduction rate correction table 204 is a table for managing the number of arithmetic units 140, which are largely involved in important data, and the reduction rate 141 in association with each other. In the column of the number of important data-related arithmetic units 140, the number of arithmetic units that are largely involved in important data is described. In the column of reduction rate 141, the reduction rate used at the time of reduction is described. The reduction rate is ratio information indicating the degree to which the weighting factor is thinned out for each layer of the neural network. For example, if the specified value of the reduction rate is set to 20%, the weighting factor of the layer is thinned out by 20%.

In the example of the reduction rate 141, the description is based on the assumption that the reduction rate originally set is corrected by multiplication or division, but it should be set by another means such as bit shift calculation or addition / subtraction. May be. Further, the reduction rate may be set as an absolute value such as 10% or 20%. When the reduction rate correction table 204 is used, for example, when the number of arithmetic units specified by the coefficient specifying unit 2110 is 25 in any layer of the neural network, the reduction rate of that layer is this table. It is the normal reduction rate divided by 2 using the information. Although the processing in units of layers is described in this example, any processing unit in the neural network other than the units of layers may be used.

Each of the above-described embodiments and modifications may be combined. Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects considered within the scope of the technical idea of the present invention are also included within the scope of the present invention.

100 ... Arithmetic unit 2 ... Neural network update system 20 ... Storage unit 21 ... Server 22 ... Electronic control device 23 ... Network 200 ... Pre-reduction neural network 201 ... Post-reduction neural network 202 ... Important data 203 ... Normal data 204 ... Reduction Approximate rate correction table 210 ... Calculation unit 220 ... Storage unit 221 ... Calculation unit 2000 ... First coupling information group 2010 ... Third coupling information group 2110 ... Coefficient specifying unit 2111 ... Operation coefficient determination unit 2112 ... Coefficient group calculation unit 2113 ... Thinning unit 2114 ... NN execution control unit 2115 ... NN learning control unit 2116 ... Reduction rate correction unit 2120 ... Second coupling information group 2200 ... NN after reduction
2210 ... CPU
22001 ... Network structure information 30, 31 ... Data set management table

Claims (10)

An electronic control device that stores a neural network including a plurality of neurons and a group of connection information including a plurality of connection information that links the neurons to each other.
The neural network is
A first binding information group is the binding information group for determining the output value when the input value is given, impact for a given data given as the input value of the plurality of above a predetermined value In a neural network including a third coupling information group created by a thinning unit deleting at least one coupling information from the first coupling information group based on the second coupling information group which is the coupling information. An electronic control device. In the electronic control device according to claim 1,
The predetermined data is an electronic control device in which the priority regarding vehicle traveling exceeds a predetermined value. In the electronic control device according to claim 2,
The thinning unit determines an operation coefficient group which is a set of operation coefficients corresponding to each of the connection information included in the second connection information group, and each said operation coefficient is used as the first connection information group. An evaluation coefficient is calculated for the corresponding coupling information included to form an evaluation coefficient group, and for the evaluation coefficient exceeding a predetermined value in the evaluation coefficient group, the corresponding coupling information is obtained from the first coupling information group. An electronic control device that creates the third combination information group by specifying and deleting the combination information other than the specified combination information from the first combination information group. In the electronic control device according to claim 3,
The coefficient specifying unit identifies and identifies the neuron involved in the determination of the output value when the output value when the predetermined data is given exceeds the predetermined value among the plurality of neurons. An electronic control device that specifies connection information associated with a neuron as the connection information constituting the second connection information group. In the electronic control device according to claim 1,
The third coupling information group is an electronic control device in which the thinning unit deletes at least one coupling information from the first coupling information group and is optimized by fine tuning. A neural network update system that includes a server and an electronic control device, and updates a neural network including a plurality of neurons and a group of connection information including a plurality of connection information that links the neurons to each other.
The server includes a storage unit and a calculation unit.
The storage unit stores the first binding information group is the binding information group for determining the output value when the input value is given,
The arithmetic unit
When the predetermined data is given as the input value, the coefficient specifying unit for specifying the second combined information group which is a plurality of the combined information whose degree of influence on the predetermined data exceeds the predetermined value, and
Based on the first binding information group and the second binding information group, a thinning unit that deletes at least one binding information from the first binding information group to create a third binding information group. Prepare,
The electronic control device is a neural network update system that stores a neural network including the third coupling information group. In the neural network update system according to claim 6,
The predetermined data is a neural network update system in which the priority regarding vehicle running exceeds the predetermined value. In the neural network update system according to claim 7,
The thinning unit determines an operation coefficient group which is a set of operation coefficients corresponding to each of the connection information included in the second connection information group, and each said operation coefficient is used as the first connection information group. An evaluation coefficient is calculated for the corresponding coupling information included to form an evaluation coefficient group, and for the evaluation coefficient exceeding a predetermined value in the evaluation coefficient group, the corresponding coupling information is obtained from the first coupling information group. A neural network update system that creates the third coupling information group by specifying and deleting the coupling information other than the identified coupling information from the first coupling information group. In the neural network update system according to claim 8,
The coefficient specifying unit identifies and identifies the neuron involved in the determination of the output value when the output value when the predetermined data is given exceeds the predetermined value among the plurality of neurons. A neural network update system that specifies the connection information associated with a neuron as the connection information constituting the second connection information group. In the neural network update system according to claim 6,
The thinning unit is a neural network update system that deletes at least one join information from the first join information group, optimizes the join information group by fine tuning, and creates the third join information group.

Images