WO2023248454A1

WO2023248454A1 - Computation device and computation method

Info

Publication number: WO2023248454A1
Application number: PCT/JP2022/025251
Authority: WO
Inventors: 咲絵小松; 浩朗伊藤; 真岸本
Original assignee: 日立Astemo株式会社
Priority date: 2022-06-24
Filing date: 2022-06-24
Publication date: 2023-12-28

Abstract

This computation device comprises: an importance-degree calculation unit that uses a weighting coefficient set for a channel of a multilayer neural network model which has channels made up of a plurality of neurons per each of an input layer, an intermediate layer, and an output layer to calculate a degree of importance of the channel; a target contraction rate setting unit that sets a target contraction rate in the entirety of the multilayer neural network model; a per-layer contraction rate calculation unit that calculates a contraction rate of each layer of the multilayer neural network model on the basis of the degree of importance and the target contraction rate; a contraction unit that contracts each layer to match the contraction rates calculated by the per-layer contraction rate calculation unit, and generates a post-contraction multilayer neural network model; and a retraining unit that retrains the post-contraction multilayer neural network model.

Description

Arithmetic device and method

The present invention relates to an arithmetic device and an arithmetic method.

In recent years, there has been progress in technology that uses machine learning to apply object recognition and behavior prediction to autonomous vehicle driving. Furthermore, a DNN model using a deep neural network (hereinafter abbreviated as "DNN") is known as a machine learning method applied to object recognition and the like. Since the DNN model is composed of many layers, it is used as an example of a multilayer neural network.

Automated driving systems use DNN models to recognize objects around the vehicle (referred to as "surrounding recognition"), perform automatic steering, and perform automatic speed control to control the vehicle to its destination. Therefore, the DNN model is used in a learning process for learning the features of an object from an image, and an inference process for extracting an object from an image based on the learning results of the learning process.

Generally, when an automatic driving system using a DNN model controls automatic driving of a vehicle, it first acquires an image of the outside world from a camera, and converts the image into a format that can be used by the DNN model. In the inference process, an image whose format has been converted is used as an input image, and an object is extracted from the input image using a DNN model that has undergone learning processing in advance. Thereafter, the automatic driving system creates a surrounding map representing the outside world of the vehicle from the object extraction results, creates a behavior plan for the vehicle based on the surrounding map, and controls the vehicle based on the action plan.

The surrounding map may be, for example, a map in which the type of object shown in the image, such as a vehicle or a person, can be determined based on an image taken in front of the vehicle. However, to create the surrounding map, images not only taken of the front of the vehicle but also images taken of the rear of the vehicle and images taken of the sides of the vehicle may be used. Furthermore, when creating a surrounding map, an image taken of the surrounding area by a camera and information obtained from sensors such as LiDAR (Light Detection and Ranging) and Radar may be used in combination.

Patent Documents 1 and 2 describe various models using neural networks.
Patent Document 1 describes, "a first combination information group for determining an output value when an input value is given, and a plurality of combinations whose degree of influence on predetermined data given as an input value exceeds a predetermined value. a neural network including a third group of connected information created by a thinning unit deleting at least one piece of connected information from a first group of connected information based on a second group of connected information that is information. ing.

In Patent Document 2, ``learning is performed using a cost function that takes into consideration the degree of influence when deleting each channel/filter, and a scaling coefficient is obtained, and at least the degree of influence when deleting a channel/filter is calculated based on the scaling coefficient. By taking this into account and deciding what to delete, we can efficiently compress the model."

JP2020-42496A JP 2021-108039 Publication

The DNN model used in automatic driving systems has a very large number of operations because convolution operations consisting of multiplication and addition are repeatedly executed. Particularly in automatic driving, since the automatic driving system needs to continually update its action plan within a very short period of time, object recognition using a DNN model requires high-speed calculations. Therefore, when implementing a DNN model in an in-vehicle autonomous driving ECU (Electronic Control Unit), it is necessary to reduce the number of calculations. Therefore, by simplifying the DNN model using reduction, it is possible to reduce the number of operations. There are various methods for reducing the DNN model, but in this specification, one of the reduction methods will be described as one that uses weighting coefficients and processing to thin out neurons (pruning).

Conventionally, when reducing a DNN model, a uniform reduction rate was determined for all layers. However, when all layers of the DNN model are reduced, layers that contribute to improving object recognition accuracy are also reduced, resulting in a significant decrease in recognition accuracy.

For this reason, methods for determining the reduction rate of each layer of the DNN model have been studied. In this method, neurons in all layers are first uniformly deleted to the extent that it is expected that accuracy deterioration is unlikely to occur, and then a DNN model in which only neurons in a certain layer are significantly deleted is trained, resulting in deterioration in recognition accuracy. seek. Then, after sequentially changing the layers in which neurons were deleted significantly and comparing the deterioration in recognition accuracy, only the layers that could suppress the deterioration in recognition accuracy were deleted. However, since this process requires learning many times, the number of man-hours required to obtain the reduced DNN model is quite large.

When the DNN model was reduced using the technology shown in Patent Document 1, the recognition accuracy deteriorated significantly. Furthermore, in the technology described in Patent Document 2, relearning is repeated using a DNN model reduced based on reduction rates set in various patterns, so the man-hours required to reduce the DNN model are extremely large. The number of cases was increasing.

The present invention has been made in view of this situation, and it is an object of the present invention to make it possible to reduce the number of man-hours required for reduction of a multilayer neural network.

The arithmetic device according to the present invention calculates the importance of a channel using a weighting coefficient set for a channel of a multilayer neural network model that has channels each composed of a plurality of neurons in an input layer, a middle layer, and an output layer. an importance calculation unit that sets a target reduction rate for the entire multilayer neural network model; a target reduction rate setting unit that sets a target reduction rate for the entire multilayer neural network model; and a reduction rate calculation unit that sets a target reduction rate for the entire multilayer neural network model. A reduction rate calculation unit for each layer that calculates the reduction rate, and a reduction unit that reduces each layer according to the reduction rate calculated by the reduction rate calculation unit for each layer and generates a multilayer neural network model after reduction. and a relearning unit that retrains the reduced multilayer neural network model.

According to the present invention, it is possible to reduce the number of man-hours required for reduction of a multilayer neural network.

1 is a block diagram showing an example of the overall configuration of an automatic driving system according to a first embodiment of the present invention. 1 is a block diagram showing an example of a hardware configuration of a computer according to a first embodiment of the present invention. FIG. 3 is a flowchart illustrating an example of processing of the DNN reduction device according to the first embodiment of the present invention. FIG. 2 is a simplified diagram showing a trained DNN model according to the first embodiment of the present invention. 1 is a diagram showing a basic form of a convolutional neural network according to a first embodiment of the present invention. FIG. FIG. 3 is a diagram illustrating an example of a reduced DNN model according to the first embodiment of the present invention. It is a block diagram showing an example of composition of an automatic driving system provided with a DNN condensation device concerning a 2nd embodiment of the present invention. It is a flow chart which shows an example of processing of the DNN condensation device concerning a 2nd embodiment of the present invention. It is a block diagram showing an example of composition of an automatic driving system provided with a DNN condensation device concerning a 3rd embodiment of the present invention. It is a flow chart which shows an example of processing of the DNN condensation device concerning a 3rd embodiment of the present invention. It is a block diagram showing an example of composition of an automatic driving system provided with a DNN condensation device concerning a 4th embodiment of the present invention.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functions or configurations are designated by the same reference numerals and redundant explanations will be omitted. In the following embodiments, an example will be described in which the present invention is applied to a computing device that can communicate with an in-vehicle ECU for vehicle control, for example, an advanced driver assistance system (ADAS) or an autonomous driving (AD). explain.

[First embodiment]
<Overall configuration of automatic driving system>
First, a configuration example of an automatic driving system 1 using a DNN condensation device 100 according to a first embodiment of the present invention will be described with reference to FIG. 1.
FIG. 1 is a block diagram showing an example of the overall configuration of an automatic driving system according to a first embodiment.

The automatic driving system 1 is composed of a DNN reduction device 100 and an automatic driving ECU 300 installed in the vehicle 500, and controls the recognition of the surroundings of the vehicle 500 and automatic steering, and automatically drives the vehicle 500 to the destination by automatic speed control. control. The vehicle 500 is also equipped with a camera 200 and an actuator 400. Automatic driving ECU 300 controls automatic driving of vehicle 500 using contracted DNN model 40 contracted from trained DNN model 10 by DNN reduction device 100 .

The camera 200 is an example of a monocular camera or a stereo camera that can photograph the outside world of the vehicle 500 using visible light and infrared rays. Camera 200 outputs an image of the outside world to automatic driving ECU 300 as outside world information.

The actuator 400 is a device that is driven under the control of the automatic driving ECU 300, and operates various parts such as an accelerator, a brake, and a steering wheel.

First, an example of the internal configuration and operation of the automatic driving ECU 300 will be described.
The automatic driving ECU 300 includes an external world recognition section 310, a risk prediction section 320, and an action planning section 330. Each functional part of automatic driving ECU 300 shown in FIG. 1 is a main part necessary for automatic driving of vehicle 500, and description and illustration of other functional parts necessary for automatic driving will be omitted.

The external world recognition unit 310 recognizes objects in the external world based on external world information acquired from the camera 200. The object recognized by the external world recognition unit 310 is output to the risk prediction unit 320 as an object recognition result. The external world recognition unit 310 uses a reduced DNN model 40 output from the DNN relearning unit 150, which will be described later, when performing external world recognition processing.

The risk prediction unit 320 uses the object recognition results input from the external world recognition unit 310 to predict possible risks based on the current external world situation. The risk predicted by the risk prediction unit 320 is output to the action planning unit 330 as a risk prediction result.

The action planning unit 330 generates an action plan including the traveling direction and traveling speed of the vehicle 500 based on the risk prediction results input from the risk predicting unit 320. The action plan generated by the action planning unit 330 is output to the actuator 400. The actuator 400 realizes automatic driving of the vehicle 500 by driving each part of the vehicle 500 according to the action plan.

Next, an example of the internal configuration of the DNN reduction device 100 will be described.
It is assumed that the DNN reduction device 100 is installed in a cloud server installed at a location away from the vehicle 500. The automatic driving ECU 300 is configured to download the reduced DNN model 40 from the DNN reduction device 100 every time the automatic driving ECU 300 is started. By providing the DNN reduction device 100 in the cloud server, the load of reduction processing of the learned DNN model 10, etc. can be shouldered by the cloud server.

The DNN reduction device 100 is a device that performs reduction processing on the trained DNN model 10. Reduction processing of the learned DNN model 10 requires a large amount of calculation processing. This DNN reduction device 100 includes an importance calculation section 110, a target reduction rate setting section 120, a reduction condition setting section 130, a DNN reduction section 140, and a DNN relearning section 150. In the diagrams below, rectangles surrounded by solid lines represent subjects of various types of processing, and rectangles surrounded by dashed-dotted lines represent various types of data or information.

The importance calculation unit 110 and the target reduction rate setting unit 120 are connected to each layer reduction rate calculation unit 131.
The importance calculation unit (importance calculation unit 110) is set to a channel of a multilayer neural network model (trained DNN model 10), which has a channel composed of a plurality of neurons for each input layer, intermediate layer, and output layer. The importance of the channel is calculated using the weighting coefficient 20 for each channel. This multilayer neural network model is a deep neural network model and is used as the trained DNN model 10. Therefore, the importance calculation unit 110 extracts the weighting coefficient 20 of each channel (hereinafter abbreviated as “ch”) from the learned DNN model 10. Then, the importance calculation unit 110 calculates the importance of each channel using the weighting coefficient 20 of each channel. The degree of importance is an index representing the influence of each channel of the trained DNN model 10 on recognition accuracy. The importance of each channel calculated by the importance calculation unit 110 is output to each layer reduction rate calculation unit 131.

Here, channels in the trained DNN model 10 will be explained with reference to FIGS. 4 and 5.
FIG. 4 is a schematic diagram of the trained DNN model 10.
The trained DNN model 10 is an example of a multilayer neural network, and is composed of an input layer, a plurality of intermediate layers, and an output layer. Each layer of the trained DNN model 10 is composed of a neural network including a plurality of neurons. A neuron in a certain layer of the intermediate layer is connected to a plurality of neurons in the front layer and a plurality of neurons in the rear layer.

FIG. 5 is a diagram showing the basic form of a convolutional neural network.
The input image data (indicated by the number "7") is divided into predetermined sizes and taken into the input layer. The value of a neuron in a certain layer is convolved with the values of a plurality of neurons in a previous layer by a filter, and is taken into a plurality of convolution layers for each filter. Then, pooling processing is performed on each convolutional layer to generate a pooling layer. Here, one cell in the pooling layer is called a neuron, and a set of each neuron in the pooling layer is called a channel (ch). The outputs of the pooling layer are combined into a fully connected layer. Then, the output layer connected to the fully connected layer outputs determination probabilities from "0" to "9". The eighth black circle from the top of the output layer indicates that the determined number has the highest probability of being "7". Note that in the trained DNN model 10 shown in FIG. 4, each intermediate layer includes a plurality of sets of convolution layers and pooling layers.

Returning to FIG. 1, the explanation will continue. A target reduction rate setting unit (target reduction rate setting unit 120) sets a target reduction rate for the entire multilayer neural network model (trained DNN model 10). For example, the target reduction rate setting unit 120 sets what percentage of channels to reduce in the entire trained DNN model 10, not for each layer of the trained DNN model 10. Therefore, the target reduction rate setting unit 120 sets a target reduction rate based on the target reduction rate 30 inputted in advance by the user. The target reduction rate set by the target reduction rate setting unit 120 is output to each layer reduction rate calculation unit 131.

The reduction condition setting unit 130 sets conditions for reducing the DNN from conditions such as the weighting coefficient 20 and the number of operations. The reduction condition setting unit 130 includes a reduction rate calculation unit 131 for each layer.

Each layer reduction rate calculation unit (each layer reduction rate calculation unit 131) calculates the reduction rate of each layer of the multilayer neural network model (trained DNN model 10) based on the importance level and the target reduction rate. . This layer reduction rate calculation section 131 is connected to the DNN reduction section 140. Each layer reduction rate calculation unit 131 calculates the learned DNN model based on the importance calculated by the importance calculation unit 110 and the reduction rate of the entire trained DNN model 10 set by the target reduction rate setting unit 120 The reduction rate of each layer is calculated so that the DNN model 10 reaches the target reduction rate. For example, each layer reduction rate calculation unit 131 calculates the reduction rate of each layer by inputting the importance of each channel and the set target reduction rate. The reduction rate of each layer calculated by the layer reduction rate calculation unit 131 is output to the DNN reduction unit 140.

The reduction unit (DNN reduction unit 140) reduces each layer according to the reduction rate calculated by the layer reduction rate calculation unit (each layer reduction rate calculation unit 131), and generates a multilayer neural network after reduction. A network model (reduced DNN model 40) is generated. This DNN reduction unit 140 is connected to a DNN relearning unit 150. The trained DNN model 10 and the reduction rate of each layer calculated by the layer reduction rate calculation unit 131 are input to the DNN reduction unit 140 . Then, the DNN reduction unit 140 reduces the learned DNN model 10 according to the reduction rate of each layer set by the layer reduction rate calculation unit 131. For example, if 10% is set as the reduction rate for a certain layer of the trained DNN model 10, reduction is performed to delete 10% of the channels in this layer. The learned DNN model 10 reduced by the DNN reduction unit 140 is output to the DNN relearning unit 150 as the reduced DNN model 40.

The relearning unit (DNN relearning unit 150) performs relearning of the reduced multilayer neural network model (reduced DNN model 40). This DNN relearning section 150 is connected to the external world recognition section 310 of the automatic driving ECU 300. The reduced DNN model 40 is input to the DNN relearning unit 150 from the DNN reduction unit 140 . Here, the reduced DNN model 40 input to the DNN relearning unit 150 is not shown. The DNN relearning unit 150 then relearns the input reduced DNN model 40. For example, the DNN relearning unit 150 reads learning data (not shown), retrains the reduced DNN model 40, reads evaluation data (not shown), and evaluates the reduced DNN model 40. Then, if the DNN relearning unit 150 confirms that the object recognition accuracy of the reduced DNN model 40 is higher than the target recognition accuracy, the DNN relearning unit 150 outputs the reduced DNN model 40 to the external world recognition unit 310 of the automatic driving ECU 300. do. In this way, the DNN relearning unit 150 retrains the reduced DNN model 40 at least once.

Note that the DNN reduction device 100 may be mounted on the vehicle 500. In this case, even if the vehicle 500 is in an environment where it cannot connect to a wireless network (such as a tunnel or an underground road), the reduced DNN model 40 generated and retrained by the DNN reduction device 100 is immediately connected to the external world recognition unit 310. is input. Therefore, even in an environment where vehicle 500 cannot connect to a wireless network, external world recognition unit 310 can accurately recognize objects.

<Computer hardware configuration>
Next, the hardware configuration of the computer 600 that constitutes the DNN reduction device 100 will be explained.
FIG. 2 is a block diagram showing an example of the hardware configuration of the computer 600. Computer 600 is an example of hardware used as a computer that can operate as DNN reduction device 100 according to this embodiment. The DNN reduction device 100 according to the present embodiment realizes a DNN reduction calculation method performed by the functional blocks shown in FIG. 1 in cooperation with each other by having the computer 600 (computer) execute a program.

The computer 600 includes a CPU (Central Processing Unit) 610, a ROM (Read Only Memory) 620, and a RAM (Random Access Memory) 630, each connected to a bus 640. Further, the computer 600 includes a nonvolatile storage 650 and a network interface 660.

The CPU 610 reads out software program codes that implement each function according to the present embodiment from the ROM 620, loads them into the RAM 630, and executes them. Variables, parameters, etc. generated during the calculation process of the CPU 610 are temporarily written to the RAM 630, and these variables, parameters, etc. are read out by the CPU 610 as appropriate. However, instead of the CPU 610, an MPU (Micro Processing Unit) may be used. The CPU 610 realizes processing of each functional unit included in the DNN reduction device 100.

As the non-volatile storage 650, for example, an HDD (Hard Disk Drive), SSD (Solid State Drive), flexible disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, or non-volatile memory is used. It will be done. In addition to an OS (Operating System) and various parameters, programs for operating the computer 600 are recorded in the nonvolatile storage 650. The ROM 620 and the non-volatile storage 650 record programs and data necessary for the operation of the CPU 610, and serve as an example of a computer-readable non-transitory storage medium that stores programs executed by the computer 600. used. The trained DNN model 10 and the reduced DNN model 40 are stored in the nonvolatile storage 650.

For example, a NIC (Network Interface Card) is used as the network interface 660, and various data can be sent and received between devices via a LAN (Local Area Network) connected to a terminal of the NIC, a dedicated line, etc. It is possible. DNN reduction device 100 transmits reduced DNN model 40 to automatic driving ECU 300 via network interface 660.

<Example of operation of DNN reduction device>
Next, the operation of the DNN reduction device 100 will be explained with reference to FIG. 3.
FIG. 3 is a flowchart illustrating an example of processing by the DNN reduction device 100 according to the first embodiment.

As described above, the DNN reduction device 100 has the trained DNN model 10 that has been trained in advance. Then, the DNN reduction device 100 obtains the weighting coefficient 20 of each channel from the learned DNN model 10.

First, the importance calculation unit 110 calculates the importance of each channel (S1). Here, the importance calculation procedure performed by the importance calculation unit 110 will be explained with reference to FIG. 4.
FIG. 4 is a simplified diagram of the trained DNN model 10.

Each intermediate layer of the trained DNN model 10 shown in FIG. 4 includes five channels. Here, in FIG. 4, 1ch in the middle layer in the trained DNN model 10 is shown as a region 501. Next, a procedure in which the importance calculation unit 110 calculates the importance of the channel shown in the area 501 will be described.

The importance of each channel represents the degree of influence of each channel on recognition accuracy. It is known that among the channels of each layer of the intermediate layer, the average value of the importance of the layer near the input layer is large, and the average value of the importance of the layer near the output layer is small. The channel having a higher degree of importance is less likely to be deleted in later processing. Conventionally, layers that were close to the output layer and had low importance required a small number of operations, so almost all layers were sometimes deleted.

Therefore, the importance calculation unit 110 calculates the importance so that a channel with a large variance of the weighting coefficient 20 or a channel whose characteristics are different from other channels has a high importance. For example, a channel with an importance level of 0.9 can be said to have a large variance. Furthermore, since a channel that differs from the characteristics of other channels reflects a feature amount that is different from that of other channels, recognition accuracy may be affected if it is deleted in later processing. Therefore, the importance calculation unit 110 calculates a high importance of a channel that has characteristics different from those of other channels, thereby making it difficult for this channel to be deleted in later processing.

The importance calculation unit (importance calculation unit 110) calculates the priority of the previous stage connected to the subsequent channel, based on the importance of the subsequent channel and the weighting coefficient 20 of each subsequent channel, starting from the layer closest to the output layer. Calculate channel importance. In the trained DNN model 10 shown in FIG. 4, the importance of the channels subsequent to the region 501 (that is, the channels on the right side of the region 501) are 0.6, 0.9, 0.4, 0.2, 0. It is 1. Therefore, the importance calculation unit 110 calculates the importance of the region 501 using the following equation (1), assuming that the weighting coefficients 20 of each channel are w1, w2, w3, w4, and w5.

Normally, when using the weighting coefficient 20 of each channel in calculations, the absolute value is not used because the weighting coefficient 20 takes either a positive or negative value. However, in Equation (1), whether the weighting coefficient 20 is positive or negative, if the value becomes large, the influence on the calculation is large and the channel should not be deleted, so the absolute value is used.

The explanation will be continued by returning to FIG. 3 again.
After step S1, the target reduction rate setting unit 120 sets the target reduction rate 30 set in advance by the user (S2). The target reduction rate 30 is an index indicating how much the number of operations is reduced in the trained DNN model 10 (reduced DNN model 40), This is the target reduction rate.

Next, each layer reduction rate calculation unit (each layer reduction rate calculation unit 131) calculates the entire multilayer neural network model (trained DNN model 10) based on the importance in each layer of the importance calculated for each channel. Calculate the target reduction rate of each layer with respect to the target reduction rate in . For example, each layer reduction rate calculation unit 131 calculates the target reduction rate of each layer based on the importance of each channel calculated by the importance calculation unit 110 and the target reduction rate 30 set by the target reduction rate setting unit 120. A reduction rate is calculated (S3). The target reduction rate of each layer is an index that indicates which channel is to be reduced and by how much among the channels in each layer of the trained DNN model 10. Compared to , it is possible to contract each layer.

Next, the DNN reduction unit 140 reduces the trained DNN model 10 based on the input trained DNN model 10 and the target reduction rate of each layer calculated by the layer reduction rate calculation unit 131. . The DNN relearning unit 150 retrains the reduced DNN model 40 using the reduced trained DNN model 10 as the reduced DNN model 40 (S4).

Next, the DNN relearning unit 150 outputs the relearned reduced DNN model 40 to the automatic driving ECU 300 of the vehicle 500 (S5). After the process of step S5, the process of the DNN reduction device 100 ends.

Here, a description will be given of how the trained DNN model 10 shown in FIG. 4 is reduced.
FIG. 6 is a diagram showing an example of the reduced DNN model 40. Here, a reduction process when the target reduction rate 30 in the trained DNN model 10 is set to 40% will be described.

Note that to simplify the explanation, it is assumed that the number of operations in each intermediate layer is the same. In this case, the DNN reduction unit 140 reduces channels in order of importance from among the plurality of channels included in the intermediate layer. For example, the DNN reduction unit 140 reduces 6 channels whose target reduction rate 30 is 40% from the 15 channels in the middle layer of the learned DNN model 10 shown in FIG. 4 in descending order of importance. As a result, the reduced DNN model 40 shown in FIG. 6 is created in which the number of channels in the middle layer is 9. This reduced DNN model 40 is retrained by the DNN retraining unit 150 and then output to the external world recognition unit 310.

In the DNN reduction device 100 according to the first embodiment described above, the channels of the trained DNN model 10 are reduced in descending order of importance according to the target reduction rate 30, and the reduced DNN model 40 is Creating. Then, the DNN reduction device 100 obtains an optimal reduction rate in advance by using the importance calculated from the weighting coefficient 20 of each channel in the importance analysis and the number of operations for each layer. Therefore, the DNN reduction device 100 can suppress a decrease in object recognition accuracy using the reduced DNN model 40 by reducing the trained DNN model 10 using the importance level. Further, the DNN reduction device 100 only needs to relearn the reduced DNN model 40 at least once. Therefore, there is no need to repeat relearning of the DNN model as in the past, and the number of man-hours required to reduce the learned DNN model 10 can be reduced.

Additionally, the number of channels of the reduced DNN model 40 is reduced compared to the trained DNN model 10 before reduction. Therefore, the external world recognition unit 310 can reduce the number of operations in the external world recognition process using the reduced DNN model 40 and quickly obtain necessary information such as the external world recognition result.

Furthermore, since the learned DNN model 10 is reduced according to the target reduction rate 30 set by the user, the learned DNN model 10 is reduced to the reduction rate intended by the user. In other words, the trained DNN model 10 may be reduced too much and the recognition accuracy of the reduced DNN model 40 may decrease, or conversely, the number of operations performed by the automatic driving ECU 300 may increase without being reduced. There is no.

Note that in the first embodiment, external information is acquired from the image captured by the camera 200, but this is done using a sensor that can acquire the distance to an object and the type of the object, such as Lidar, Radar, or a far-infrared camera. The means for acquiring external information is not limited to the camera 200. Further, a sensor capable of acquiring external information may be used alone, or a plurality of sensors may be used in combination.

Furthermore, the automatic driving ECU 300 may store the external world recognition result by the external world recognition unit 310, the risk prediction result by the risk prediction unit 320, and the action plan by the action planning unit 330. Then, the DNN reduction device 100 may acquire this information from the automatic driving ECU 300, calculate a DNN model anew, and use it as the learned DNN model 10.

[Second embodiment]
The layer reduction rate calculation unit 131 of the DNN reduction device 100 according to the first embodiment shown in FIG. 1 calculates the reduction rate of each layer based on the importance of each channel. However, if only the importance of each channel is used, the weight coefficient 20 of the latter layer (layer close to the output layer) of the trained DNN model 10 is likely to be calculated small, so the importance of the channels in the latter layer becomes small. Therefore, channels in the latter half of the layer are likely to be subject to contraction. However, since the latter layer of the trained DNN model 10 originally has a small number of operations, it is assumed that it is difficult to obtain the effect of reduction. Therefore, when the layer reduction rate calculation unit 131 calculates the reduction rate of each layer, the number of operations is also taken into consideration. As a result, it is possible to avoid reducing only layers with a small number of operations.

Therefore, a configuration example and an operation example of an automatic driving system according to a second embodiment of the present invention will be described with reference to FIGS. 7 and 8.
FIG. 7 is a block diagram showing a configuration example of an automatic driving system 1A including a DNN condensation device 100A according to the second embodiment. In FIG. 7, functional blocks that perform the same processing as those in FIG. 1 are given the same names and numbers, and unless otherwise specified, descriptions will be omitted as they have the same or similar functions.

An automatic driving system 1A according to the second embodiment shown in FIG. 7 includes a DNN reduction device 100A and an automatic driving ECU 300.
The reduction condition setting section 130A included in the DNN reduction device 100A additionally includes a parameter setting section 132 in addition to the layer reduction rate calculation section 131. The parameter setting unit 132 sets parameters that contribute when the layer reduction rate calculation unit 131 calculates the reduction rate of each layer. This parameter is also called a contribution parameter.

The parameter setting unit (parameter setting unit 132) sets the degree of importance and the number of operations for each layer extracted from the multilayer neural network model (trained DNN model 10) as parameters. In addition to the importance of each channel calculated by the importance calculation unit 110, the number of operations 50 for each layer extracted from the trained DNN model 10 is input to the parameter setting unit 132. For example, in the trained DNN model 10 shown in FIG. 4, the number of operations 50 in each layer is the number of connection lines of a plurality of channels in the next layer connected to one cn in a certain layer. Since FIG. 4 is an example of the simplified trained DNN model 10, 5 channels in the next layer (the middle layer of the intermediate layer) are connected to 1 channel in the layer on the input side of the intermediate layer. The number of operations between channel 1 of the layer on the input side of the intermediate layer and channel of the next layer is "5". Since the number of operations for each channel in a certain layer with respect to the channel in the next layer is determined, the total number of operations for the channels in each layer is determined as the number of operations 50 for each layer.

Then, the parameter setting unit 132 sets parameters that determine the degree of importance and the contribution rate of the number of calculations, depending on whether the recognition accuracy and calculation speed of the trained DNN model 10 achieve each goal. At this time, the parameter setting unit (parameter setting unit 132) calculates the degree of contribution of the importance calculated for each channel and the number of calculations 50 for each layer, which are used when each layer reduction rate calculation unit 131 calculates the reduction rate. The contribution degree is set as a parameter (contribution degree parameter). By setting the contribution of the importance calculated for each channel and the contribution of the 50 operations in each layer as parameters (contribution parameters), a learned DNN is created based on the importance and the number of operations 50 in each layer. Model 10 is reduced. Note that parameter setting may also be referred to as parameter adjustment. The contribution parameter set by the parameter setting unit 132 is input to each layer reduction rate calculation unit 131.

Each layer reduction rate calculation unit (each layer reduction rate calculation unit 131) calculates the target reduction rate of each layer based on the parameter (contribution parameter) and the target reduction rate. For example, each layer reduction rate calculation unit 131 calculates the reduction rate of each layer based on the reduction rate, importance, number of operations, and contribution parameters of the entire trained DNN model 10 determined by the target reduction rate setting unit 120. calculate.

FIG. 8 is a flowchart illustrating an example of processing by the DNN reduction device 100 according to the second embodiment. In addition, in the process shown in FIG. 8, the same process as in FIG. 3 is given the same number, and the description will be omitted as having the same or similar function unless otherwise specified.

As shown in FIG. 3, the DNN reduction device 100 has a trained DNN model 10 that has been trained in advance. Then, the DNN reduction device 100 obtains the weighting coefficient 20 of each channel from the learned DNN model 10. Then, the importance calculation unit 110 calculates the importance of each channel based on the weighting coefficient 20 of each channel (S1).

Next, the parameter setting unit 132 of the DNN reduction device 100 obtains the number of operations 50 for each layer in addition to the importance of each channel calculated from the weighting coefficient 20 of each channel according to the first embodiment ( S11).

Next, the parameter setting unit 132 sets the contribution parameter of each channel when contracting the trained DNN model 10 based on the contribution of the importance of each channel and the number of operations of each channel ( S12). Then, the parameter setting unit 132 applies the set contribution parameter to the following equation (2) for calculating the reduction rate of each layer, and calculates the reduction rate of each layer (S13). For example, the parameter setting unit 132 sets the degree of contribution x of importance to 0.5 and the degree of contribution y of number of operations to 0.5.

In formula (2), C(L) represents the reduction rate of each layer. P(L) represents the importance of each layer. x represents the degree of contribution of importance. O(L) represents the number of operations 50 in each layer. y represents the degree of contribution of the number of operations. base_rate represents the target reduction rate of 30.

In the above example, the contribution x of the importance and the contribution y of the number of operations are both the same 0.5, but the contribution x of the importance is 0.3 and the contribution y of the number of operations is 0. A different value of .6 is also envisaged. For example, the initial value of the contribution degree x of importance and the initial value of the contribution degree y of operation number can be determined in advance by the user. The processing after the reduction ratio of each layer is calculated by the layer reduction ratio calculation unit 131 in step S13 is the same as the processing in the DNN reduction apparatus 100 according to the first embodiment.

In the automatic driving system 1A according to the second embodiment described above, the importance of each channel analyzed by the importance calculation unit 110 using importance analysis and the number of operations of each layer extracted from the trained DNN model 10. 50 is used to set the contribution parameter of the importance level and the number of operations. Then, each layer reduction rate calculation unit 131 calculates the reduction rate of each layer based on the contribution parameter. By using the contribution degree parameter of the number of operations to calculate the reduction rate, each layer reduction rate calculation unit 131 can avoid only layers with a small number of operations being targeted for reduction.

Furthermore, by using the importance contribution parameter in calculating the reduction rate, each layer reduction rate calculation unit 131 can avoid layers with high importance from becoming reduction targets. As a result, the DNN reduction unit 140 reduces the learned DNN model 10 according to the target reduction rate 30 by leaving the channels with high importance in the trained DNN model 10 and reducing the channels with a large number of operations. becomes possible. As a result, the automatic driving ECU 300 can quickly execute the calculation process for external world recognition using the reduced DNN model 40 in which the total number of calculations is greatly reduced. Moreover, automatic driving ECU 300 can reliably recognize an object to be recognized and control the driving of vehicle 500 while avoiding this object.

[Third embodiment]
When the learned DNN model 10 is reduced using the method according to the first embodiment described above (see FIG. 3), the recognition accuracy of the reduced DNN model 40 may decrease. Similarly, even when the learned DNN model 10 is reduced using the method according to the second embodiment (see FIG. 8), the recognition accuracy of the reduced DNN model 40 may decrease. . Therefore, the DNN reduction device according to the third embodiment checks the recognition accuracy of the reduced DNN model 40 and adjusts the reduction rate based on the confirmation result of the recognition system, thereby achieving recognition accuracy that ensures safety. Try to keep it.

Therefore, a configuration example and an operation example of an automatic driving system according to a third embodiment of the present invention will be described with reference to FIGS. 9 and 10.
FIG. 9 is a block diagram showing a configuration example of an automatic driving system 1B including a DNN reduction device 100B according to the third embodiment. In FIG. 9, blocks that perform the same processing as those in FIG. 7 are given the same names and numbers, and unless otherwise explained, they are assumed to have the same or similar functions and will not be described.

An automatic driving system 1B according to the third embodiment includes a DNN reduction device 100B and an automatic driving ECU 300. The DNN reduction device 100B has a configuration in which an FPS (Frames Per Second) confirmation unit 160 and a recognition accuracy confirmation unit 170 are newly added to the DNN reduction device 100A according to the second embodiment. In the following explanation, fps is also referred to as calculation speed.

The DNN relearning section 150 is connected to the FPS confirmation section 160. Therefore, the reduced DNN model 40 retrained by the DNN relearning unit 150 is output to the FPS confirmation unit 160.

The calculation speed checking unit (FPS checking unit 160) checks the calculation speed from the re-learned reduced multilayer neural network model (reduced DNN model 40). This FPS confirmation section 160 is connected to the parameter setting section 132 and the recognition accuracy confirmation section 170. The FPS confirmation unit 160 measures the calculation speed of the reduced DNN model 40 input from the DNN relearning unit 150, and checks whether the calculation speed of the reduced DNN model 40 satisfies the target calculation speed. do. For example, the recognition accuracy confirmation unit 170 causes the reduced DNN model 40 to read several dozen images whose data amount is known, and calculates the number of images that the reduced DNN model 40 can process within a predetermined time. Find it as speed. The faster the calculation speed is, the more images the reduced DNN model 40 can process per second. The result of checking the calculation speed of the reduced DNN model 40 by the FPS checking unit 160 is output to the parameter setting unit 132. Further, the FPS confirmation unit 160 outputs the reduced DNN model 40 to the recognition accuracy confirmation unit 170.

The recognition accuracy confirmation unit (recognition accuracy confirmation unit 170) confirms the recognition accuracy of the re-learned reduced multilayer neural network model (reduced DNN model 40). This recognition accuracy confirmation section 170 is connected to the parameter setting section 132 and the external world recognition section 310 of the automatic driving ECU 300. The recognition accuracy confirmation unit 170 measures the recognition accuracy of the reduced DNN model 40 and checks whether the recognition accuracy of the reduced DNN model 40 satisfies the target recognition accuracy. For example, the recognition accuracy confirmation unit 170 causes the reduced DNN model 40 to read several dozen images in which objects are known in advance, and calculates the rate at which the reduced DNN model 40 can correctly recognize the object as the recognition accuracy. . The recognition accuracy confirmation result of the reduced DNN model 40 by the recognition accuracy confirmation unit 170 is output to the parameter setting unit 132. Further, the recognition accuracy confirmation unit 170 determines that the calculation speed of the reduced DNN model 40 satisfies the target calculation speed by the FPS confirmation unit 160, and the recognition accuracy confirmation unit 170 determines that the calculation speed of the reduced DNN model 40 satisfies the target calculation speed. If it is determined that the recognition accuracy meets the target recognition accuracy, the reduced DNN model 40 is output to the external world recognition unit 310 of the automatic driving ECU 300.

Note that if the configuration of the reduced DNN model 40 becomes too simple, the calculation speed of the reduced DNN model 40 will become too fast, but the recognition accuracy may also decrease. Therefore, the DNN reduction device 100B checks not only the calculation speed of the reduced DNN model 40 but also the recognition accuracy to determine whether the reduced DNN model 40 with high calculation speed and high recognition accuracy has been obtained. This allows you to check.

Furthermore, the calculation speed and recognition accuracy of the reduced DNN model 40 may vary depending on the device on which it is executed. Therefore, the FPS confirmation unit 160 and the recognition accuracy confirmation unit 170 confirm the calculation speed and recognition accuracy according to the type of CPU of the automatic driving ECU 300 on which the reduced DNN model 40 is executed. Therefore, the DNN reduction device 100B can generate a reduced DNN model 40 that can achieve optimal calculation speed and recognition accuracy for each CPU of the automatic driving ECU 300.

Then, the parameter setting unit (parameter setting unit 132), based on the comparison result between the confirmed calculation speed and the target calculation speed, and the comparison result between the confirmed recognition accuracy and the target recognition accuracy, The degree of contribution of the degree of importance and the degree of contribution of the number of operations of each layer, 50, are determined. In this way, the parameter setting unit 132 is able to set optimal parameters for generating the reduced DNN model 40 based on the fed-back comparison results of calculation speed and recognition accuracy comparison results. .

Note that in FIG. 9, processing is performed in the order of the FPS confirmation unit 160 and the recognition accuracy confirmation unit 170, but the FPS confirmation unit 160 and the recognition accuracy confirmation unit 170 may be processed in parallel. In the case of parallel processing, either the FPS confirmation unit 160 or the recognition accuracy confirmation unit 170 is configured to make the determination in step S22 shown in FIG. be done.

Next, the operation of the DNN reduction device 100B will be explained.
FIG. 10 is a flowchart illustrating an example of processing by the DNN reduction device 100B according to the third embodiment. Note that in FIG. 10, the same numbers are given to the parts that perform the same processing as in FIG. 8, and unless otherwise specified, the description will be omitted as they have the same or similar functions.

The processing from steps S1, S11 to S13, and S2 to S4 is the same as the processing performed by the DNN reduction device 100A according to the second embodiment.
After the learned DNN model 10 is reduced and retrained in step S4, the FPS confirmation unit 160 (an example of a computation speed confirmation unit) uses the reduced DNN model 40 that has been retrained by the DNN relearning unit 150. Measure the calculation speed (FPS) of Further, the recognition accuracy confirmation unit 170 measures recognition accuracy from the DNN model retrained by the DNN relearning unit 150 (S21).

Next, the FPS confirmation unit 160 compares the measured calculation speed with a preset target calculation speed, and determines whether the measured calculation speed has achieved the target calculation speed (target FPS). do. The recognition accuracy confirmation unit 170 also compares the measured recognition accuracy with a preset target recognition accuracy, and determines whether the measured recognition accuracy has achieved the target recognition accuracy (S22). .

If the calculation speed measured by the FPS confirmation unit 160 does not achieve the target calculation speed (NO in S22), the FPS confirmation unit 160 feeds back the calculation speed comparison result to the parameter setting unit 132. The calculation speed comparison results include, for example, a value of 70% of the target calculation speed. Further, if the recognition accuracy measured by the recognition accuracy confirmation unit 170 does not achieve the target recognition accuracy (NO in S22), the recognition accuracy confirmation unit 170 feeds back the recognition accuracy comparison result to the parameter setting unit 132. The recognition accuracy comparison results include a value of 80% of the target recognition accuracy.

Here, the recognition accuracy measured by the recognition accuracy confirmation unit 170 is, for example, how much the reduced DNN model 40 answered correctly when a sample image for which the correct answer is known is input to the reduced DNN model 40. It is calculated by For example, if correct answers are obtained for 80 sample images as a result of inputting 100 sample images to the reduced DNN model 40, the recognition accuracy confirmation unit 170 determines the recognition accuracy of the reduced DNN model 40. is measured to be 80%. Note that the target accuracy is set to the recognition accuracy required by the user. For example, if the target accuracy is 85%, the recognition accuracy of the reduced DNN model 40 is insufficient, so the learned DNN model 10 is re-reduced in the processing after the parameter setting unit 132, and the reduced DNN model 40 is reduced. It is then necessary to retrain the DNN model 40. On the other hand, if the target accuracy is 75%, the recognition accuracy of the reduced DNN model 40 is sufficient, so it is possible to re-reduce the trained DNN model 10 and re-learn the reduced DNN model 40. No longer needed.

In step S22, at least one of the calculation speed measured by the FPS confirmation unit 160 has not achieved the target calculation speed, and the recognition accuracy measured by the recognition accuracy confirmation unit 170 has not achieved the target recognition accuracy. If it is a result (NO in S22), the comparison result is fed back to the parameter setting unit 132. The parameter setting unit 132 modifies the contribution degree parameters of the degree of importance and the number of calculations based on the comparison results fed back from the FPS confirmation unit 160 and the recognition accuracy confirmation unit 170 (S23).

In step S23, if the calculation speed has not achieved the target calculation speed, the parameter setting unit 132 sets the importance contribution parameter to increase and the calculation number contribution parameter to decrease. Furthermore, when the recognition accuracy has not achieved the target recognition accuracy, the parameter setting unit 132 sets the contribution degree parameter of the importance level to decrease and the contribution degree parameter of the number of calculations to increase. In addition, if the calculation speed has not achieved the target calculation speed and the recognition accuracy has not achieved the target recognition accuracy, the parameter setting unit 132 increases the contribution degree parameter of the degree of importance and the contribution of the number of calculations. Set the degree parameter to increase.

As an example, assume that the calculation speed and recognition accuracy of a certain reduced DNN model 40 are measured. At this time, it is assumed that the measured calculation speed = 200 fps and the measured recognition accuracy = 60 pt (%). Further, it is assumed that the target calculation speed is 220 fps and the target recognition accuracy is 50 pt (%). At this time, the FPS confirmation unit 160 compares the measured calculation speed and the target calculation speed, and determines that the measured calculation speed has not reached the target calculation speed. The recognition accuracy confirmation unit 170 also compares the measured recognition accuracy with the target recognition accuracy, and determines that the measured recognition accuracy satisfies the target recognition accuracy. Therefore, since the calculation speed has not reached the target, the parameter setting unit 132 increases the contribution parameter of the degree of importance from 0.5 to 0.6, and increases the contribution parameter of the number of calculations from 0.5 to 0.4. Modify the parameters to reduce them. Note that there is no restriction that the contribution of the degree of importance and the contribution of the number of operations can be added to 1.0.

Thereafter, the layer reduction rate calculation unit 131 again calculates the reduction rate of each layer of the DNN model, and the DNN reduction unit 140 reduces the DNN model based on the calculated reduction rate of each layer. Further, the DNN relearning unit 150 performs relearning of the DNN model. Then, the calculation speed and recognition accuracy in steps S21 and S22 are measured again, and the measured calculation speed and recognition accuracy are compared with the target calculation speed and target recognition accuracy.

In step S23, if the measured calculation speed has achieved the target calculation speed and the measured recognition accuracy has achieved the target recognition accuracy (YES in S23), the DNN relearning unit 150 The rear DNN model 40 is output to the automatic driving ECU 300 of the vehicle 500 (S5). After the process of step S5, the process of the DNN reduction device 100B ends.

In the automatic driving system 1B according to the third embodiment described above, when the measured calculation speed satisfies the target calculation speed and the measured recognition accuracy satisfies the target recognition accuracy, the reduced DNN model 40 automatically It is output to the driving ECU 300. Therefore, the automatic driving ECU 300 can recognize the external world with sufficient calculation speed and recognition accuracy using the reduced DNN model 40, and no processing is wasted.

[Fourth embodiment]
In the DNN reduction devices according to the first to third embodiments, reduction of the trained DNN model 10 is not performed in consideration of the situation where the vehicle 500 is placed, the surrounding situation of the vehicle 500, and the like. However, it has been desired to generate a reduced DNN model 40 suitable for the situation of the vehicle 500. Therefore, the DNN reduction device according to the fourth embodiment generates the reduced DNN model 40 that matches the situation of the vehicle 500.

Therefore, a configuration example and an operation example of an automatic driving system according to a fourth embodiment of the present invention will be described with reference to FIG. 11.
FIG. 11 is a block diagram showing a configuration example of an automatic driving system 1B including a DNN condensation device 100C according to the fourth embodiment. In FIG. 11, blocks that perform the same processing as those in FIG. 9 are given the same names and numbers, and unless otherwise specified, descriptions will be omitted as they have the same or similar functions.

An automatic driving system 1C according to the fourth embodiment includes a DNN reduction device 100C, an automatic driving ECU 300 of a vehicle 500, and a server 550.

For example, when the weather is clear, the image taken by the camera 200 shows a distant object in front of the vehicle 500, and the external world recognition unit 310 can recognize this object. However, when the weather is rainy, it is difficult to distinguish an object from the background even if it is a short distance from the front of the vehicle 500 in the image taken by the camera 200, and the external world recognition unit 310 may not be able to recognize this object.

Therefore, when contracting the learned DNN model 10, the DNN reduction device 100C provides information on the situation of the vehicle 500 (running, stopped, etc.) or the situation around the vehicle 500 (traffic jam, bad weather, etc.). It is possible to replace the trained DNN model 10 before reduction. This DNN reduction device 100C has a configuration in which a model reception section 180 and a driving situation observation section 190 are newly added to the DNN reduction device 100B according to the third embodiment.

The driving situation observation unit (driving situation observation unit 190) observes the driving situation of the vehicle in which the reduced multilayer neural network model (reduced DNN model 40) is used, and collects multiple multilayer neural network models (trained DNN model 40). The observation results of the driving situation are output to the server (server 550) that stores the model 10). This driving situation observation section 190 is connected to the server 550, and the external world recognition section 310 and action planning section 330 of the automatic driving ECU 300. Driving situation observation section 190 receives the action plan of vehicle 500 from action planning section 330 of automatic driving ECU 300. The driving situation observation unit 190 then observes the driving situation of the vehicle 500 based on the received action plan. Further, the driving condition observation unit 190 transmits the driving condition observation results representing the observed driving condition of the vehicle 500 to the server 550.

Further, upon receiving the reduced DNN model 40 retrained by the DNN relearning unit 150 via the FPS confirmation unit 160 and the recognition accuracy confirmation unit 170, the driving situation observation unit 190 outputs it to the external world recognition unit 310. . When the reduced DNN model 40 that reflects the driving situation of the vehicle 500 is created, the driving situation observation unit 190 outputs the reduced DNN model 40 that reflects the driving situation of the vehicle 500 to the external world recognition unit 310. You can also do it.

The server 550 is, for example, a cloud server, and is capable of transmitting and receiving various data through wireless or wired communication with the DNN reduction device 100C. The server 550 stores a plurality of DNN models in a storage device (not shown). The DNN model stored by server 550 is prepared in advance according to typical driving conditions of vehicle 500.

Then, upon receiving the driving situation observation results from the driving situation observation unit 190 included in the DNN reduction device 100C, the server 550 selects a DNN model to be transmitted to the model receiving unit 180 based on the received driving situation observation results. Thereafter, server 550 transmits the selected model to model receiving section 180.

The DNN model selected by the server 550 and transmitted to the model receiving unit 180 is one learned in advance by a learning device (not shown). Then, server 550 selects a DNN model learned for each driving situation of vehicle 500 based on the driving situation observation results received from driving situation observation section 190. For example, the range of the outside world that the outside world recognition unit 310 should recognize may be different when the vehicle 500 is running and when the vehicle 500 is stopped. Therefore, if the vehicle 500 is stopped, the server 550 selects the DNN model for the stopped state, and if the vehicle 500 is running, selects the DNN model for the running state, and transmits the selected DNN model to the model receiving unit. 180.

The model receiving unit (model receiving unit 180) receives a multilayer neural network model (trained DNN model 10) suitable for the driving situation, which is selected from the server (server 550) based on the observation results of the driving situation. The model receiving unit (model receiving unit 180) is a multilayer neural network model in which the importance calculation unit (importance calculation unit 110) reads the multilayer neural network model (trained DNN model 10) received from the server (server 550). (trained DNN model 10). Therefore, the reduced DNN model 40 is created using the learned DNN model 10 that corresponds to the driving situation of the vehicle 500.

In the automatic driving system 1C according to the fourth embodiment described above, a DNN model corresponding to the driving situation of the vehicle 500 is selected by the server 550, and is imported into the DNN reduction device 100C as the learned DNN model 10. Then, the DNN reduction device 100C performs predetermined processing on the learned DNN model 10 that has been imported, creates a reduced DNN model 40, and outputs the reduced DNN model 40 to the automatic driving ECU 300. Therefore, the automatic driving ECU 300 can accurately control the vehicle 500 using the reduced DNN model 40 created according to the driving situation of the vehicle 500.

Further, the reduced DNN model 40 generated by the DNN model selected according to the driving situation of the vehicle 500 is used by the automatic driving ECU 300. For example, the reduced DNN model 40 used while the vehicle 500 is running has a large number of calculations, whereas the reduced DNN model 40 used while the vehicle 500 is stopped has a small number of calculations. Power consumption can also be reduced.

Furthermore, the driving situation observation unit 190 may receive the GPS (Global Positioning System) information received by the vehicle 500 together with the action plan from the action planning unit 330 of the automatic driving ECU 300 to determine the current position of the vehicle 500. If the current position of the vehicle 500 is known, the driving condition observation unit 190 can receive weather information tailored to this current position, and therefore may transmit the weather information to the server 550 together with the driving condition observation results. In this case, the server 550 can select a DNN model according to the weather information.

In addition, as a modification of the automatic driving system 1C according to the fourth embodiment, the model receiving unit 180 holds a plurality of DNN models in advance, and the model receiving unit 180 receives the driving situation observation results from the driving situation observation unit 190. It may also be configured to receive the information. In such a configuration, the model receiving unit 180 may select a DNN model to be used as the trained DNN model 10 in the DNN reduction device 100C based on the driving situation observation results.

Additionally, the driving condition observation unit 190 observes the driving condition of the vehicle 500 (running or stopped) and the surrounding conditions of the vehicle 500 (presence of traffic jams, weather, etc.). Therefore, if the model reception unit 180 is configured to select the DNN model, the driving situation observation unit 190 can directly feed back the driving situation to the model reception unit 180 without going through the server 550. Good too. For example, the driving situation observation unit 190 can also observe the actual driving situation of the vehicle 500 based on information obtained from the actuator 400, sensors, automatic driving ECU 300, and the like.

Furthermore, since the driving condition observation unit 190 observes the driving condition of the vehicle 500, when the vehicle 500 breaks down, the information detected by each sensor of the vehicle 500 is sent to the automatic driving ECU 300 as the driving condition observation result. You may receive it. In this case, a DNN model matching the failure mode of vehicle 500 is selected, and DNN reduction device 100C generates reduced DNN model 40 from the selected DNN model. Therefore, the automatic driving ECU 300 can perform automatic driving processing using the reduced DNN model 40 generated according to the failure mode.

In addition, the server 500 determines the quality of the DNN model selected before the vehicle 500 breaks down after the fact, and determines by itself or asks the user to determine under what driving conditions the DNN model could have been appropriately selected. You can entrust it to others. Then, the server 500 can select a DNN model in which failure of the vehicle 500 is unlikely to occur, and transmit the DNN model to the model receiving unit 180.

Additionally, the DNN reduction device 100C may be mounted on the vehicle 500. In this case, the driving situation of the vehicle 500 is observed in real time by the driving situation observation unit 170. As a result, the server 550 selects a DNN model that matches the current driving situation of the vehicle 500, and the DNN reduction device 100C can generate the reduced DNN model 40 from this DNN model in real time. Therefore, the external world recognition unit 310 can receive the reduced DNN model 40 tailored to the current driving situation of the vehicle 500 in real time, and the external world recognition unit 310 can accurately recognize objects. As described above, if the model receiving unit 180 is configured to select the DNN model, the driving situation observing unit 190 directly feeds back the driving situation to the model receiving unit 180 without going through the server 550. However, the model receiving unit 180 may select the DNN model.

Note that the DNN reduction device according to each of the embodiments described above has been described as an example of an arithmetic device that reduces a DNN. However, the arithmetic device according to the present invention may be applied to a device that reduces a multilayer neural network other than a DNN.

Further, the present invention is not limited to the embodiments described above, and it goes without saying that various other applications and modifications can be made without departing from the gist of the present invention as set forth in the claims.
For example, in each of the embodiments described above, configurations of devices and systems are explained in detail and specifically in order to explain the present invention in an easy-to-understand manner, and the embodiments are not necessarily limited to having all the configurations described. Further, it is possible to replace a part of the configuration of the embodiment described here with the configuration of another embodiment, and furthermore, it is also possible to add the configuration of another embodiment to the configuration of a certain embodiment. Furthermore, it is also possible to add, delete, or replace some of the configurations of each embodiment with other configurations.
Further, the control lines and information lines are shown to be necessary for explanation purposes, and not all control lines and information lines are necessarily shown in the product. In reality, almost all components may be considered to be interconnected.

1... Autonomous driving system, 10... Learned DNN model, 20... Weight coefficient of each channel, 30... Target reduction rate, 40... DNN model after reduction, 50... Number of operations in each layer, 100... DNN reduction device, 110...Importance calculation section, 120...Target reduction rate setting section, 130...Reduction condition setting section, 131...Layer reduction rate calculation section, 132...Parameter setting section, 140...DNN reduction section, 150...DNN regeneration Learning section, 160...FPS confirmation section, 170...Recognition accuracy confirmation section, 180...Model reception section, 190...Driving situation observation section, 300...Automatic driving ECU, 310...External world recognition section, 320...Risk prediction section, 330...Action Planning Department, 500...Vehicle

Claims

an importance calculation unit that calculates the importance of the channel using a weighting coefficient set for the channel of a multilayer neural network model having channels each composed of a plurality of neurons in an input layer, a middle layer, and an output layer; ,
a target reduction rate setting unit that sets a target reduction rate for the entire multilayer neural network model;
a layer reduction rate calculation unit that calculates a reduction rate of each layer of the multilayer neural network model based on the importance level and the target reduction rate;
a reduction unit that reduces each layer according to the reduction rate calculated by the layer reduction rate calculation unit to generate a reduced multilayer neural network model;
An arithmetic device comprising: a relearning unit that retrains a reduced multilayer neural network model.
The importance calculation unit calculates the importance of the previous channel connected to the subsequent channel, based on the importance of the subsequent channel and the weighting coefficient of the subsequent channel, in order from the layer closest to the output layer. Calculate the importance,
The layer reduction rate calculation unit calculates a target reduction rate of each layer with respect to the target reduction rate of the entire multilayer neural network model based on the degree of importance in each layer of the degree of importance calculated for each channel. The arithmetic device according to claim 1, which calculates a ratio.
comprising a parameter setting unit that sets the degree of importance and the number of operations for each layer extracted from the multilayer neural network model as parameters;
The arithmetic device according to claim 2, wherein the layer reduction rate calculation unit calculates the target reduction rate of each layer based on the parameter and the target reduction rate of the entire multilayer neural network model. .
The arithmetic device according to claim 3, wherein the parameter setting unit sets the degree of contribution of the degree of importance calculated for each channel and the degree of contribution of the number of operations of each layer as the parameters.
a computation speed confirmation unit that verifies the computation speed from the re-trained reduced multilayer neural network model;
a recognition accuracy confirmation unit that confirms the recognition accuracy of the re-trained reduced multilayer neural network model,
The parameter setting unit determines the contribution of the importance level based on a comparison result between the confirmed calculation speed and a target calculation speed, and a comparison result between the confirmed recognition accuracy and the target recognition accuracy. The arithmetic device according to claim 4, wherein the degree of contribution of the number of operations in each layer is determined.
a driving situation observation unit that observes the driving situation of the vehicle in which the reduced multilayer neural network model is used and outputs the observation result of the driving situation to a server that stores a plurality of multilayer neural network models;
the multilayer neural network model suitable for the driving situation selected from the server based on the observation result of the driving situation, and the importance calculation unit reading the multilayer neural network model received from the server; The arithmetic device according to claim 5, further comprising: a model receiving section that uses a multilayer neural network model.
The arithmetic device according to claim 2, wherein the multilayer neural network model is a deep neural network model.
A process of calculating the importance of the channel using a weighting coefficient set for the channel of a multilayer neural network model having a channel composed of a plurality of neurons for each input layer, intermediate layer, and output layer;
a process of setting a target reduction rate for the entire multilayer neural network model;
a process of calculating a reduction rate of each layer of the multilayer neural network model based on the importance level and the target reduction rate;
A process of reducing each of the layers to achieve the reduction rate;
A calculation method that includes relearning using a reduced multilayer neural network model.