JP2021163433A - Arithmetic processing unit - Google Patents

Abstract

To provide an arithmetic processing unit configured to complete arithmetic processing using a neural network by a target calculation end time while preventing reduction in accuracy regarding a calculation result.SOLUTION: An arithmetic processing unit includes: a storage unit 22 which stores configurations of a plurality of neural networks for executing predetermined arithmetic processing; a selection unit 31 which selects one or more configuration candidates of neural network of which predetermined arithmetic processing is completed by a target calculation end time from among the configurations of the neural networks, and selects a neural network configuration for executing the arithmetic processing from among the selected candidates so that accuracy for a result of the arithmetic processing may be highest; and an arithmetic operation unit 32 which executes the arithmetic processing on predetermined data by using the selected neural network configuration.SELECTED DRAWING: Figure 3

Description

The present invention relates to an arithmetic unit that executes arithmetic processing by a so-called neural network.

In processing such as object detection, voice recognition, and optimization, a technique using a so-called deep neural network (hereinafter, simply referred to as DNN) has been proposed. When DNN is applied to these processes, the amount of calculation by DNN becomes very large, and as a result, the processing time required for the calculation process may increase. Therefore, it is determined whether or not to configure the artificial intelligence model based on the state of the vehicle, and when it is determined that the artificial intelligence model is to be configured, a plurality of arithmetic units are combined to configure an artificial intelligence model that executes a predetermined process. A technique has been proposed (see, for example, Patent Document 1).

JP-A-2018-190045

Real-time performance may be required for arithmetic processing using DNN. In such a case, it is required that the arithmetic processing using DNN is completed within a predetermined time limit. However, in the above technique, there is a possibility that the arithmetic processing using DNN cannot be completed within a predetermined time limit.

Therefore, an object of the present invention is to provide an arithmetic unit capable of completing an arithmetic process using a neural network by a target arithmetic end time while suppressing a decrease in accuracy of the arithmetic result.

According to one embodiment, an arithmetic unit is provided. This arithmetic device has a storage unit that stores the configurations of a plurality of neural networks for executing a predetermined arithmetic processing, and a neural network that completes the predetermined arithmetic processing by the target arithmetic end time from the configurations of the plurality of neural networks. A selection unit that selects one or more network configuration candidates and selects the neural network configuration that executes the arithmetic processing from the selected candidates so that the accuracy of the arithmetic processing result is maximized. It has an arithmetic unit that executes arithmetic processing on predetermined data using the configuration of the selected neural network.

The arithmetic unit according to the present invention has an effect that the arithmetic processing using the neural network can be completed by the target arithmetic end time while suppressing the decrease in the accuracy of the arithmetic result.

It is a schematic block diagram of the vehicle control system in which the arithmetic unit is mounted. It is a hardware block diagram of the electronic control device which is one Embodiment of an arithmetic unit. It is a functional block diagram of the processor of the electronic control apparatus concerning the vehicle control processing including the object detection processing which is an example of the arithmetic processing using DNN. It is a figure which shows an example of the structure of DNN used as a classifier. It is a figure which shows an example of the configuration table. It is an operation flowchart of the selection process executed by the selection part. It is an operation flowchart of the vehicle control processing including the object detection processing.

Hereinafter, the arithmetic unit will be described with reference to the drawings. This arithmetic unit executes a predetermined arithmetic process using the DNN so that the arithmetic process is completed by the target arithmetic end time. However, due to factors such as fluctuations in the arithmetic load of the processor of the arithmetic unit (for example, execution status of other arithmetic processing, occurrence of interrupt processing or exception processing, etc.), or a decrease in the clock frequency for suppressing heat generation of the processor. , The execution speed of arithmetic processing using DNN may fluctuate. Therefore, this arithmetic unit stores in advance the configurations of a plurality of DNNs for executing the predetermined arithmetic processing, and from among the configurations of the plurality of DNNs, the predetermined arithmetic processing is completed by the target arithmetic end time. Select one or more candidates for. Further, the arithmetic unit further selects the configuration of the DNN that maximizes the accuracy of the result of the arithmetic processing from the candidates of the selected configuration. Then, this arithmetic unit executes the predetermined arithmetic processing on the predetermined data by using the DNN having the configuration selected so as to maximize the accuracy. As a result, the arithmetic unit finishes the arithmetic processing using the DNN by the target arithmetic end time while suppressing the decrease in the accuracy of the arithmetic result.

In the following, an example in which the arithmetic unit is applied to the vehicle control system will be described. In this example, the arithmetic unit executes object detection processing on the image obtained by the camera mounted on the vehicle, so that various objects existing around the vehicle, for example, other vehicles, people, etc. It detects road signs or road markings, and automatically controls the vehicle based on the detection results. The object detection process is an example of a predetermined arithmetic process.

FIG. 1 is a schematic configuration diagram of a vehicle control system in which an arithmetic unit is mounted. FIG. 2 is a hardware configuration diagram of an electronic control device, which is one embodiment of the arithmetic unit. In the present embodiment, the vehicle control system 1 mounted on the vehicle 10 and controlling the vehicle 10 includes a camera 2 for photographing the surroundings of the vehicle 10, a distance measuring sensor 3, and an electronic device which is an example of an arithmetic unit. It has a control device (ECU) 4. The camera 2, the distance measuring sensor 3, and the ECU 4 are communicably connected to each other via an in-vehicle network conforming to a standard such as a controller area network. The vehicle control system 1 may further have a storage device that stores a map used for automatic driving control of the vehicle 10. Further, the vehicle control system 1 includes a receiver such as a GPS receiver for positioning the self-position of the vehicle 10 in accordance with a satellite positioning system, a wireless terminal for wireless communication with other devices, and a vehicle 10. It may have a navigation device or the like for searching a planned travel route.

The camera 2 is an example of an imaging unit, and is an image capture target on a two-dimensional detector composed of an array of photoelectric conversion elements having sensitivity to visible light such as a CCD or C-MOS, and the two-dimensional detector. It has an imaging optical system that forms an image of a region. The camera 2 is mounted, for example, in the vehicle interior of the vehicle 10 so as to face the front of the vehicle 10. Then, the camera 2 photographs the front region of the vehicle 10 at predetermined imaging cycles (for example, 1/30 second to 1/10 second), and generates an image in which the front region is captured. The image obtained by the camera 2 may be a color image or a gray image. The vehicle control system 1 may have a plurality of cameras 2 having different shooting directions or angles of view.

Each time the camera 2 generates an image, the camera 2 outputs the generated image to the ECU 4 via the in-vehicle network.

The distance measuring sensor 3 is, for example, a LiDER sensor or a radar, and measures the distance to another object existing around the vehicle 10 in each direction in each direction at a predetermined cycle. Then, the distance measuring sensor 3 outputs a sensor signal indicating the distance to another object for each direction to the ECU 4 via the in-vehicle network at predetermined intervals.

The ECU 4 controls the vehicle 10. In the present embodiment, the ECU 4 controls the vehicle 10 so as to automatically drive the vehicle 10 based on an object detected from a series of time-series images obtained by the camera 2. Therefore, the ECU 4 has a communication interface 21, a memory 22, and a processor 23.

The communication interface 21 is an example of a communication unit, and has an interface circuit for connecting the ECU 4 to the in-vehicle network. That is, the communication interface 21 is connected to the camera 2 and the distance measuring sensor 3 via the in-vehicle network. Then, each time the communication interface 21 receives an image from the camera 2, the received image is passed to the processor 23. Further, the communication interface 21 passes the sensor signal received from the distance measuring sensor 3 to the processor 23. Alternatively, the communication interface 21 passes the map read from the storage device, the positioning information from the GPS receiver, and the like received via the in-vehicle network to the processor 23.

The memory 22 is an example of a storage unit, and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. Then, the memory 22 receives from a computer program for realizing various processes executed by the processor 23 of the ECU 4, various data used in the object detection process, for example, an image received from the camera 2, and a distance measuring sensor 3. It stores sensor signals, various parameters for configuring the classifier used in object detection processing, and so on. Further, the memory 22 stores the calculation result in the middle of the object detection process. Furthermore, the memory 22 stores various parameters (for example, weighting coefficient, activation function, etc.) for configuring the DNN used in the object detection process, and the basic configuration of the DNN. Furthermore, the memory 22 is a configuration table representing a plurality of DNN structural units, a predicted value of the degree of contribution to the accuracy of the calculation required time and the calculation result of each structural unit (hereinafter, referred to as a calculation accuracy contribution), and a configuration table. , Stores the time required for operations other than operations by individual structural units (for example, non-maximum suppression). In the present embodiment, the arithmetic processing executed by the DNN is the object detection processing, so the accuracy of the arithmetic result is an accuracy indicating the certainty of the object detection result.

The processor 23 is an example of a control unit, and includes one or a plurality of CPUs (Central Processing Units) and peripheral circuits thereof. The processor 23 may further include other arithmetic circuits such as a logical operation unit, a numerical operation unit, or a graphic processing unit. The processor 23 may further have a dedicated arithmetic circuit for executing a specific arithmetic (for example, a convolution arithmetic) included in the arithmetic processing by the DNN. Then, each time the processor 23 receives an image from the camera 2 while the vehicle 10 is traveling, the processor 23 executes a vehicle control process including an object detection process on the received image. Then, the processor 23 controls the vehicle 10 so as to automatically drive the vehicle 10 based on the detected object around the vehicle 10.

FIG. 3 is a functional block diagram of the processor 23 of the ECU 4 regarding vehicle control processing including object detection processing. The processor 23 includes a selection unit 31, an object detection unit 32, an operation planning unit 33, and a vehicle control unit 34. Each of these parts of the processor 23 is, for example, a functional module realized by a computer program running on the processor 23. Alternatively, each of these parts included in the processor 23 may be a dedicated arithmetic circuit provided in the processor 23. Further, among these parts of the processor 23, the selection unit 31 and the object detection unit 32 execute the object detection process, which is an example of the arithmetic processing using the DNN.

FIG. 4 is a diagram showing an example of the configuration of a DNN used as a classifier in the object detection process. The DNN 400 is, for example, a neural network having a convolutional neural network type architecture, and has a plurality of blocks 401-1 to 401-n (n is an integer of 2 or more) in order from the input side. Each block processes, for example, data having the same resolution. For example, the closer the block is to the input side, the higher the resolution of the data is processed. In the present embodiment, the DNN 400 is used to detect an object represented in the image generated by the camera 2, so that each block is an area in which the object is represented by the resolution of the data input to the block. Is detected, and a process for identifying the type of the detected object is executed.

Each block is defined by a plurality of configurations that are selectable units. For example, block 401-1 is defined with configurations 411-1 to 411-k (k is an integer of 2 or more), and block 401-2 is configured with configurations 421-1 to 421-i (i is an integer of 2 or more). ) Is specified, and configurations 413-1 to 413-j (j is an integer of 2 or more) are specified in the block 401-n. The unit of each configuration is, for example, the block itself, one or more branches contained in the block that execute processing in parallel with the input data, or one or more layers contained in the block. Can be. Each layer performs a predetermined operation on the data (for example, feature map) input to the layer. Types of layers include, for example, convolution layers, pooling layers, fully connected layers (affine layers), activation layers and output layers. The convolution layer is a layer that executes a convolution operation on data input by a kernel of a predetermined size. The pooling layer is a layer that executes a pooling process such as Max pooling. The fully connected layer is a layer that performs a fully connected operation on the input data. The activation layer is a layer that executes an activation operation using an activation function such as a ReLU or a sigmoid function. The output layer is a layer for which the output value is obtained by using the softmax function or the sigmoid function.

For example, if the block itself or a branch is a selectable unit of neural network configuration, the number of layers or the types of layers contained in each configuration unit may be different from each other. Further, when any layer in the block is a unit of the neural network configuration that can be selected, a parameter (for example, of a convolution layer) that defines the operation of the layer that is the unit of the configuration for each unit of the configuration. Kernel size, stride, etc.) may be different. Further, a plurality of DNN400 overall configurations themselves may be prepared in advance. In this case, the overall configuration of the DNN 400 itself may be a selectable unit of configuration. Furthermore, parameters related to the amount of calculation by DNN may also be a selectable unit of configuration. Such parameters include, for example, the size of the data input to each layer or block (eg, the degree of reduction relative to the feature map output from the previous layer), a type that represents the individual values contained in the input data. And the quantization bit width (floating point type 32 bits, integer type 8 bits, etc.), or the Pruning amount, which is a quantity indicating the degree of the weighting coefficient to be replaced with 0, is included. Furthermore, a combination of any two or more of the above-mentioned configuration units may be a selectable neural network configuration unit.

The selection unit 31 selects one or more candidates for the configuration of the neural network whose object detection processing is completed by the target calculation end time from the plurality of DNN configurations, and determines the accuracy of the calculation processing result, in the present embodiment. , The configuration of the DNN that executes the object detection process is selected from the selected candidates so that the object detection accuracy is maximized. The configuration of each DNN is defined by combining the units of the selectable configuration.

The selection unit 31 causes the object detection unit 32 to start the object detection process for the image each time the processor 23 acquires an image from the camera 2. At that time, the selection unit 31 instructs the object detection unit 32 to use the DNN having the basic configuration set in advance.

After that, the selection unit 31 sets the calculation order of each process operating on the processor 23, the scheduled start time of each process, and the like for each predetermined cycle (for example, an image acquisition cycle or a cycle shorter than the acquisition cycle). Inquires of the scheduler that manages the scheduled start time of the next process (process executed by the operation planning unit 33 in the present embodiment) using the result of the object detection process. Then, the selection unit 31 sets the target calculation end time based on the scheduled start time. For example, the selection unit 31 sets the target calculation end time to be the scheduled start time of the next process itself or a time prior to the scheduled start time of the next process by a predetermined offset time. If it is specified in advance that the object detection process is terminated every predetermined cycle, the selection unit 31 may set the target calculation end time according to the cycle.

In addition, the selection unit 31 elapses from the scheduled required time of the object detection process according to the configuration of the DNN used in the currently executing object detection process to the current time from the start time of the currently executing object detection process. The difference from the time is calculated as an estimated value of the time required until the remaining calculation of the object detection process is completed (hereinafter, referred to as an estimated remaining required time). The selection unit 31 sets the estimated required time as the sum of the estimated required time of each structural unit included in the currently used DNN and the estimated required time of other operations stored in the memory 22, for example. Can be calculated. At that time, the selection unit 31 may correct the scheduled required time according to the current clock frequency of the processor 23 or the occupancy rate of the processor 23 available for DNN calculation. Then, the selection unit 31 compares the estimated remaining required time with the remaining time until the target calculation end time, and if the estimated remaining required time is longer than the remaining time, refers to the configuration table and refers to the object by the target calculation end time. The configuration of the DNN used for the remaining operations of the object detection process is selected so that the detection process ends. Further, the selection unit 31 refers to the configuration table and performs the remaining operations of the object detection process so that the object detection accuracy is the highest among the DNN configurations in which the object detection process is completed by the target calculation end time. Further select the configuration of the DNN used for.

FIG. 5 is a diagram showing an example of the configuration table. The configuration table 500 shows a plurality of configurations 501, which are selectable units, for each block, and for each configuration 501, an estimated required time 502 and a calculation accuracy contribution 503 required for the calculation of the configuration 501. The block m (m = 1,2, ..., N) shown in the configuration table 500 corresponds to the block 401-m of the DNN 400 shown in FIG. Each configuration 501 is, for example, the block itself, a branch, any layer or parameter, or a combination thereof, as described above. The estimated required time 502 and the calculation accuracy contribution 503 of each configuration 501 are, for example, actually measured in advance under predetermined conditions, or estimated in advance from the design information of the configuration. For example, the estimated required time 502 of each configuration 501 can be a measured value when the processor 23 is operated at the reference clock frequency and the occupancy rate of the processor 23 for the DNN calculation is set as the maximum value. The unit representing the estimated required time 502 may be, for example, either the number of clocks or the time. Further, the calculation accuracy contribution 503 may be an index showing the contribution to the relative improvement of the calculation accuracy between the configurations 501, and may not be a value representing the actual calculation accuracy itself. In the example shown in FIG. 5, the higher the value of the calculation accuracy contribution 503, the higher the contribution of the corresponding configuration 501 to the improvement of the calculation accuracy.

The selection unit 31 refers to the configuration table and selects, among the combinations of the units of the configuration applicable to the remaining operations by the DNN, the combination in which the object detection process is completed by the target calculation end time, as a candidate DNN. Select as configuration. For example, when the operations up to block 401-1 have been completed at this point, the selection unit 31 selects a candidate DNN configuration by combining the units of the configuration of each block after block 401-2. At that time, the selection unit 31 calculates the total estimated required time (hereinafter referred to as the estimated total required time) for each of the possible combinations of the units of the configuration, and selects the combination in which the estimated total required time is equal to or less than the remaining time. Select as a candidate configuration. Alternatively, the selection unit 31 may select a combination in which the estimated total required time is less than or equal to the remaining time as a candidate configuration according to a dynamic programming method or another optimization method. If the parameters related to the amount of calculation by DNN (input feature map size, quantization width, pruning amount, etc.) can also be selected separately from the units of the above individual configurations, the selection unit 31 selects them. The sum of the values obtained by multiplying the estimated time required for the unit of each selected configuration by the shortening rate of the calculation according to the parameter may be calculated as the estimated total time required. In this case, for each selectable parameter, the shortening rate of the calculation according to the parameter may be stored in the memory 22 in advance.

Further, the selection unit 31 selects the ratio of the reference clock frequency to the current clock frequency of the processor 23 or the ratio of the maximum value of the occupancy rate to the current occupancy rate of the processor 23 available for DNN calculation. The sum of the values obtained by multiplying the estimated required time of the configuration of the above may be calculated as the estimated total required time.

The selection unit 31 refers to the configuration table for each of the selected candidate DNN configurations (that is, combinations of configuration units), and refers to the calculation accuracy contribution of the units of the individual configurations included in the combination. The total is calculated, and the combination that maximizes the total is specified as the combination that maximizes the object detection accuracy. Alternatively, the selection unit 31 refers to the configuration table for each combination of the selected configurations, calculates the product of the calculation accuracy contributions of the units of the individual configurations included in the combination, and the product becomes the maximum. The combination may be specified as the combination that maximizes the object detection accuracy. Then, the selection unit 31 notifies the object detection unit 32 as a configuration of the DNN using the selected combination.

When the estimated remaining required time is less than or equal to the remaining time, the selection unit 31 does not have to change the configuration of the DNN. Alternatively, when the estimated remaining required time is shorter than the remaining time, the selection unit 31 performs the same processing as described above, the object detection process is completed by the target calculation end time, and the object detection accuracy is currently high. The configuration of the DNN that executes the remaining operations may be changed so as to be higher than the configuration of the DNN of the above, and the changed configuration of the DNN may be notified to the object detection unit 32.

Further, the selection unit 31 may change the estimated required time of each configuration according to the past calculation history. For example, each time the object detection process is completed, the object detection unit 32 stores in the memory 22 the calculation time required for each unit of the configuration included in the DNN used in the object detection process. Then, the selection unit 31 may use the average value or the median value of the operation required time of the most recent predetermined number of times as the estimated required time of the unit of the configuration for each configuration unit.

Alternatively, the selection unit 31 may change the estimated required time and the calculation accuracy contribution of each configuration according to the data input to the DNN. In the present embodiment, since the data input to the DNN is an image, for example, for each type of scene represented in the image, a configuration table representing the estimated required time of each configuration and the contribution of calculation accuracy is stored in the memory 22 in advance. It will be remembered. Then, the selection unit 31 may determine the type of the scene represented in the image, read the configuration table corresponding to the type from the memory 22, refer to the read configuration table, and execute the above processing. For example, the selection unit 31 calculates a statistical representative value (average value, variance, etc.) of the pixel value in the entire image or a predetermined area of the image, and the statistical representative value and a reference defined for each scene type. The representative value is compared, and the type corresponding to the reference representative value closest to the statistical representative value is determined to be the type of the scene shown in the image. Further, the selection unit 31 determines the weather around the vehicle 10 based on the operation information of the wiper, the weather forecast at the current position of the vehicle 10 received via the wireless terminal, and the like, and determines the weather around the vehicle 10. The corresponding type may be determined as the type shown in the image. Alternatively, the selection unit 31 may determine the type of the scene according to another method for determining the type of the scene shown in the image.

Further, the selection unit 31 receives the latest image from the camera 2, that is, before the object detection unit 32 starts the DNN calculation for the latest image, or the DNN for the previous image. Even during the execution of the calculation, the same processing as the above processing may be performed to determine the configuration of the DNN to be used for the latest image. Then, the selection unit 31 may instruct the object detection unit 32 to use the determined DNN configuration.

FIG. 6 is an operation flowchart of the selection process of the DNN configuration by the selection unit 31 of the processor 23. The selection unit 31 may execute the selection process at predetermined intervals according to the operation flowchart shown below.

The selection unit 31 of the processor 23 inquires the scheduler of the scheduled start time of the next process using the detection result of the object detection process, and sets the target calculation end time according to the scheduled start time (step S101). The selection unit 31 determines whether or not the estimated remaining time required until the DNN calculation is completed is longer than the remaining time until the target calculation end time (step S102). When the estimated remaining required time is longer than the remaining time (step S102-Yes), the selection unit 31 refers to the configuration table and the estimated total required time of the remaining operations by the DNN is equal to or less than the remaining time until the target operation end time. One or more candidates for the DNN configuration to be used are selected (step S103). Then, the selection unit 31 refers to the configuration table and selects the candidate having the highest object detection accuracy among the selected DNN configuration candidates as the DNN configuration for executing the remaining operations (step S104).

On the other hand, in step S102, when the estimated remaining required time is less than or equal to the remaining time (step S102-No), the selection unit 31 selects the currently used DNN configuration as the configuration to be used for the remaining operations (step S102-No). Step S105).

After step S104 or S105, the selection unit 31 instructs the object detection unit 32 to perform the remaining operations using the DNN of the selected configuration (step S106). Then, the selection unit 31 ends the selection process. The selection unit 31 may execute the processes of steps S103 and S104 as described above even when the estimated remaining required time is shorter than the remaining time in step S102.

The object detection unit 32 is an example of a calculation unit, and each time an image is obtained from the camera 2, the object detection unit 32 inputs the image to the DNN used as a discriminator, so that the surroundings of the vehicle 10 represented in the image are displayed. Detects the object to be detected that exists in. The object to be detected includes, for example, a moving object such as a car or a person. Further, the object to be detected may further include a road marking or a road sign such as a lane marking line, and a stationary object such as a traffic light.

In the present embodiment, the object detection unit 32 uses a DNN having a configuration instructed by the selection unit 31 as a discriminator. Therefore, the object detection unit 32 reads the parameters for configuring the DNN having the configuration instructed by the selection unit 31 from the memory 22, configures the DNN, and inputs an image to the configured DNN. Further, when the object detection unit 32 is instructed by the selection unit 31 to change the configuration of the DNN during the calculation by the DNN, the object detection unit 32 stores the parameters for configuring the DNN having the changed configuration in the memory 22. The DNN configuration is changed to the instructed configuration by reading from, and the remaining operations are executed by the DNN having the changed configuration. When data such as a feature map is passed between DNN blocks and the size of the data is reduced, the object detection unit 32, for example, sets a stride in the first layer of the block that receives the data. It may be set according to the reduction rate of the size of the data.

Further, the object detection unit 32 is based on a sensor signal obtained by another sensor available for detecting an object around the vehicle 10 other than the camera 2, such as a sensor signal from the distance measuring sensor 3. 10 Peripheral objects may be detected.

In this case, the object detection unit 32 may detect an object around the vehicle 10 according to an object detection method according to another type of sensor. Alternatively, the object detection unit 32 may input not only the image but also the sensor signal of the same region as the region represented by the image obtained by another sensor to the DNN.

The object detection unit 32 outputs information indicating the type of the detected object and the position on the image to the operation planning unit 33.

Based on the objects detected from each image, the operation planning unit 33 generates one or more planned travel routes of the vehicle 10 so that the objects existing around the vehicle 10 and the vehicle 10 do not collide with each other. The planned travel route is represented as, for example, a set of target positions of the vehicle 10 at each time from the current time to a predetermined time ahead. For example, each time the operation planning unit 33 receives an image from the camera 2, the operation planning unit 33 executes a viewpoint conversion process using information such as the mounting position of the camera 2 on the vehicle 10 to convert the received image into a bird's-eye view image. Convert. Then, the operation planning unit 33 tracks a detected object for each image by executing a tracking process using a Kalman Filter or the like on a series of bird's-eye views images, and from the trajectory obtained from the tracking result. , Estimate the predicted trajectory of each object up to a predetermined time ahead. Based on the predicted locus of each object being tracked, the operation planning unit 33 makes the predicted value of the distance between each of the objects being tracked up to a predetermined time ahead and the vehicle 10 equal to or more than a predetermined distance. In addition, the planned travel route of the vehicle 10 is generated. At that time, the operation planning unit 33, for example, has the current position of the vehicle 10 represented by the positioning information obtained from the GPS receiver (not shown) mounted on the vehicle 10 and the map information stored in the memory 22. The number of lanes in which the vehicle 10 can travel may be confirmed with reference to. Then, when there are a plurality of lanes in which the vehicle 10 can travel, the operation planning unit 33 may generate a planned travel route so as to change the lane in which the vehicle 10 travels. At that time, the operation planning unit 33 may determine the positional relationship between the lane in which the vehicle 10 is traveling or the lane to be changed and the vehicle 10 by referring to the position of the lane marking line detected from the image. .. Further, the operation planning unit 33 may set a planned travel route so that the vehicle 10 is stopped at the stop line corresponding to the traffic light when the traffic light detected from the image indicates a stop sign. ..
The operation planning unit 33 may generate a plurality of planned travel routes. In this case, the operation planning unit 33 may select the route that minimizes the total sum of the absolute values of the accelerations of the vehicle 10 from the plurality of planned travel routes.

The operation planning unit 33 notifies the vehicle control unit 34 of the generated planned travel route.

The vehicle control unit 34 controls each part of the vehicle 10 so that the vehicle 10 travels along the notified travel schedule route. For example, the vehicle control unit 34 obtains the acceleration of the vehicle 10 according to the notified planned travel route and the current vehicle speed of the vehicle 10 measured by the vehicle speed sensor (not shown), and accelerates the accelerator so as to be the acceleration. Set the opening or brake amount. Then, the vehicle control unit 34 obtains the fuel injection amount according to the set accelerator opening degree, and outputs a control signal corresponding to the fuel injection amount to the fuel injection device of the engine of the vehicle 10. Alternatively, the vehicle control unit 34 outputs a control signal according to the set brake amount to the brake of the vehicle 10.

Further, when the vehicle 10 changes the course of the vehicle 10 in order to travel along the planned travel route, the vehicle control unit 34 obtains the steering angle of the vehicle 10 according to the planned travel route, and responds to the steering angle. The control signal is output to an actuator (not shown) that controls the steering wheels of the vehicle 10.

FIG. 7 is an operation flowchart of a vehicle control process including an object detection process, which is an example of an arithmetic process using a DNN, executed by the processor 23. For example, each time the processor 23 receives an image from the camera 2, the processor 23 executes the vehicle control process according to the operation flowchart shown in FIG. In the operation flowchart shown below, the processes of steps S201 to S202 correspond to the object detection process.

The selection unit 31 of the processor 23 selects the configuration of the DNN used in the object detection unit 32 according to the flowchart shown in FIG. 6 (step S201). Then, the object detection unit 32 of the processor 23 inputs the image obtained from the camera 2 to the DNN having the selected configuration, and detects the object around the vehicle 10 represented by the image (step S202).

The operation planning unit 33 of the processor 23 tracks the detected object, and generates a planned travel route of the vehicle 10 so as to be equal to or more than a predetermined distance from the predicted locus of the object estimated based on the tracking result. (Step S203). Then, the vehicle control unit 34 of the processor 23 controls the vehicle 10 so that the vehicle 10 travels along the planned travel route (step S204). Then, the processor 23 ends the vehicle control process.

As described above, this arithmetic unit stores in advance the configurations of a plurality of DNNs for executing a predetermined arithmetic process, and from the plurality of DNN configurations, a predetermined one is determined by the target calculation end time. Select one or more configuration candidates for which the arithmetic processing ends. This arithmetic unit further selects the configuration of the DNN that maximizes the accuracy of the result of the arithmetic processing from the candidates of the selected configuration. Then, this arithmetic unit executes the predetermined arithmetic processing on the predetermined data by using the DNN having the configuration selected so as to maximize the accuracy. As a result, this arithmetic unit finishes the arithmetic processing using the DNN by the target arithmetic end time while suppressing the decrease in the accuracy of the arithmetic result regardless of the state of the hardware that executes the DNN arithmetic. be able to. Therefore, it is not necessary to prepare an excessive margin of hardware resources for executing the DNN operation, so that the arithmetic unit can reduce the required hardware resources.

The arithmetic unit according to the above embodiment or modification may be applied to other than the vehicle control system. For example, the arithmetic unit according to the above embodiment or modification is applied to a monitoring device installed to monitor a predetermined area outdoors or indoors, and an object to be detected is detected from an image showing the predetermined area. It may be used to detect. Further, the arithmetic unit according to the above embodiment or modification may be used for other than object detection. For example, the DNN used by the arithmetic unit may be a DNN for semantic segmentation. Alternatively, the DNN used by the arithmetic unit may be used by the operation planning unit 33 in the above embodiment to generate a planned travel route.

Further, the computer program that realizes the functions of each part of the processor 23 of the arithmetic unit according to the above embodiment or modification is recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. May be provided in the form provided.

As described above, those skilled in the art can make various changes within the scope of the present invention according to the embodiment.

1 Vehicle control system 2 Camera 3 Distance measurement sensor 4 Electronic control device (arithmetic logic unit)
21 Communication interface 22 Memory 23 Processor 31 Selection unit 32 Object detection unit (Calculation unit)
33 Operation Planning Department 34 Vehicle Control Department

Claims (1)

A storage unit that stores the configurations of a plurality of neural networks for executing a predetermined arithmetic process,
From the configurations of the plurality of neural networks, one or more candidates for the configuration of the neural network whose predetermined arithmetic processing is completed by the target calculation end time are selected, and the accuracy of the result of the predetermined arithmetic processing is maximized. A selection unit that selects the configuration of the neural network that executes the predetermined arithmetic processing from the candidates, and
An arithmetic unit that executes the predetermined arithmetic processing on predetermined data using the selected neural network configuration, and
Arithmetic logic unit.

Images