JP7449193B2 - Computing device and vehicle control device - Google Patents

Description

The present invention relates to an arithmetic device. Among these, it particularly relates to the arrangement of arithmetic units constituting an arithmetic device.

An example of an application of an arithmetic device realized by a plurality of arithmetic units is a neural network such as a deep neural network. Furthermore, it is known that a convolutional neural network (CNN), which is one of these deep neural networks, is applied to image recognition processing, which is a type of information processing. It is known that such image processing dramatically improves recognition accuracy compared to conventional rule-based algorithms, and is being put into practical use in various fields.

For example, in the automotive industry, CNN could be applied to driving assist and peripheral recognition for autonomous driving, contributing to the prevention of serious accidents. As a requirement for a device that processes CNN, it is required to process images sent from a camera at a high speed with a period on the order of tens of milliseconds. Therefore, a device equipped with an expensive GPU was required. However, recent advances in lightweight algorithms and implementation technology for compressing models have made it possible to select low-cost devices, and ECUs equipped with CNNs are being put into practical use in popular cars.

On the other hand, in-vehicle components must meet high safety and reliability standards stipulated by ISO26262 and other standards. That is, there are high demands on the failure rate and failure detection rate of arithmetic circuits and software for image processing.

The following techniques are known as means for detecting failures in arithmetic devices equipped with general combinational logic circuits (random logic). For example, a technique is known that uses a lockstep (duplex) method to detect whether the respective output values always match. Other known techniques include inputting a failure detection pattern (test signal) for detecting a failure in an arithmetic unit into a circuit and determining whether the output value of the circuit matches an expected value.

For example, in Patent Document 1, for the purpose of self-diagnosing the presence or absence of a failure in an image processing device, which processor is used among pipeline type processors arranged in multiple stages is memorized and tested. A configuration has been disclosed that allows testing to be performed in a short time without testing the entire surface by testing only the arithmetic unit used to generate the pattern.

JP2017-092757A

However, when information processing is performed using multiple arithmetic units, for example, in a CNN that requires large-scale multi-parallel product-sum arithmetic units, even if conventional technology is applied, failures occur because the processing is not pipelined. There was a problem that the circuit overhead for detection and inspection time increased.

In such a case, for example, a device used for calculation, such as an arithmetic unit, becomes expensive. Another problem is that the number of parallel arithmetic units that can be implemented in a device decreases, and the information processing speed, for example, the frame rate of image processing, decreases.

The present invention has been made to solve such problems, and an object of the present invention is to provide an arithmetic device that can reduce the circuit overhead for failure detection and realize a lower cost of the device. Note that in this specification, failure includes failures, malfunctions, and the like.

An example of a computing device according to the present invention is a computing device that performs a recognition process on input data, each of which executes a computation on the input and outputs the result of the computation, thereby realizing the recognition process . It has a plurality of arithmetic units and a failure detector that is arranged in at least one of the plurality of arithmetic units and detects a failure of the arithmetic unit, and the failure detector is arranged in at least one of the plurality of arithmetic units. The computing unit is arranged in a computing unit that is determined according to an influence level indicating the degree of influence of the computing result of the computing unit on the recognition process , and the influence level is the influence that the computing result of the computing unit has on the recognition result of the recognition process. and is determined according to the recognition accuracy of the result of the recognition processing that occurs when at least one of a failure of the arithmetic unit and a bit inversion occurs .

Further, an example of a vehicle control device according to the present invention is a vehicle control device that outputs a control signal for controlling a vehicle based on image data, and each of the vehicle control devices performs an operation on an input based on the image data, so that the a plurality of arithmetic units that realize the generation of control signals; and a failure detector that is disposed in at least one of the plurality of arithmetic units and detects a failure of the arithmetic unit; Among the computing units , the fault detector is arranged in the computing unit that causes the emergency control of the vehicle by the control signal, and when the fault detector detects a failure in the computing unit to which the fault detector is arranged, This is a vehicle control device that replaces the functions of an arithmetic unit in which a failure has been detected and an arithmetic unit in which the failure detector is not installed .

According to the present invention, in fault detection in an arithmetic device using a neural network or the like, it is possible to suppress an increase in the number of fault detectors to be arranged, that is, an increase in circuit overhead.

Problems, configurations, and effects other than those described above will be made clear by the following description of the mode for carrying out the invention.

1 is a block diagram showing an example of the configuration of an arithmetic device in Example 1 of the present invention. FIG. FIG. 2 is a block diagram showing a configuration example of a computing unit (without a failure detector) in Embodiment 1 of the present invention. FIG. 2 is a block diagram showing a configuration example of a computing unit and a failure detector disposed therein in Embodiment 1 of the present invention. FIG. 2 is a block diagram showing a configuration example of an arithmetic device provided with a fault detector placement device and a fault detector placement storage unit in Embodiment 1 of the present invention. FIG. 3 is a diagram showing an example of failure detector placement information used in Example 1 of the present invention. 7 is a flowchart illustrating an example of a process for determining whether or not a failure detector is arranged in Embodiment 1 of the present invention. FIG. 2 is a block diagram showing a configuration example of an arithmetic device according to a second embodiment of the present invention. FIG. 3 is a diagram showing an example of failure detector placement plan information used in the embodiment of the present invention. FIG. 7 is a diagram showing an example of failure detector placement information used in Example 2 of the present invention. FIG. 3 is a block diagram showing an example of the configuration of an arithmetic device according to a third embodiment of the present invention. FIG. 7 is a block diagram showing a diagram showing failure detector placement information in Embodiment 3 of the present invention. FIG. 7 is a block diagram showing an example of the configuration of a computing unit in Embodiment 4 of the present invention. FIG. 7 is a block diagram showing an example in which a failure detector is arranged in a calculation unit device according to a fourth embodiment of the present invention. FIG. 7 is an enlarged view showing an example of a computing device in which a failure detector is arranged in a computing unit device according to a fourth embodiment of the present invention. FIG. 7 is an enlarged view showing an example of exchanging functions of arithmetic unit devices in Example 4 of the present invention. FIG. 7 is a diagram showing an example of failure detector placement information used in Example 4 of the present invention. FIG. 7 is a diagram illustrating layers of neurons (calculating unit 110) according to the fifth embodiment of the present invention. It is a figure which shows an example of arrangement|positioning of the failure detector in Example 5 of this invention. FIG. 3 is a diagram for explaining an application example of each embodiment of the present invention.

Embodiments of the present invention will be described below with reference to the drawings. Each example is an illustration for explaining the present invention, and is omitted and simplified as appropriate for clarity of explanation. The present invention can also be implemented in various other forms. Unless specifically limited, each component may be singular or plural.

The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

As examples of information handled in each embodiment, the expressions "table" and "information" are used, but they may also be expressed as "list", "queue", "information", and "table", respectively. Further, the identification information can be read as "identifier", "name", "ID", "number", etc.

When there are multiple components having the same or similar functions, the same reference numerals may be given different suffixes for explanation. Furthermore, if there is no need to distinguish between these multiple components, the subscripts may be omitted from the description.

In each embodiment, processing performed by executing a program may be explained. Here, a computer executes a program using a processor (eg, CPU, GPU), and performs processing determined by the program using storage resources (eg, memory), interface devices (eg, communication port), and the like. Therefore, the main body of processing performed by executing a program may be a processor. Similarly, the subject of processing performed by executing a program may be a controller, device, system, computer, or node having a processor. The main body of processing performed by executing the program may be an arithmetic unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuits include, for example, FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), and CPLD (Complex Programmable Logic D). evice) etc.

A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server includes a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. Furthermore, in the embodiments, two or more programs may be implemented as one program, or one program may be implemented as two or more programs.

Embodiments of the present invention will be described below with reference to the drawings. In the following embodiments, an example in which the present invention is applied to vehicle control, for example, an in-vehicle ECU for advanced driver assistance systems (ADAS) and autonomous driving (AD), will be described. However, the present invention is not limited to in-vehicle ECUs for ADAS and AD. In addition, it is applicable to general information processing such as image processing. Among these, it is particularly applicable to a wide range of computing devices that require safety, such as plant control.
<Example 1>
A first embodiment in which the present invention is applied to an in-vehicle ECU for ADAS and AD will be described using FIGS. 1A to 1F. FIG. 1A is a block diagram illustrating a configuration example of an arithmetic device according to the first embodiment.

As an example of information processing, the computing device 1 includes a computing unit 11 that performs image processing on image data 2 that is input data and outputs recognition results 3 that are output data. The calculation unit 11 is assumed to perform large-scale parallel calculations such as CNN, and the calculation unit 110 shown in FIG. 1B, that is, the layers in which neurons are arranged in parallel, performs processing over many layers. The input 130 of the arithmetic unit 110, for example, an input signal, may be coupled to each output 140 of the arithmetic unit 110 of the previous layer, or may be coupled to only some arbitrary outputs, and each layer The number of neurons (calculating units 110) may be arbitrary. In this way, the arithmetic unit 11 includes a plurality of arithmetic units 110. Further, the arithmetic device 1 outputs information indicating normality/abnormality 4, which is a detection result of a failure detector 120, which will be described later.

Further, as a mechanism for detecting that the arithmetic unit 110 has failed, a failure detector 120 shown in FIG. 1C, for example, is arranged. Failure detector 120 has the following configuration. That is, a test signal generation circuit 121 that generates a test signal 150 at the input part of the arithmetic unit 110, a selector 122 on the input side for switching between the input 130 to the arithmetic unit and the test signal 150, and a test signal 150 that matches an expected value. This is a comparator 123 for checking whether Then, the arithmetic unit 110 outputs an output 140. Further, the comparator 123 outputs a normal/abnormal signal 160 of the failure detector 120. This configuration is an example, and the configuration is not limited to this as long as it has a function for detecting a failure of the arithmetic unit 110. Further, "placing" a computing unit means to put the computing unit in a state where a failure of the computing unit can be detected, and includes, for example, connecting the computing unit to the computing unit. Further, in order to realize the arrangement, as described later, an arrangement using an FPGA or the like or a fixed arrangement as hardware is included.

Here, in this embodiment, the configuration is such that there are arithmetic units 110 in which the failure detector 120 is arranged and arithmetic units 110 in which the failure detector 120 is not arranged. Hereinafter, first, the reason why the presence or absence of the failure detector is mixed will be explained below. Note that in this embodiment, a failure detector 120 is arranged in at least one arithmetic unit 110.

In the case of an arithmetic unit in general random logic, if one circuit element or wiring fails in one place (1 bit), the calculation result or information processing result will deviate from the expected value, and the system using that arithmetic unit It often has a big impact. On the other hand, in an arithmetic unit 110 such as a CNN, which is composed of a circuit that performs product-accumulation operations on a large number of input signals and has rounding functions such as bit shifting and activation functions, even if a 1-bit failure occurs, the operation result cannot be changed. Changes are often so small that they can be ignored. In addition, even if there is a large change in the calculation result of one calculation unit, the results of other calculation units are also calculated in the next layer onwards, so the final calculation result is the information processing result (recognition result). In 3), the change in the expected value may be small.

In addition, the CNN calculation parameters (weights, bias, etc.) determined by training using the learning data set are assigned different values for each layer and calculation unit. Among the assigned parameters, a computing unit that is assigned a small value close to 0 will only produce a small result no matter what data is input, so a failure will have little effect on the computing results or recognition results 3. I can say it.

For the above reasons, the degree of influence that a failure has on the calculation results of the calculation unit 110 and the information processing results of the calculation device 1 differs depending on the characteristics of the algorithm itself and how calculation parameters are assigned (results of learning). This degree of influence is investigated in advance when designing the CNN, and arithmetic units that have a large influence on the calculation results are identified, and the failure detector 120 is arranged for these arithmetic units.

In order to realize this, in this embodiment, as shown in FIG. 1D, a fault detector placement device 101 and a fault detector placement storage section 102 are provided. That is, based on the fault detector arrangement information 1021 stored in the fault detector arrangement storage section 102, the fault detector locator 101 determines the arrangement of the fault detectors 120. The function of this failure detector placement device 101 is processing according to a program, but the function may be realized as hardware. Note that in each embodiment described later, the arithmetic device 1 shown in FIG. 1A or FIG. 1D may be used as a vehicle control device.

Here, FIG. 1E shows an example of the failure detector placement information 1021. The failure detector placement information 1021 includes “calculation unit” which is identification information for identifying a calculation unit, “influence degree” indicating the influence of the calculation result of the calculation unit on recognition processing, and “placement presence/absence” of the failure detector 120. has. Here, the "computing unit" may use other identification information such as a name other than a number.

Here, the degree of influence is an index that is calculated by a predetermined algorithm and indicates the degree of influence of the calculation results of the calculation unit on the information processing of the calculation device 1. Therefore, the degree of influence includes being determined according to at least one of the results of information processing described below and the control according to the results of this information processing. This enables appropriate failure detection depending on the information processing and application of the arithmetic device 1.

More specifically, the degree of influence includes the degree of deterioration (recognition accuracy), which is the degree of change in the recognition result as a result of information processing, and the degree of influence determined according to the influence of the recognition result on the control result of the vehicle. As described above, the degree of influence indicates the degree of influence on information processing by the arithmetic results of an arithmetic unit or the like.

In addition, in this embodiment, the threshold value of the degree of influence is stored as "0.5", and when this threshold is exceeded, the "presence or absence of placement" is placed, that is, the failure detector 120 is placed in the corresponding arithmetic unit 110. I judge that. Conversely, if it is less than or equal to the threshold, it is determined that the "presence or absence of placement" is null, that is, the failure detector 120 is not placed in the corresponding arithmetic unit 110. Furthermore, the failure detector placement information 1021 does not need to include the "degree of influence."

Furthermore, "placement presence/absence" may be determined by combining a plurality of requirements. For example, the determination may be made based on the degree of change in the recognition result or the product or sum of the degree of influence of the recognition result on the vehicle control result. Further, the degree of influence may be newly calculated using both the degree of change in the recognition result and the degree of influence of the recognition result on the control result of the vehicle. Furthermore, if both the degree of change in the recognition result and the degree of influence of the recognition result on the control result of the vehicle exceed thresholds, it may be determined that the arrangement is appropriate.

Note that the arrangement of the fault detectors 120 may be determined manually without using the fault detector placement device 101 and the fault detector arrangement storage unit 102. Furthermore, the failure detector placement unit 101 and the failure detector placement storage unit 102 may be provided in a separate device from the arithmetic device 1. In this case, it is desirable to provide this in the design device or manufacturing device of the arithmetic device 1 and determine whether or not the failure detector 120 is arranged during design or manufacturing. Processing in this case will be explained using the flowchart shown in FIG. 1F.

First, in step S1, the design device inputs a learning image stored in advance in response to a designer's operation, and performs learning, that is, parameter updating. Next, in step S2, the design device uses pre-stored inference images to evaluate the accuracy of the learning results, that is, the accuracy of the image processing. This evaluation calculates an index indicating the performance, that is, the accuracy, of the arithmetic device specified by a predetermined algorithm. Steps S1 and S2 are then repeated until the accuracy satisfies a certain standard or until a predetermined number of times is completed.

Next, in step S3, the design device injects an error into a location (calculating unit) whose degree of influence should be investigated based on the inference image. Then, in step S4, the design device evaluates the degree of influence of each arithmetic unit. That is, the design device calculates the degree of influence as shown in FIG. 1E, and determines whether or not a failure detector is placed in the arithmetic unit. Using this result, a failure detector is placed in the manufacturing equipment. Incidentally, steps S3 and S4 are repeated until the error location (calculating unit) whose influence degree is to be investigated is completed.

Next, an example of the arrangement of the failure detector 120 will be described below. As described above, one method for determining the degree of influence of a failure in a CNN for image processing is to evaluate the degree of change in recognition results as described above. The simplest evaluation method in this example is to calculate recognition accuracy from the inference result when image data for inference is input to a trained CNN. Recognition accuracy can be evaluated, for example, by determining how much the coordinates and size of the bounding box of a recognized car differ from the correct value. A threshold value for how much the recognition accuracy value deteriorates in the event of a failure is defined, and the failure detector 120 is placed in the arithmetic unit 110 or the arithmetic unit device constituting the arithmetic unit 110 that deteriorates more than the threshold value. This will serve as a standard for That is, in this example, the degree of deterioration indicating recognition accuracy is used as the degree of influence.

In this case, as the degree of deterioration, it is possible to use the degree of change in the result of recognition processing that occurs when at least one of a failure of the arithmetic unit 110 and bit inversion occurs.

Furthermore, it is also conceivable to use, as the degree of influence, an evaluation standard that takes into consideration not only the recognition accuracy but also the influence that recognition results due to failures have on ADAS and AD applications. Specifically, the point of view is whether a change in the bounding box of the recognition result due to a failure will lead to sudden control. That is, in images taken while driving, erroneous detection at a far distance from the host vehicle (where the vehicle is recognized as being present when it actually does not exist) or minute changes in the bounding box are considered to have no effect on control. On the other hand, if a fault occurs and a false detection occurs several meters ahead of the own vehicle, it may lead to emergency controls such as sudden braking or sudden steering. It can be said that it is desirable to arrange a failure detector 120 for a failure part (calculating unit 110) that has the latter effect. That is, in this example, whether or not a predetermined control will be executed is also used as the degree of influence. As an example, the possibility of executing a predetermined control in a plurality of hypothetical situations through simulation or the like can be used as the degree of influence. Further, the predetermined control includes emergency control including sudden steering and sudden braking when the computing device 1 outputs a control signal for the vehicle. In this case, a failure detector 120 is placed in the arithmetic unit 110 that performs calculations that cause emergency control. With this, fail-safe control can be realized in ADAS and AD.

When this embodiment is actually applied, it is important to consider how false detections and non-detections will have a large impact on control, depending on the driving location such as a highway or city area, the speed of the own vehicle, the degree of congestion of surrounding vehicles, etc. The criteria will be created based on a comprehensive assessment of the definitions.
<Effects of Example 1>
When detecting faults in arithmetic units, by placing fault detectors preferentially in areas that have a high degree of influence on recognition results, it is possible to reduce the number of fault detectors and achieve high safety. It is possible to reduce device costs while ensuring security.
<Example 2>
The first embodiment can take measures against the influence on the recognition results when a failure of the arithmetic unit 110 occurs on a fixed basis. Here, even in the arithmetic unit 110 in which the failure detector 120 is not arranged, if a number of failures occur in the arithmetic unit 110, the influence on the arithmetic result will be large, so countermeasures are required. Therefore, in the second embodiment, it is possible to detect a failure even in the arithmetic unit 110 in which the failure detector 120, which is determined to have a small influence on the failure, is not installed. For this reason, in the second embodiment, the arithmetic unit in which the failure detector 120 is arranged is dynamically changed. In other words, the failure detector 120 is dynamically relocated with respect to the arithmetic unit 110. In this way, in the second embodiment, the arrangement relationship between the failure detector 120 and the arithmetic unit 110 is dynamically changed. The details will be explained below.

First, FIG. 2A shows the configuration of the arithmetic device 1 of this embodiment. As shown in FIG. 2A, the arithmetic device 1 of this embodiment includes a calculation section 11, a fault detector reconfigurator 12, a fault detector placement scheduler 13, and a fault detector placement planning section 14. The arithmetic unit 11 includes a plurality of arithmetic units 110. Here, the functions of the fault detector reconfigurator 12 and the fault detector placement scheduler 13 are processing according to a program, but the functions may be realized as hardware.

Here, when the fault detector arrangement scheduler 13 receives the arrangement replacement trigger 6, the fault detector arrangement scheduler 13 executes the arrangement (configuration) according to the fault detector arrangement plan information stored in the fault detector arrangement planning section 14. An instruction is output to the detector reconstructor 12. For this reason, the fault detector reconfigurator 12 executes rearrangement of the fault detectors 120 according to the fault detector arrangement plan information. Although not shown in FIG. 2A, like the first embodiment, this embodiment may also include a fault detector placement device 101 and a fault detector placement storage section 102. In this case, the fault detector reconfigurator 12 and the fault detector arrangement planning section 14 may have the functions of the fault detector placement device 101 and the fault detector arrangement storage section 102, respectively. Hereinafter, in this embodiment, the function of the fault detector arrangement storage section 102 will be explained as being included in the fault detector arrangement planning section 14.

The details of this embodiment will be explained below. First, the fault detector arrangement planning section 14 stores fault detector arrangement planning information 141 shown in FIG. 2B. According to this content, the fault detector reconfigurator 12 executes rearrangement of the fault detector 120. Note that, for this reason, it is desirable that the arithmetic device 1 be a reconfigurable device such as an FPGA.

Here, this fault detector placement plan information 141 is created by the fault detector placement scheduler 13 using the fault detector placement information 1021 in FIG. 2C (same as FIG. 1E). For this purpose, the fault detector placement scheduler 13 first selects No. 1 from the fault detector placement information 1021, which is a comparative computing unit of influence, that is, a computing unit having a value below a threshold value. Identify 3-5, 7, 10, and M. Then, the fault detector placement scheduler 13 stores the identification information of these arithmetic units in the arithmetic units with a low degree of influence (lower than the threshold value) in the fault detector placement plan information 141. Here, in this embodiment, the identification information used in the failure detector arrangement information 1021 is used, but it may be given again. Then, the failure detector placement scheduler 13 replaces the arithmetic units in which the failure detectors 120 are placed at each time, and records the result in the failure detector placement information 1021. Note that in FIG. 2B, the presence or absence of placement is determined for each time, but a trigger other than time may be used. In this manner, in this embodiment, the arithmetic unit 110 in which the failure detector 120 is arranged is periodically changed.

The details of this replacement, that is, the arrangement conversion, which is the reconfiguration of the failure detector 120, will be described below. First, it is assumed that the number of arithmetic units in which no fault detector is arranged is N (N is a natural number). Here, if n is defined as the number of detected replacements, it is assumed that the failure detectors 120 are arranged for n computing units (eg, the top n pieces of the failure detector placement plan information 141). For example, when n=2, the failure detector placement scheduler 13 selects No. 1 at time=t. Record the “placement” in 3 and 4.

Here, n<N, and n is also a natural number. At this time, no failure detector is arranged in the arithmetic units other than those mentioned above (No. 5 and after in FIG. 2B).

Next, at the specified timing of the placement reshuffling trigger 6, such as when waiting for a signal or initializing when the engine is turned on, the fault detector placement scheduler 13 sends the fault detector reconfigurator 12 to the No. An instruction is given to arrange the failure detector 120 in the arithmetic units 5 and 7. At this time, the failure detector placement scheduler 13 selects No. 5, and No. other than 7. 3 and 4, no. The failure detector 120 is not arranged in the arithmetic units 10 and onwards. The fault detector reconfigurator 12 rearranges and reconfigures the fault detectors within the calculation unit according to this command.

The fault detector arrangement scheduler 13 repeats such arrangement replacement of the fault detectors, that is, arrangement conversion, every time the arrangement replacement trigger 6 is received. If you repeat this x times, No. This will cover up to M computing units. Here, the relationship n*x>N holds true (x is also a natural number). Note that the x+1 arrangement conversion (one round) returns to the initial arrangement. As a result, as shown in FIG. 2A, ``A computer with a small effect on the recognition result has no fault detector placed at the time shown in the figure'' and ``a computer with a small effect on the recognition result has a fault detector placed at the time shown in the figure.'' ``Arithmetic units, fault detectors, and placement targets that have a large effect on recognition results'' will be mixed together. In other words, the fault detector 120 is arranged for at least one computing unit, and the fault detector 120 is not arranged for at least one of the other computing units.

Furthermore, in the above example, the configuration is such that the numbers are replaced by a constant (n=2), but this number may be variable. However, the fault detector placement scheduler 13 sets an upper limit so that the total number of fault detectors 120 is not exceeded. Further, the number of times that the failure detectors 120 of "arithmetic units that have little influence on recognition results" are arranged each time the arrangement conversion goes through is the same, but it may be variable. For example, the fault detector placement scheduler 13 may increase the number of times the arithmetic unit is arranged as the influence of the fault detector placement information 1021 increases. In this case, it is desirable that the fault detector placement scheduler 13 determines the number of placements in proportion to the degree of influence.

Further, the fault detector arrangement planning information 141 indicating this arrangement may be programmed based on a counter or a mathematical formula, or may be stored in the fault detector arrangement planning unit 14 in a format such as a table as shown in FIG. 2B. You can leave it as is. Furthermore, as described above, the trigger for the location change may be waiting for a traffic signal or turning on the engine, or may use any cycle.
<Effects of Example 2>
According to this embodiment, reliability can be ensured even for an arithmetic unit that has a low influence such as sensitivity to a 1-bit failure, that is, an arithmetic unit in which the failure detector 120 is not arranged.
<Example 3>
Next, a third embodiment will be described in which the failure detector 120 is arranged according to various situations and environments. Vehicles often travel under various driving environments such as time of day and location (roads, etc.). For this reason, it is necessary to perform control appropriate to the environment in ADAS and AD for vehicles as well. At this time, the degree of influence of highly sensitive neurons, that is, the arithmetic unit 110, may differ depending on the environment. Here, if we try to arrange the fault detector 120 in a way that corresponds to each assumed environment, the overall impact value of each computing unit will be high and the sensitivity will be high, making it difficult to distinguish between them. A problem arises in that the number of arithmetic units required increases. In other words, an overhead occurs regarding the arrangement of the failure detector 120.

Therefore, in this embodiment, the degree of influence of the computing unit is calculated for each environment and stored in the fault detector arrangement storage unit 102 in FIG. 3A, and is used to determine the arrangement of the fault detectors 120. The details will be explained below, but the environment includes scenes, situations, conditions, time of day, weather, location, and the like.

First, FIG. 3A shows the configuration of the arithmetic device 1 of this embodiment. As shown in FIG. 3A, the arithmetic device 1 of this embodiment includes, in addition to the arithmetic unit 11, a fault detector reconstructor 12, an environment determiner 15, and a fault detector arrangement storage unit 102. The functions of the failure detector reconfigurator 12 and the environment determiner 15 are processed according to a program, but the functions may be implemented as hardware.

Although not shown in FIG. 3A, like the first and second embodiments, this embodiment may also include the fault detector placement device 101 and the fault detector placement planning section 14. In this case, the functions of the fault detector placement device 101 and the fault detector placement planning section 14 may be provided by the fault detector reconstructor 12 and the fault detector placement storage section 102, respectively. That is, Examples 1 to 3 may be implemented by combining their functions. Furthermore, each of these parts is as described in Example 1.
2 may be provided in an external device of the arithmetic device 1.

Here, the fault detector arrangement storage unit 102 has a storage function similar to that of the first embodiment, but differs from the first embodiment in that it stores the fault detector arrangement information 1021 for each environment. In other words, the fault detector arrangement storage unit 102 of this embodiment includes fault detector arrangement information 1021-1 for environment 1 (expressway), fault detector arrangement information 1021-2 for environment 2 (urban area), etc. shown in FIG. 3B. It stores information on each fault detector arrangement. Although not shown, the environment of the failure detector placement information 1021 may include time zones such as nighttime and daytime, weather such as sunny and rainy weather, and location and region such as urban and suburban areas.

The processing of this embodiment will be explained below. First, the environment determiner 15 inputs conditions 5 regarding image shooting and driving. Here, the environment determiner 15 may determine the environment based on the input condition 5, or may use a result determined by an external arithmetic device as the condition 5 as the environment.

Next, the environment determiner 15 outputs the environment, which is the above determination result, to the failure detector reconstructor 12. Then, the fault detector reconfigurator 12 specifies the fault detector placement information 1021 according to the received environment. For example, if the environment is "expressway", the fault detector reconstructor 12 specifies the fault detector placement information 1021-1 of the fault detector of environment 1. Then, the fault detector reconstructor 12 uses this fault detector arrangement information 1021-1 to arrange the fault detectors 120.

Further, when the environment determiner 15 detects that the environment has changed according to the input condition 5, it outputs a notification to that effect to the failure detector reconfigurator 12. Then, the fault detector reconfigurator 12 refers to the fault detector arrangement information 1021 in the fault detector arrangement storage section 102 and rearranges the fault detectors 120 in the calculation section 11 . For this reason, it is desirable that the arithmetic device 1 be a reconfigurable device such as an FPGA. Note that the environment determiner 15 may output changes in the environment when the car is stopped, such as waiting at a traffic light, or when the car is running at a low speed (a situation where safety is ensured). Further, the fault detector reconfigurator 12 may perform the relocation in the above-mentioned state.
<Effects of Example 3>
Since the failure detector 120 can be arranged according to various driving environments, it is possible to control the vehicle according to the driving environment.
<Example 4>
In Examples 1 to 3, the fault detector 120 was placed in each neuron, that is, in each calculation unit 110, but in this Example 4, the failure detector 120 is further placed in each calculation unit device. Decide what to do. This calculation unit device includes an integrator 111 and the like as described later. In CNN, as an example of the calculation configuration of one neuron (computation unit 110), it is known that the configuration shown in FIG. 4A is adopted. In other words, each computing unit 110 constituting the computing unit 11 performs operations such as an integrator 111, a cumulative adder 112, a bit shift (rounding) 113, a bias coefficient adder 114, and an activation function computing unit (ReLU computing unit) 115. Has unit equipment. Here, in such a configuration, the values of weights and bias coefficients are generally different for each neuron. Therefore, in addition to calculating the degree of influence for each neuron (calculating unit 110) based on the values of the weights and bias coefficients, it is also possible to calculate the degree of influence for each calculation unit device within the neuron. In this embodiment, circuit overhead can be further suppressed by calculating the degree of influence for each calculation unit device and arranging the failure detector 120 in the arithmetic unit 110 with a high degree of influence, for example, as shown in FIG. 4B. The processing for this will be explained below.

The configuration of this example can be realized by any one of Examples 1 to 3 or a combination thereof. Here, the configuration of Example 1, that is, the configuration of FIG. 1D will be described as an example. However, the fault detector arrangement information 1021 uses the fault detector arrangement information 1021-3 shown in FIG. 4E. Here, the failure detector arrangement information 1021-3 shown in FIG. 4E stores the degree of influence for each calculation unit of each calculation unit constituting the calculation device 1. Then, through the process described above, the fault detector locator 101 determines the arrangement of the fault detectors 120 based on the fault detector arrangement information 1021-3 stored in the fault detector arrangement storage section 102.

This concludes the description of the process for determining the placement of the failure detector 120 of this embodiment. Next, a description will be given of what to do when a failure in a calculation unit device is detected.

First, as shown in FIG. 4B, it is assumed that the failure detector 120 for the calculation unit device is placed based on the failure detector placement information 1021-3. Among these, the upper left arithmetic unit 110-1 (No. 1) has a failure detector 120 arranged thereon because the degree of influence of integrator 1 and integrator F is greater than the threshold as shown in FIG. 4E. There is. An enlarged view of this arithmetic unit 110-1 is shown in FIG. 4C.

Here, the failure detector locator 101 is connected to the integrator No. 1 is notified from the failure detector 120 that the integrator 1 has failed. Next, the fault detector locator 101 uses the fault detector arrangement information 1021-3 to identify an integrator in which a fault detector 120 is not arranged. Then, the failure detector placement unit 101 identifies an integrator that meets a predetermined condition, such as an integrator that has the least influence. This is because it can be determined that the smaller the degree of influence is, the smaller the influence will be even if the integrator 1, which is a failed arithmetic unit device, is replaced.

As a result, in this example, it is assumed that the failure detector locator 101 has identified the integrator 2. Then, the fault detector placement unit 101 outputs an instruction to the arithmetic unit 110 to exchange the functions of the integrator 1 and the integrator 2, and to arrange the fault detector 120 to the integrator 2. As a result, the arithmetic unit 110-1 is configured as shown in FIG. 4D. For this reason, also in this embodiment, it is desirable that the arithmetic device 1 be a reconfigurable device such as an FPGA.

Note that the failure detector placement device 101 first exchanges the functions of the integrator 1 and the integrator 2 every time a failure is detected, and executes an alternative process. Then, the failure detector placement device 101 outputs an instruction to place the failure detector 120 when this alternative processing is performed a certain number of times or more. Further, it is preferable that the failure detector placement device 101 outputs warning information to an external device such as an in-vehicle terminal when alternative processing is performed a certain number of times or more.

Further, it is preferable that the fault detector placement unit 101 selects the calculation unit device whose function is to be replaced from among devices that satisfy a predetermined condition, such as adjacent devices.

Note that this process can be performed in the same manner in Examples 1 to 3. In other words, it is also possible to replace the processing of the arithmetic unit in which a failure has been detected with another arithmetic unit.
<Effects of Example 4>
By arranging fault detectors in arithmetic unit devices, it is possible to more precisely arrange failure detectors for arithmetic units. Furthermore, even if a failure is detected, it is possible to prevent the system from directly entering degenerate mode or stop mode, and the system can safely continue to operate, such as vehicle control, for a while (Example 5)
Next, a fifth embodiment will be described in which a failure detector 120 is commonly arranged for each type of processing unit device that constitutes each processing unit 110 arranged in parallel.

In the arithmetic device 1, the operations of each neuron (arithmetic unit 110) may not be implemented in the device due to circuit resource constraints. For example, a network having successive layers in which neurons are arranged as shown in FIG. 5A has the following configuration. In other words, the calculation configuration of one neuron (operation unit 110) is an integrator 111, an accumulation adder 112, a bit shift (rounding) 113 which is a rounding operator, a bias coefficient adder 114, and an activation function operator (ReLU operator). 115, etc. In this case, the combination of input data and the values of weights and bias coefficients differ for each neuron. In such an arithmetic device 1, the K arithmetic units 110 within the frame are mounted in the arithmetic unit 11, and when the calculation is completed, the weight is switched to the next K arithmetic units 11 and the calculation is performed. In such a case, the same arithmetic unit is continued to be used by changing only the parameters, so the placement of the failure detector cannot be determined based on the influence of weights and bias coefficients on the arithmetic results.

On the other hand, due to the characteristics of the algorithm, it can be defined by the type of processing unit device that has a high degree of influence on failure. For example, the cumulative adder 112 of FIG. 5B is a sensitive part because the effects of failures accumulate. Furthermore, since the activation function calculator (ReLU calculator) 115 outputs negative values as 0 and outputs other values as is, it can be said that the sensitivity of the sign bit, that is, the degree of influence, is relatively high. Therefore, in this embodiment, the failure detector 120 is arranged in advance for these calculation unit devices. Note that the configuration of this example can be realized by any one of Examples 1 to 4 or a combination thereof. Here, the configuration of Example 1, that is, the configuration of FIG. 1D will be described as an example.

For this purpose, the fault detector locator 101 uses configuration information of the arithmetic unit 110 that is stored in advance. This configuration information determines the presence or absence of placement for each calculation unit device. For this reason, the calculation unit devices to be placed are identified from the failure detector placement device 101, and it is determined that the failure detectors 120 are to be placed for these devices. Here, the fault detector placement device 101 may decide to arrange the fault detector 120 that is shared by the same type of unit of processing devices. That is, as shown in FIG. 5B, a fault detector 120 shared by a plurality of cumulative adders 112 is arranged. In this way, the arrangement of the shared fault detector 120 may also be implemented in the first to fourth embodiments.

As described above, in this embodiment, the failure detector 120 is placed in a predetermined unit of operation, such as a sign bit of the cumulative adder 112 or a Round or Bias adder (not shown).
<Effects of Example 5>
According to this embodiment, the failure detector 120 can be efficiently arranged even in an arithmetic unit device with relatively small circuit resources.
<Combination of Examples 1 to 5>
This concludes the description of Examples 1 to 5. Note that the processing in each embodiment may be executed independently, or may be realized by combining at least two of them.
<Application examples of Examples 1 to 5>
Next, an application example regarding the arrangement of the failure detector 120 in Examples 1 to 5 will be described using FIG. 6. In this example, an example will be described in which the arithmetic device 1 is applied to a vehicle control device, so-called a vehicle-mounted ECU, but Examples 1 to 5 can be similarly applied to a vehicle-mounted control device. In this application example, control results in the vehicle 3000 are learned, and these results are fed back to activities in the design and manufacturing departments. The process will be described below using the configuration of the first embodiment, that is, the configuration of FIG. 1D, as an example. However, this application example can also be realized by any one of Examples 1 to 5 or a combination thereof.

First, in FIG. 6, a vehicle 3000 includes a vehicle control device 1A-1 and a communication device 3001 that can be realized by the computing device 1. These devices are connected to each other via a communication path. Further, it is assumed that the vehicle control device 1A-1 has the configuration shown in FIG. 1D.

Here, the failure detector placement device 101 of the vehicle control device 1A-1 performs the processing of each embodiment. Further, the failure detector placement device 101 uses the processing results and control results of the vehicle control device 1A-1 to perform at least one of steps S1 to S4 in FIG. 1F in accordance with the order. Then, the failure detector placement device 101 transmits its own processing results to the server device 1000 used in the design/manufacturing department via the communication device 3001.

Here, the server device 1000 is realized by a so-called computer, and a processing device such as a CPU executes the processing according to a program. Then, the server device 1000 uses the processing results of the failure detector placement device 101 to perform the remaining processing of steps S1 to S4 in FIG. 1F, and stores the results in the file system 4000. Note that when the failure detector placement device 101 executes steps S1 to S4, the server device 1000 stores the results in the file system 4000. Note that the file system 4000 only needs to be able to store information, and may be provided inside the server device 1000 or outside. Then, the vehicle control device 1A-2 is manufactured by reflecting this result in designing and manufacturing.

Therefore, it is possible to realize a more realistic arrangement of the failure detectors 120 that reflects the results of actual control.

DESCRIPTION OF SYMBOLS 1... Arithmetic device, 11... Arithmetic unit, 12... Fault detector reconfigurator, 13... Fault detector placement scheduler, 14... Fault detector placement planning unit, 15... Environment determiner, 102... Fault detector placement storage unit , 110... Arithmetic unit, 111... Accumulator, 112... Cumulative adder, 113... Bit shift (rounding), 114... Bias coefficient adder, 115... Activation function operator (ReLU operator), 120... Fault detector , 121... Test signal generation circuit, 122... Selector, 123... Comparator, 130... Input, 140... Output, 150... Test signal, 160... Normal/abnormal signal, 170... Range of arithmetic unit that can be implemented in the device, 2 ...Image data, 3...Recognition result, 4...Normal/Abnormal

Claims (9)

In a computing device that performs recognition processing on input data,
a plurality of arithmetic units each performing an operation on an input and outputting a result of the operation to realize the recognition process ;
a failure detector disposed in at least one of the plurality of computing units to detect a failure of the computing unit;
The failure detector is placed in a computing unit among the plurality of computing units, which is determined according to a degree of influence indicating the degree of influence on the recognition processing by the computing result of the computing unit,
The degree of influence indicates the influence that the calculation result of the calculation unit has on the recognition result of the recognition process, and the recognition accuracy of the result of the recognition process that occurs when at least one of a failure of the calculation unit and a bit inversion occurs. An arithmetic device characterized by : In a calculation device that performs recognition processing on input data that is image data and outputs a control signal that controls a vehicle ,
a plurality of arithmetic units each performing an operation on an input and outputting a result of the operation to realize the recognition process;
a failure detector disposed in at least one of the plurality of computing units and detecting a failure of the computing unit;
an environment determiner that determines a photographing environment for photographing the image data;
a failure detector placement storage unit that stores presence/absence of placement of the failure detector for each of the photographing environments;
The failure detector is arranged in a computing unit among the plurality of computing units, which is determined according to a degree of influence indicating the degree of influence on the recognition processing by the computing result of the computing unit,
The degree of influence is determined by the influence that the calculation result of the arithmetic unit has on the control according to the result of the recognition process,
The arithmetic device is characterized in that the failure detector is arranged according to the photographing environment determined by the environment determination device using the failure detector arrangement storage unit. The arithmetic device according to any one of claims 1 to 2 ,
An arithmetic device characterized in that an arithmetic unit in which the failure detector is arranged is dynamically changed according to predetermined conditions. The arithmetic device according to claim 3 ,
A calculation device characterized in that a calculation unit in which the failure detector is arranged is periodically changed. The arithmetic device according to claim 3 ,
An arithmetic device characterized in that functions of an arithmetic unit in which the fault detector is arranged and an arithmetic unit in which the fault detector is not arranged are exchanged. The arithmetic device according to claim 5 ,
When the failure detector detects a failure in an arithmetic unit in which the failure detector is arranged, the functions of the arithmetic unit in which the failure has been detected and the arithmetic unit in which the failure detector is not arranged are swapped. A computing device characterized by: The arithmetic device according to claim 6 ,
The computing device is a computing device that inputs image data and outputs a control signal for controlling the vehicle, and when the number of computing units whose functions have been replaced exceeds a predetermined threshold, the computing device inputs image data and outputs a control signal to control the vehicle. An arithmetic device characterized in that it terminates control of. In a vehicle control device that outputs a control signal for controlling a vehicle based on image data,
a plurality of arithmetic units, each of which realizes generation of the control signal by executing an arithmetic operation on an input based on the image data;
a failure detector disposed in at least one of the plurality of computing units to detect a failure of the computing unit;
The failure detector is disposed in a computing unit that causes emergency control of the vehicle based on the control signal, among the plurality of computing units,
When the failure detector detects a failure in an arithmetic unit in which the failure detector is arranged, the functions of the arithmetic unit in which the failure has been detected and the arithmetic unit in which the failure detector is not arranged are exchanged. A vehicle control device characterized by : The vehicle control device according to claim 8 ,
A vehicle control device characterized in that when the number of computing units whose functions have been replaced exceeds a predetermined threshold, control over the vehicle is terminated.

Images