WO2023245823A1

WO2023245823A1 - Compute-in-memory chip and control method

Info

Publication number: WO2023245823A1
Application number: PCT/CN2022/109925
Authority: WO
Inventors: 黄柱光; 张波
Original assignee: 深圳市芯存科技有限公司
Priority date: 2022-06-22
Filing date: 2022-08-03
Publication date: 2023-12-28
Also published as: CN115071757A

Abstract

Disclosed in the present application are a compute-in-memory chip and a control method. The method comprises: acquiring environmental data around a vehicle; when a preset safety level is satisfied, a first neural network unit outputting a similarity to a plurality of environments according to the input environmental data; and according to the similarity to the plurality of environments, acquiring a network level of a hidden layer and neuron nodes of the hidden layer, a plurality of neuron nodes of an input layer, a plurality of neuron nodes of an output layer, and a closed transmission channel, and a second neural network outputting a control instruction according to the similarity to the plurality of environments.

Description

A storage and calculation integrated chip and control method

This application claims priority to the Chinese patent application submitted to the China Patent Office on June 22, 2022, with the application number 202210708777.8 and the invention title "A storage and calculation integrated chip, control method", the entire content of which is incorporated herein by reference. Applying.

Technical field

The present application relates to the field of semiconductor integrated circuits, and in particular, to an integrated storage and calculation chip and a control method.

Background technique

In recent years, in order to solve the bottleneck of traditional von Neumann computing architecture, the in-memory computing chip structure has received widespread attention. The basic idea is to directly use the memory to perform logical calculations, thereby reducing the amount of data transmission between the memory and the processor and transmission distance, reducing power consumption while improving performance.

Once the existing storage and calculation integrated chip structure for autonomous driving is customized, its circuit structure is fixed and cannot be flexibly adjusted according to the actual changing environment, nor can it meet the control timeliness and comfort requirements of different customers.

technical problem

This application provides a storage and calculation integrated chip and control method to solve the technical problem that the above-mentioned automatic driving vehicle control cannot be flexibly adjusted according to the actual changeable environment, and cannot meet the control timeliness and comfort requirements of different customers.

Technical solutions

This application provides an integrated storage and calculation chip for use in autonomous vehicles. The integrated storage and calculation chip includes: an acquisition unit that acquires environmental data around the vehicle; a first neural network unit that, when meeting the preset security level, Output similarities with multiple environments according to the input environmental data; a second neural network unit, the second neural network unit includes an input layer, a plurality of hidden layers, an output layer, the input layer, the output layer and the hidden layer. The layer distribution includes a plurality of neuron nodes, and the plurality of neuron nodes of the input layer, the plurality of neuron nodes of the hidden layer, and the plurality of neuron nodes of the output layer are connected in turn through transmission channels, and the transmission channels The channel is controlled on and off by a control switch; the network level of the hidden layer and the neuron nodes of the hidden layer, the multiple neuron nodes of the input layer, and the multiple neuron nodes of the output layer are obtained according to the similarity of multiple environments. neuron nodes and closed transmission channels, and the second neural network outputs control instructions according to the similarity of the multiple environments.

Optionally, when training the second neural network, it includes: using the similarity of multiple environments as the input data set, using the control time as the output data set, using the control time and comfort as the evaluation function, and using a genetic algorithm to find the second neural network. The network structure of the neural network, the evaluation function is set to: p=w1Aw2B, where: w1, w2 are weights, A is comfort, and B is control duration.

Optionally, the first neural network unit, when meeting a preset security level, outputs similarities with multiple environments based on the input environmental data; including: the first neural network unit has the same characteristics as the second neural network The network structure of the first neural network is obtained according to the preset conditions, and the first neural network outputs the similarity of the multiple environments according to the environmental data.

Optionally, the first neural network unit, when meeting a preset safety level, outputs similarities with multiple environments based on the input environmental data; including: a determination unit that determines the risk of collision between the vehicle and surrounding obstacles, based on Different collision risks determine the safety level of the vehicle; the emergency processing unit directly controls the vehicle for emergency avoidance when the safety level is lower than the preset level.

Optionally, the storage and calculation integrated chip includes: when the number of the transmission channels is greater than the number of channels of the storage and calculation integrated chip, the data of the two transmission channels are spliced to obtain the spliced data to be processed; the computing unit can The spliced data to be processed is processed through one processing batch.

This application provides a control method that is applied to autonomous vehicles to obtain environmental data around the vehicle; when the preset safety level is met, the first neural network unit outputs similarities with multiple environments based on the input environmental data; according to The similarity of multiple environments obtains the network level of the hidden layer and the neuron nodes of the hidden layer, the multiple neuron nodes of the input layer, the multiple neuron nodes of the output layer and the closed transmission channel , the second neural network outputs control instructions according to the similarity of the multiple environments.

Optionally, when the preset safety level is met, the first neural network unit outputs similarities with multiple environments based on the input environmental data; including: determining the collision risk between the vehicle and surrounding obstacles, and based on different collision risks Determine the safety level of the vehicle; when the safety level is lower than the preset level, directly control the vehicle for emergency avoidance.

Optionally, when the number of transmission channels is greater than the number of channels of the storage and calculation integrated chip, the data of the two transmission channels are spliced to obtain the spliced data to be processed; the computing unit can complete the splicing through one processing batch. The subsequent processing of data to be processed.

beneficial effects

This application obtains the network level of the hidden layer and the neuron nodes of the hidden layer by combining the similarity of multiple environments, as well as the closed transmission channel. The second neural network outputs control instructions according to the input environmental familiarity, which can be based on Different environments transform different second neural network structures, so that the best control method can be achieved, the control time is shortened, the control accuracy is improved, and the adaptability to the environment is improved. In addition, through the first neural network and The cooperation of the second neural network can further increase the transmission speed of control and further shorten the control time of autonomous driving. In addition, the environment recognition and decision-making modules in autonomous driving are integrated into one chip, that is, the first neural network and the second neural network. A neural network chip can improve the reliability of the chip and further reduce the risk of damage, that is, improving the reliability of autonomous vehicles.

Description of the drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. In order to more clearly explain the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to describe the embodiments or the prior art. Obviously, for those of ordinary skill in the art, It is said that other drawings can be obtained based on these drawings without exerting creative labor.

Figure 1 is a structural diagram of the storage and calculation integrated chip of this application.

Figure 2 is another structural diagram of the storage and calculation integrated chip of this application.

Figure 3 is a flow chart of the control method of this application.

Embodiments of the invention

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of this application are only used to explain the relationship between components in a specific posture (as shown in the drawings). Relative positional relationship, movement conditions, etc., if the specific posture changes, the directional indication will also change accordingly.

In addition, descriptions involving "first", "second", etc. in this application are for descriptive purposes only and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of indicated technical features. Therefore, features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In addition, the technical solutions in various embodiments can be combined with each other, but it must be based on the realization by those of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that such a combination of technical solutions does not exist. , nor is it within the scope of protection required by this application.

In the first aspect, this application discloses an integrated storage and calculation chip, which is applied to autonomous driving vehicles. As shown in Figure 1-2, the integrated storage and calculation chip includes: an acquisition unit to acquire environmental data around the vehicle; a first neural network unit , when the preset security level is met, the similarity with multiple environments is output based on the input environment data.

A second neural network unit. The second neural network unit includes an input layer, a plurality of hidden layers, and an output layer. The input layer, the output layer, and the hidden layer distribution include a plurality of neuron nodes. The input layer A plurality of neuron nodes, a plurality of neuron nodes of the hidden layer and a plurality of neuron nodes of the output layer are connected in sequence through transmission channels, and the transmission channels are controlled on and off by control switches.

Referring to Figure 1, for example, in this embodiment, the network level of the hidden layer is set to n layers, the number of neuron nodes of the hidden layer is set to n, the number of neuron nodes of the output layer is set to n, and the number of neuron nodes of the output layer is set to n. The number of neuron nodes is set to n, and the above-mentioned neuron nodes are connected through transmission channels to form an n-layer neural network. The specific structure of the second neural network can be obtained through training, and only the control accuracy and control time need to meet the preset Conditions are enough.

Obtain the network level of the hidden layer and the neuron nodes of the hidden layer, the multiple neuron nodes of the input layer, the multiple neuron nodes of the output layer and the closed transmission according to the similarity of multiple environments channel, the second neural network outputs control instructions according to the similarity of the multiple environments.

Referring to Figure 2, for example, in this embodiment, the network level of the hidden layer is set to 1 layer, the neuron nodes of the hidden layer are set to 5, the neuron nodes of the output layer are set to 2, and the neuron nodes of the output layer are set to 2. The number of neuron nodes is set to 3, and the above neuron nodes are connected through transmission channels to form a 3-layer neural network.

In this field, finding different second neural network structures through genetic algorithm optimization can help improve the timeliness of training. During the autonomous driving process, the driving scenario of the autonomous vehicle at this time can be determined based on the similarity of multiple environments. Different driving scenarios require different control time and comfort, such as: autonomous driving at this time The vehicle is in an accident-prone spot, and the control time of the autonomous vehicle needs to be short enough to avoid risks. In an environment where there are few vehicles around and the road is relatively smooth, the control time is the most important factor to consider, so that in different safety level situations Under the circumstances, it is hoped that the control time is the shortest and the comfort is the best. Different second neural network structures can meet the conditions, but a single neural network structure cannot better take advantage of flexible processing.

In this field, the input layer, output layer, and hidden layer of the first neural network are also performed in a transformation manner, which will not be described in detail here. However, in the process of processing environmental data, the processing speed of different neural network structures is obviously are inconsistent, therefore, the preset condition can be a neural network structure that is set and requires the accuracy of environmental recognition to meet the preset requirements, or it can be a neural network structure with low power consumption, and its specific requirements can be Choose according to actual situation. By setting the first neural network structure to a variable neural network structure, the control time of the autonomous vehicle can be further shortened, and the accuracy of environment recognition can also be improved.

Optionally, the first neural network unit, when the preset safety level is met, outputs similarities with multiple environments based on the input environmental data; including a determination unit and an emergency processing unit, the determination unit determines the distance between the vehicle and surrounding obstacles The safety level of the vehicle is determined based on different collision risks; the emergency processing unit directly controls the vehicle for emergency avoidance when the safety level is lower than the preset level.

Optionally, the storage and calculation integrated chip includes: when the number of the transmission channels is greater than the number of channels of the storage and calculation integrated chip, the data of the two transmission channels are spliced to obtain the spliced data to be processed; the computing unit can pass a The processing batch completes the processing of the spliced data to be processed.

Theoretically, the number of channels processed by the chip can be consistent with the number of transmission channels of the second neural network. However, due to consideration of processing costs, the number of chips is generally limited to a certain range, and the transmission channels of the chip are smaller than the input layer and hidden layer. , the number of transmission channels of the hidden layer and the output layer requires data processing. For example: in the process of training the second neural network, the number of transmission channels of the input layer and the hidden layer is required. The number of transmission channels of the hidden layer and the output layer may be set to 4. However, during the processing of the storage and computing integrated chip, the possible transmission channels are 3. These process the data to prevent data accumulation and try to increase the control time, for example, the input layer and the hidden layer, the hidden layer and the output layer. The data to be processed in the transmission channel includes data of 4 channels, and the data of the 4 channels are: first channel data, second channel data, third channel data, and fourth channel data. The number of input channels of the chip is 3. By splicing the first channel data and the second channel data, the fifth channel data is obtained. The third channel data, the fourth channel data and the fifth channel data are used as the data to be processed after splicing. In this way, the number of channels of the spliced data to be processed is 3. The chip can complete the processing of the spliced first data to be processed through one processing batch, that is, the chip can complete the processing of the first data to be processed.

This application includes a control method applied to autonomous vehicles, as shown in Figure 3: The control method includes the following steps.

Step S1: Obtain environmental data around the vehicle.

Step S2: When the preset security level is met, the first neural network unit outputs similarities with multiple environments based on the input environment data.

Step S3: Obtain the network level of the hidden layer and the neuron nodes of the hidden layer, the multiple neuron nodes of the input layer, the multiple neuron nodes of the output layer and In a closed transmission channel, the second neural network outputs control instructions according to the similarity of the multiple environments.

Optionally, step S2, when the preset safety level is met, the first neural network unit outputs similarities with multiple environments based on the input environmental data; including: determining the collision risk between the vehicle and surrounding obstacles, and determining the collision risk between the vehicle and surrounding obstacles according to different collisions. The risk determines the safety level of the vehicle; when the safety level is lower than the preset level, the vehicle is directly controlled for emergency avoidance.

Optionally, when the number of transmission channels is greater than the number of channels of the storage and calculation integrated chip, the data of the two transmission channels are spliced to obtain the spliced data to be processed; the computing unit can complete the spliced data through one processing batch. Processing of data to be processed.

The above descriptions are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be practiced in other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

An integrated storage and calculation chip, used in autonomous vehicles, wherein the integrated storage and calculation chip includes:

The acquisition unit acquires environmental data around the vehicle;

The first neural network unit, when meeting the preset security level, outputs similarities with multiple environments based on the input environmental data; and

A second neural network unit. The second neural network unit includes an input layer, a plurality of hidden layers, and an output layer. The input layer, the output layer, and the hidden layer distribution include a plurality of neuron nodes. The input layer A plurality of neuron nodes, a plurality of neuron nodes of the hidden layer and a plurality of neuron nodes of the output layer are connected in turn through transmission channels, and the transmission channels are controlled on and off by control switches;

Obtain the network level of the hidden layer and the neuron nodes of the hidden layer, the multiple neuron nodes of the input layer, the multiple neuron nodes of the output layer and the closed transmission according to the similarity of multiple environments channel, the second neural network outputs control instructions according to the similarity of the multiple environments.
The storage and calculation integrated chip according to claim 1, wherein when training the second neural network, it includes: using the similarity of multiple environments as an input data set and using the control time as an output data set to control time and comfort. is the evaluation function, and the network structure of the second neural network is found through genetic algorithm. The evaluation function is set as: p=w1A-w2B, where: w1, w2 are weights, A is comfort, and B is the control duration.
The storage and calculation integrated chip according to claim 1 or 2, wherein the first neural network unit, when meeting the preset security level, outputs similarities with multiple environments based on the input environmental data; including:

The first neural network unit has the same network structure as the second neural network;

The network structure of the first neural network is obtained according to the preset conditions, and the first neural network outputs the similarity of the multiple environments according to the environmental data.
The storage and calculation integrated chip according to claim 3, which includes: the first neural network unit, when meeting the preset security level, outputs similarities with multiple environments according to the input environmental data; including:

The determination unit determines the risk of collision between the vehicle and surrounding obstacles, and determines the safety level of the vehicle based on different collision risks;

The emergency processing unit directly controls the vehicle for emergency evacuation when the safety level is lower than the preset level.
The storage and calculation integrated chip according to claim 1, wherein the storage and calculation integrated chip includes: when the number of the transmission channels is greater than the number of channels of the storage and calculation integrated chip, the data of the two transmission channels are spliced to obtain the splicing The final data to be processed; the computing unit can complete the processing of the spliced data to be processed through one processing batch.
A control method applied to autonomous vehicles, including:

Obtain environmental data around the vehicle;

When the preset security level is met, the first neural network unit outputs similarities with multiple environments based on the input environment data;

Obtain the network level of the hidden layer and the neuron nodes of the hidden layer, the multiple neuron nodes of the input layer, the multiple neuron nodes of the output layer and the closed transmission according to the similarity of multiple environments channel, the second neural network outputs control instructions according to the similarity of the multiple environments.
The control method according to claim 6, comprising: when the preset security level is met, the first neural network unit outputs similarities with multiple environments according to the input environmental data; including:

Determine the risk of collision between the vehicle and surrounding obstacles, and determine the safety level of the vehicle based on different collision risks;

When the safety level is lower than the preset level, the vehicle is directly controlled for emergency avoidance.
The control method according to claim 6, which includes: when the number of the transmission channels is greater than the number of channels of the storage and calculation integrated chip, the data of the two transmission channels are spliced to obtain the spliced data to be processed; the computing unit , the processing of the spliced data to be processed can be completed through one processing batch.