CN114781628A - Memristor noise-based data enhancement method and device, electronic equipment and medium - Google Patents

Abstract

The application discloses a memristor noise-based data enhancement method, a memristor noise-based data enhancement device, electronic equipment and a medium, wherein the method comprises the following steps: determining a mapping relation representing a relation between input data and output data; mapping a mapping network corresponding to the mapping relation to the target memristor array based on the mapping relation; and inputting the input data to the mapped target memristor array, and applying random noise to the target memristor array to obtain data-enhanced output data. According to the embodiment of the application, random noise of the memristor is utilized to enhance data, the enhanced data have diversity and randomness, and the problems that in the related technology, a data set applicable to an offline data enhancement mode is small, an online data enhancement mode is long in time consumption and low in efficiency, and the data enhancement mode is single are solved.

Description

Data enhancement method, device, electronic device and medium based on memristor noise

technical field

The present application relates to the technical field of data enhancement, and in particular, to a data enhancement method, device, electronic device and storage medium based on memristor noise.

Background technique

Artificial intelligence is a technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human behavior. , thinking, planning, etc.), mainly including the principle of computer realization of intelligence, the manufacture of computers similar to human brain intelligence, so that computers can achieve higher-level applications. Usually, in order to obtain an ideal artificial intelligence model, people use a large amount of labeled data to make the model perform supervised learning. The model can learn a function from the given training data set, that is, the model parameters. When new data arrives, the result can be predicted according to this function, so as to achieve the purpose of prediction.

The predictive effect of supervised learning is largely positively related to the amount and variety of data received during the training phase. With the continuous increase in the scale of neural networks, the demand for data volume and data diversity is also increasing. However, compared with the huge data volume requirements, existing datasets often cannot meet its requirements. Therefore, a method to solve this problem is to use data augmentation, that is, to generate new training data by performing certain transformation on the limited data, so as to achieve the purpose of expanding the data. In addition to expanding data volume and data diversity, data augmentation can also be used to solve the class imbalance problem in classification tasks, such as using data augmentation to adjust the ratio of positive and negative samples.

There are many implementation methods for data enhancement. The following takes image data as an example, and realizes data enhancement by regularly transforming existing data such as cropping, flipping, and rotating. Data augmentation is usually divided into offline data augmentation and online data augmentation. Offline data augmentation refers to caching the expanded data set after processing the data set for model training and inference. Online data augmentation refers to changing only the data of the current training batch during model training and inference. One of the most similar online data enhancement schemes in this paper is called autoencoder, which firstly compresses and encodes data into a specific vector, and then adds a sampled Gaussian noise to the specific vector. The data enhancement scheme that restores it to the original image through the decoder is called an autoencoder. The core of this method is to transform the data by adding a random Gaussian disturbance after passing the data through the neural network. Since random perturbations are added, the robustness of the trained model can be improved.

Among them, offline data enhancement directly processes the data set, and the number of data becomes the enhancement factor multiplied by the number of the original data set. The advantage of this scheme is that there is no need to increase the model training and inference time, and the pre-processed data can be used for different tasks of multiple models. However, the shortcomings of the offline data enhancement scheme are also very obvious. Because the data set is directly processed, this scheme has high requirements on the chip cache. Therefore this scheme is usually used for smaller datasets.

Online data augmentation is to transform only the data of the currently used batch during the model training process. During training, data that has already been used will not be saved. The advantage of this solution is that no additional cache is required, and data can be deleted immediately during model training and inference. However, this scheme also has obvious shortcomings. When encountering more complex transformations, online data enhancement will lengthen the time for model training and inference. Taking the autoencoder scheme mentioned above as an example, the process of generating random numbers will take a lot of time. Randomly generating a large number of random numbers for each batch will seriously increase the time cost of model training and inference.

Using a single transformation method, such as translation, rotation, scaling, etc., to transform an image is one of the commonly used solutions for image data enhancement. The main defect is that the generated images lack diversity and randomness, so the improvement of the robustness of the model cannot achieve the expected effect.

SUMMARY OF THE INVENTION

The present application provides a data enhancement method, device, electronic device and storage medium based on memristor noise. The random noise of memristor is used for data enhancement, and the enhanced data has diversity and randomness, which solves the problem of offline processing in related technologies. The data enhancement method is suitable for small datasets, the online data enhancement method is time-consuming and inefficient, and the data enhancement method has a single problem.

An embodiment of the first aspect of the present application provides a method for data enhancement based on memristor noise, including the following steps: determining a mapping relationship representing the relationship between input data and output data; based on the mapping relationship, mapping the mapping relationship to The mapping network is mapped to the target memristor array; and the input data is input to the mapped target memristor array, and after random noise is applied to the target memristor array, the data-enhanced data is obtained. the output data.

Optionally, in an embodiment of the present application, the method further includes: using the mapping relationship as training data to perform neural network training, and when a training termination condition is met, stopping the training to obtain the mapping network.

Optionally, in an embodiment of the present application, the input data is input into the mapped target memristor array, and data enhancement is obtained after random noise is applied to the target memristor array. The latter output data includes: inputting the voltage signal of the input data to the mapped target memristor array; after applying random noise through the memristor array, outputting the current signal of the input data ; Convert the current signal to obtain the data-enhanced output data.

Optionally, in an embodiment of the present application, the training termination condition includes: the loss function of the mapping network is less than a preset threshold; and/or, the error between the input data and the output data is less than Preset error threshold.

An embodiment of the second aspect of the present application provides a data enhancement device based on memristor noise, including: an acquisition module for determining a mapping relationship representing the relationship between input data and output data; a mapping module for based on the mapping relationship, mapping the mapping network corresponding to the mapping relationship to the target memristor array; and an enhancement module, used for inputting the input data to the mapped target memristor array, and in the target memristor array After random noise is applied to the generator array, the output data after data enhancement is obtained.

Optionally, in an embodiment of the present application, it further includes: a training module configured to use the mapping relationship as training data to perform neural network training, and when the training termination condition is met, stop training to obtain the mapping network.

Optionally, in an embodiment of the present application, the enhancement module is specifically configured to input the voltage signal of the input data to the mapped target memristor array, and apply the voltage signal through the memristor array. After random noise, the current signal of the input data is output, and the current signal is converted to obtain the data-enhanced output data.

Optionally, in an embodiment of the present application, the training termination condition includes: the loss function of the mapping network is less than a preset threshold; and/or the error between the input data and the output data is less than a preset threshold. Set the error threshold.

An embodiment of a third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to execute The data augmentation method based on memristor noise as described in the above embodiments.

Embodiments of the fourth aspect of the present application provide a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to execute the method for data enhancement based on memristor noise as described in the foregoing embodiments.

In the embodiment of the present application, the noise of the memristor is used to realize the function of adding noise disturbance in data enhancement, which avoids the time required to generate random numbers. Gaussian noise can realize data enhancement of limited data, and the generated images after data enhancement have diversity and strong randomness, and the output data of the memristor is directly input into the task network for application, and can be deleted after use without caching.

Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a flowchart of a method for data enhancement based on memristor noise provided according to an embodiment of the present application;

2 is a schematic structural diagram of a memristor provided according to an embodiment of the present application;

3 is a schematic diagram of a memristor array provided according to an embodiment of the present application;

4 is a schematic diagram of data generation of a generation network according to an embodiment of the present application;

FIG. 5 is a schematic diagram of embedding a memristor array according to an embodiment of the present application;

6 is a schematic diagram of another memristor array embedding provided according to an embodiment of the present application;

FIG. 7 is an exemplary diagram of a data enhancement apparatus based on memristor noise according to an embodiment of the present application;

FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the application.

Detailed ways

The following describes in detail the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.

The method, apparatus, electronic device, and medium for data enhancement based on memristor noise according to the embodiments of the present application are described below with reference to the accompanying drawings. Aiming at the problems that the offline data enhancement method mentioned by the above-mentioned background technology center is applicable to a small dataset, the online data enhancement method is time-consuming, has low efficiency, and the data enhancement method is single, the embodiment of the present application uses the noise of the memristor to achieve The function of adding noise perturbation in data enhancement avoids the time required to generate random numbers. In the process of data enhancement, the real noise of the memristor is replaced by the random Gaussian noise used in related technologies, which can realize data enhancement for limited data. , and the image generated after data enhancement has diversity and strong randomness, and the output data of the memristor is directly input into the task network for application, and can be deleted after use without caching. As a result, the problems in the related art that the offline data enhancement method is applicable to a small dataset, the online data enhancement method is time-consuming, and the efficiency is low, and the data enhancement method is single, are solved.

Specifically, FIG. 1 is a flowchart of a method for data enhancement based on memristor noise provided according to an embodiment of the present application.

As shown in Figure 1, the memristor noise-based data augmentation method includes the following steps:

In step S101, a mapping relationship representing the relationship between the input data and the output data is determined.

It can be understood that the mapping relationship in this embodiment of the present application may be an inherent mapping relationship between input data and output data. For example, with a known resistance size, according to the input current value and Ohm’s law, the output voltage value may also be based on the actual value. The mapping relationship between the input data and the output data needs to be set, for example, the difference between the output data value and the input data value is a fixed value or the output data value is a preset multiple of the input data value, etc., which are not specifically limited.

In step S102, based on the mapping relationship, the mapping network corresponding to the mapping relationship is mapped to the target memristor array.

After the mapping relationship between the input data and the output data is determined through step S101, the neural network is trained according to the mapping relationship, and the obtained mapping network is mapped to the memristor array.

As shown in Figure 2, a structural form of the memristor is shown, and it can be understood that the memristor itself has a noise conforming to a Gaussian distribution, which fluctuates randomly within a certain range. Memristor device noise has the characteristics of uncertainty and randomness. Therefore, the data generated by using memristor device noise as a perturbation addition source also has the characteristics of diversity and randomness. This advantage can effectively avoid the drawbacks brought about by a single method of data enhancement, making the model more robust after training.

Optionally, in an embodiment of the present application, the neural network training is performed using the mapping relationship as training data, and when the training termination condition is met, the training is stopped to obtain the mapping network. Among them, the input dimension of the mapping network may be equal to the output dimension, or may not be equal to the output dimension.

Optionally, in an embodiment of the present application, the training termination condition includes: the loss function of the mapping network is less than a preset threshold, and/or the error between the input data and the output data is less than a preset error threshold.

The loss function in this embodiment of the present application may be a pixel-level mean square error or other errors, which are not specifically limited.

Specifically, when training the mapping network, when a preset training termination condition is reached, the embodiment of the present application stops training, and the training termination condition in the embodiment of the present application may be that the loss function of the mapping network is less than a set threshold, for example, When the threshold is A, the training stops when the value of the loss function is less than A. The training termination condition in this embodiment of the present application may also be that the error between the input data and the output data is less than a preset error threshold. For example, the input data and the output data are quantized according to a certain standard, and the quantized input data and the output data are compared. The difference between the data, when the difference is less than the preset error threshold, stop training. The training termination condition in the embodiment of the present application may also be that the training round or the duration reaches a preset value, etc., which can be set by those skilled in the art according to the actual situation, which is not specifically limited.

FIG. 2 shows the structure of a single memristor. In the embodiment of the present application, multiple memristor structures can be connected. As shown in FIG. 3 , a memristor array with multiple rows and columns is formed by multiple memristors. , the mapping network trained in the above embodiment is mapped to a multi-row and multi-column memristor array.

As shown in Fig. 3, x1 to x5 represent original data, and x1' to x5' are data obtained after transformation. A mapping network is obtained by training a simple neural network, as shown in Figure 4. It should be noted that Figure 4 only shows the basic principle of training. The specific network structure and parameters can be set according to the actual situation. The network has an input dimension equal to the output dimension. specialty. The training of the mapping network takes the pixel-level mean square error as the loss function, and trains with the goal of reducing the error between the input and the output. The trained mapping network is mapped into the memristor array, that is, the network weights are written into the memristor.

In step S103, input data is input to the mapped target memristor array, and after random noise is applied to the target memristor array, data-enhanced output data is obtained.

It can be understood that the mapped memristor array can output transformed output data according to the input data. After the input data passes through the memristor array, it is input into the task network for operation. The batch of data can be deleted directly after use, and no additional space is needed for caching, which avoids the disadvantage that offline data enhancement requires a large amount of caching.

Optionally, in an embodiment of the present application, input data is input to the mapped target memristor array, and after random noise is applied to the target memristor array, data-enhanced output data is obtained, including: The voltage signal of the input data is input to the mapped target memristor array, and after applying random noise through the memristor array, the current signal of the input data is output, and the current signal is converted to obtain the data-enhanced output data.

Specifically, when data enhancement is performed, the voltage signal of the data flows through the memristor array to obtain a current output signal. Since the memristor itself has random noise, the resulting output signal can be regarded as the data obtained by adding noise to the original data. Noise itself has the characteristics of strong uncertainty and strong randomness, which makes the generated data also have the characteristics of diversity and randomness.

The data enhancement method based on the memristor noise in the embodiment of the present application can be embedded in any task in the form of online data enhancement, as shown in FIG. 5 . Before the input data is input into the task network, the input data is input into the mapped memristor array, and the input data of the current batch is first enhanced by the noised data, and then input into the task network, so as to achieve the purpose of online data enhancement. After the input data flows through the array, it is directly applied to each network, and then deleted. There is no need for additional space to cache the generated data, thus avoiding the drawbacks of offline data enhancement. At the same time, due to the uncertainty of noise, the noise added to each input data is different, thus avoiding the drawbacks brought by a single data enhancement method.

The data enhancement method based on the memristor noise in the embodiment of the present application can also be used as a separate module, embedded in some special data enhancement networks, and random perturbation is added to the intermediate code instead of the original data, such as an autoencoder, such as shown in Figure 6. An autoencoder network is embedded within the memristor array, replacing the part that would otherwise need to generate random numbers to add noise. Since the noise of the memristor device is used as the added source of random disturbance, it is not necessary to generate additional random numbers, so as to accelerate the speed of model training and inference, thus avoiding the drawbacks brought by online data augmentation.

By designing the memristor array as a separate data enhancement module, embedded in any task, the design scheme is flexible and has the advantage of plug-and-play.

It should be noted that, in the embodiments of the present application, the memristor can be replaced with other devices, such as phase change memory (PCM) or ferroelectric memory (FeRAM), etc. according to actual needs.

The data enhancement method based on the memristor noise in the embodiment of the present application uses the noise of the memristor to realize the function of adding noise disturbance in the data enhancement, which avoids the time required to generate random numbers. The real noise replaces the random Gaussian noise used in the related technology, which can realize data enhancement of limited data, and the generated image after data enhancement has diversity and strong randomness, and the output data of the memristor is directly input into the task network for application, It can be deleted after use without caching. As a result, the problems in the related art that the offline data enhancement method is applicable to a small dataset, the online data enhancement method is time-consuming, and the efficiency is low, and the data enhancement method is single, are solved.

Next, a data enhancement device based on memristor noise proposed according to an embodiment of the present application will be described with reference to the accompanying drawings.

FIG. 7 is an exemplary diagram of a data enhancement apparatus based on memristor noise according to an embodiment of the present application.

As shown in FIG. 7 , the device 10 for data enhancement based on memristor noise includes: an acquisition module 100 , a mapping module 200 and an enhancement module 300 .

The obtaining module 100 is configured to determine a mapping relationship representing the relationship between the input data and the output data. The mapping module 200 is configured to map the mapping network corresponding to the mapping relationship to the target memristor array based on the mapping relationship. The enhancement module 300 is used for inputting input data into the mapped target memristor array, and after random noise is applied to the target memristor array, to obtain data-enhanced output data.

Optionally, in an embodiment of the present application, it further includes: a training module configured to use the mapping relationship as training data to perform neural network training, and when the training termination condition is met, stop training to obtain a mapping network.

Optionally, in an embodiment of the present application, the enhancement module 300 is specifically configured to input the voltage signal of the input data into the mapped target memristor array, and after applying random noise through the memristor array, output the voltage signal of the input data. The current signal is converted to obtain the output data after data enhancement.

Optionally, in an embodiment of the present application, the training termination condition includes: the loss function of the mapping network is less than a preset threshold, and/or the error between the input data and the output data is less than a preset error threshold.

It should be noted that the foregoing explanation of the embodiment of a memristor noise-based data enhancement method is also applicable to a memristor noise-based data enhancement apparatus of this embodiment, and details are not repeated here.

According to a data enhancement device based on memristor noise proposed in the embodiment of the present application, the noise of the memristor is used to realize the function of adding noise disturbance in data enhancement, which avoids the time required to generate random numbers. , the real noise of the memristor is replaced by the random Gaussian noise used in the related technology, which can realize data enhancement of limited data, and the generated image after data enhancement has diversity and strong randomness, and the output data of the memristor is directly input. The task network is used for application, and it can be deleted after use without caching. As a result, the problems in the related art that the offline data enhancement method is applicable to a small dataset, the online data enhancement method is time-consuming, and the efficiency is low, and the data enhancement method is single, are solved.

FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may include:

Memory 801 , processor 802 , and computer programs stored on memory 801 and executable on processor 802 .

When the processor 802 executes the program, the method for data enhancement based on the memristor noise provided in the above embodiments is implemented.

Further, the electronic device also includes:

The communication interface 803 is used for communication between the memory 801 and the processor 802 .

The memory 801 is used to store computer programs that can be executed on the processor 802 .

The memory 801 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

If the memory 801, the processor 802 and the communication interface 803 are independently implemented, the communication interface 803, the memory 801 and the processor 802 can be connected to each other through a bus and complete communication with each other. The bus may be an Industry Standard Architecture (referred to as ISA) bus, a Peripheral Component (referred to as PCI) bus, or an Extended Industry Standard Architecture (referred to as EISA) bus or the like. The bus can be divided into address bus, data bus, control bus and so on. For ease of presentation, only one thick line is used in FIG. 8, but it does not mean that there is only one bus or one type of bus.

Optionally, in terms of specific implementation, if the memory 801, the processor 802 and the communication interface 803 are integrated on one chip, the memory 801, the processor 802 and the communication interface 803 can communicate with each other through an internal interface.

The processor 802 may be a central processing unit (Central Processing Unit, CPU for short), or a specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), or is configured to implement one or more of the embodiments of the present application integrated circuit.

This embodiment also provides a computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the above-mentioned data enhancement method based on memristor noise is implemented.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or N of the embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present application, "N" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined.

Any process or method description in the flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or N more executable instructions for implementing custom logical functions or steps of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

Claims (10)

1. A data enhancement method based on memristor noise is characterized by comprising the following steps:

determining a mapping relation representing the relation between input data and output data;

mapping a mapping network corresponding to the mapping relation to a target memristor array based on the mapping relation; and

inputting the input data to the mapped target memristor array, and obtaining the output data after data enhancement after random noise is applied to the target memristor array.

2. The method of claim 1, further comprising:

and performing neural network training by using the mapping relation as training data, and stopping training when a training termination condition is met to obtain the mapping network.

3. The method of claim 1, wherein the inputting the input data to the mapped target memristor array and obtaining the data-enhanced output data after random noise is applied by the target memristor array comprises:

inputting a voltage signal of the input data to the mapped target memristor array;

outputting a current signal of the input data after applying random noise through the memristor array;

and converting the current signal to obtain the output data after the data enhancement.

4. The method of claim 2, wherein the training termination condition comprises:

the loss function of the mapping network is smaller than a preset threshold value; and/or

An error between the input data and the output data is less than a preset error threshold.

5. A memristor noise-based data enhancement device, comprising:

the acquisition module is used for determining a mapping relation representing the relation between the input data and the output data;

the mapping module is used for mapping a mapping network corresponding to the mapping relation to a target memristor array based on the mapping relation; and

and the enhancing module is used for inputting the input data to the mapped target memristor array and obtaining the output data after data enhancement after random noise is applied to the target memristor array.

6. The apparatus of claim 5, further comprising:

and the training module is used for carrying out neural network training by taking the mapping relation as training data, and stopping training when a training termination condition is met to obtain the mapping network.

7. The apparatus according to claim 5, wherein the enhancement module is specifically configured to input a voltage signal of the input data to the mapped target memristor array, output a current signal of the input data after applying random noise through the memristor array, and convert the current signal to obtain the data-enhanced output data.

8. The apparatus of claim 6, wherein the training termination condition comprises:

the loss function of the mapping network is smaller than a preset threshold value; and/or

An error between the input data and the output data is less than a preset error threshold.

9. An electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the memristor noise-based data enhancement method as defined in any one of claims 1-4.

10. A computer-readable storage medium having stored thereon a computer program, the program being executable by a processor for implementing a memristor noise-based data enhancement method as defined in any one of claims 1-4.

Images