CN113762482B - Training method and related device for neural network model for automatic driving - Google Patents
Training method and related device for neural network model for automatic driving Download PDFInfo
- Publication number
- CN113762482B CN113762482B CN202111080765.7A CN202111080765A CN113762482B CN 113762482 B CN113762482 B CN 113762482B CN 202111080765 A CN202111080765 A CN 202111080765A CN 113762482 B CN113762482 B CN 113762482B
- Authority
- CN
- China
- Prior art keywords
- data
- implicit
- layer
- neural network
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003062 neural network model Methods 0.000 title claims abstract description 130
- 238000012549 training Methods 0.000 title claims abstract description 53
- 238000000034 method Methods 0.000 title claims abstract description 46
- 239000011159 matrix material Substances 0.000 claims abstract description 34
- 239000002131 composite material Substances 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 8
- 238000010606 normalization Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 5
- 238000003475 lamination Methods 0.000 claims 1
- 239000010410 layer Substances 0.000 description 123
- 230000006870 function Effects 0.000 description 9
- 238000013135 deep learning Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 230000002085 persistent effect Effects 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013501 data transformation Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000001755 vocal effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 239000002355 dual-layer Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003121 nonmonotonic effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a training method and a related device for a neural network model for automatic driving. The method comprises the following steps: acquiring a preset neural network model; constructing an implicit data enhancement network structure for outputting implicit noise data; accessing the implicit data enhancement network structure into the preset neural network model, so that the implicit noise data output by the implicit data enhancement network structure and hidden layer characteristic data in the preset neural network model are subjected to matrix addition; training the preset neural network model accessed with the implicit data enhancement network structure to obtain the preset neural network model accessed with the implicit data enhancement network structure after network parameter updating. According to the scheme, the universality of training data enhancement can be improved, and the anti-interference capability of the model is improved.
Description
Technical Field
The present disclosure relates to the field of navigation technologies, and in particular, to a training method and related apparatus for a neural network model for autopilot.
Background
The neural network model (i.e., artificial neural network model) is an operation model, and is formed by interconnecting a plurality of nodes (or neurons). The deep learning concept is derived from the research of an artificial neural network, and a multi-layer sensor with a plurality of hidden layers is a deep learning structure. Deep learning forms more abstract high-level representation attribute categories or features by combining low-level features to discover distributed feature representations of data. The neural network based on deep learning has excellent application effect in the fields of computer vision, natural language, text processing and the like. For different application fields, there are many different deep learning neural network models, such as a computer vision network model (e.g., a target detection model). The target detection model is widely applied in the technical field of navigation, and provides support for the realization of the automatic driving function of the automobile.
Taking a computer vision network model as an example, in order to improve the detection performance of the model, the model needs to be trained by using a data set. However, the data sets available for training learning are limited, and data enhancement may be performed on the data sets in order to improve data diversity and quality (data enhancement modes may include rotation, translation, flipping, scaling, cropping, color conversion, contrast conversion, mixup, mosaic, etc. on the input image), so that the learning ability, generalization ability, and robustness of the model may be improved with the limited data sets.
However, the current data enhancement modes all adopt specified rules to perform data transformation, and have no universality; in addition, the improvement effect on the learning capacity, generalization capacity and robustness of the model is limited, and the anti-interference capacity of the model cannot reach ideal strength.
Disclosure of Invention
In order to solve or partially solve the problems in the related art, the application provides a training method and a related device for an automatic driving neural network model, which can improve the universality of training data enhancement and improve the anti-interference capability of the model.
A first aspect of the present application provides a training method for a neural network model for autopilot, including:
acquiring a preset neural network model;
constructing an implicit data enhancement network structure for outputting implicit noise data;
accessing the implicit data enhancement network structure into the preset neural network model, so that the implicit noise data output by the implicit data enhancement network structure and hidden layer characteristic data in the preset neural network model are subjected to matrix addition;
training the preset neural network model accessed with the implicit data enhancement network structure to obtain the preset neural network model accessed with the implicit data enhancement network structure after network parameter updating.
In one embodiment, the implicit data enhancement network structure is configured to receive random noise data and to convert the random noise data into the implicit noise data corresponding to the implicit noise profile.
In one embodiment, the backbone network of the preset neural network model at least includes a first network layer and a second network layer, and the first network layer outputs the hidden layer feature data to the second network layer;
the step of accessing the implicit data enhancement network structure to the preset neural network model so that the implicit noise data output by the implicit data enhancement network structure and the implicit layer characteristic data in the preset neural network model are subjected to matrix addition, comprising the following steps:
accessing the implicit data enhancement network structure into the preset neural network model, so that the input data of the second network layer are as follows: and the hidden layer characteristic data output by the first network layer and the hidden noise data output by the hidden data enhancement network structure are subjected to matrix addition.
In one embodiment, the training the preset neural network model with the implicit data enhancement network structure to obtain the preset neural network model with the implicit data enhancement network structure after network parameter updating includes:
inputting a preset training data set into the preset neural network model, inputting random noise data into the implicit data enhancement network structure, so that in the training process of the preset neural network model, the implicit noise data output by the first network layer and the implicit noise data output by the implicit data enhancement network structure are subjected to matrix addition, and are input into the second network layer;
updating the preset neural network model and the network parameters in the implicit data enhancement network structure through back propagation to obtain the preset neural network model with the implicit data enhancement network structure after network parameter updating.
In one embodiment, the implicit noise data output by the implicit data enhancement network structure is the same in size dimension as the implicit noise data.
In one embodiment, the implicit noise data is equal to the sum of random noise data multiplied by a noise expansion coefficient parameter and a data domain deviation parameter; wherein the noise expansion coefficient parameter and the data domain deviation parameter are network parameters of the implicit data enhancement network structure.
In one embodiment, the implicit data enhancement network structure comprises:
an input layer for receiving random noise data;
the network structure layer at least comprises a convolution layer and is used for converting random noise data into recessive noise data;
and the output layer is used for outputting the recessive noise data.
In one embodiment, the network structure layer at least comprises a composite module and a deconvolution layer connected with the composite module, and the composite module comprises a convolution layer, a batch normalization layer and an activation function layer which are sequentially connected.
A second aspect of the present application provides a training device for an autopilot neural network model, comprising:
the acquisition module is used for acquiring a preset neural network model;
the construction module is used for constructing an implicit data enhancement network structure for outputting the implicit noise data;
the access module is used for accessing the implicit data enhancement network structure constructed by the construction module into the preset neural network model acquired by the acquisition module, so that the implicit noise data output by the implicit data enhancement network structure and the implicit layer characteristic data in the preset neural network model are subjected to matrix addition;
the training module is used for training the preset neural network model which is processed by the access module and is accessed with the implicit data enhancement network structure, and obtaining the preset neural network model which is updated in network parameters and is accessed with the implicit data enhancement network structure.
A third aspect of the present application provides a neural network model, comprising:
the network parameters obtained by the method are updated and then are connected with a preset neural network model of an implicit data enhancement network structure.
A fourth aspect of the present application provides an electronic device, comprising:
a processor; and
a memory having executable code stored thereon which, when executed by the processor, causes the processor to perform the method as described above.
A fifth aspect of the present application provides a computer readable storage medium having stored thereon executable code which, when executed by a processor of an electronic device, causes the processor to perform a method as described above.
The technical scheme that this application provided can include following beneficial effect:
according to the method, the built implicit data enhancement network structure is accessed into the preset neural network model, so that the implicit noise data output by the implicit data enhancement network structure can be subjected to matrix addition with the implicit layer characteristic data in the preset neural network model. That is, the implicit noise data output by the implicit data enhancement network structure adds noise to the input data of the second network layer, so that the model cannot fit all the features of the input data in the training process, and further plays a role in preventing overfitting. Training the preset neural network model with the implicit data enhancement network structure, and obtaining the preset neural network model with the implicit data enhancement network structure after network parameter updating. It can be understood that the hidden noise data can cause disturbance to hidden layer characteristic data of the preset neural network model, so that a wider noise space can be manufactured in the training process, and the model anti-interference capability of the preset neural network model with the hidden data enhancement network structure is enhanced after the network parameters are updated.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The foregoing and other objects, features and advantages of the application will be apparent from the following more particular descriptions of exemplary embodiments of the application as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the application.
FIG. 1 is a flow chart of a training method for an autopilot neural network model, as shown in an embodiment of the present application;
FIG. 2 is a schematic diagram of a default neural network model with an implicit data enhancement network structure according to an embodiment of the present disclosure;
FIG. 3 is a schematic structural view of a training device for an autopilot neural network model shown in an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first message may also be referred to as a second message, and similarly, a second message may also be referred to as a first message, without departing from the scope of the present application. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the related art, the data enhancement modes aiming at the neural network model all adopt specified rules to perform data transformation, and the method has no universality; in addition, the improvement effect on the learning capacity, generalization capacity and robustness of the model is limited, and the anti-interference capacity of the model cannot reach ideal strength.
Aiming at the problems, the embodiment of the application provides a training method for an automatic driving neural network model, which can improve the universality of training data enhancement and improve the anti-interference capability of the model.
The following describes the technical scheme of the embodiments of the present application in detail with reference to the accompanying drawings.
Fig. 1 is a flow chart illustrating a training method for an autopilot neural network model according to an embodiment of the present application.
Referring to fig. 1, the method includes:
step S101, acquiring a preset neural network model.
The preset neural network model may be a neural network model based on deep learning, such as a computer vision network model. Specifically, the preset neural network model may be a network model for processing any kind of computer vision task, for example, may be an object detection model (such as YOLO model), a semantic segmentation model, an image classification model, and the like.
Step S102, an implicit data enhancement network structure for outputting the implicit noise data is constructed.
In this step, the implicit data enhancement network structure may be configured to receive random noise data and convert the random noise data to implicit noise data.
In an alternative embodiment, the implicit data enhancement network structure may comprise: an input layer, a network structure layer and an output layer.
The input layer is for receiving random noise data. The random noise data may be a random one-dimensional vector matrix, and further, the input layer may convert the random noise data of the one-dimensional vector matrix into the random noise data of the two-dimensional matrix using a reshape algorithm.
The network structure layer includes at least one convolution layer for converting random noise data into implicit noise data. In one embodiment, the network structure layer at least comprises a composite module and a deconvolution layer connected with the composite module, and the composite module comprises a convolution layer, a batch normalization layer and an activation function layer which are sequentially connected.
The composite module is a CBM, and the composite module CBM comprises a convolution layer volume (conv for short), a Batch normalization layer Batch normalization (BN for short) and a Mish activation function layer. The convolution layer conv plays a role in extracting data features and increasing the number of feature channels. The batch normalization layer BN can enable the data space to be smooth, quicken the convergence process and improve the training speed. The activation function layer enhances the nonlinear change of the neural network model, and the Mish activation function is a self-regularized non-monotonic neural activation function and is a smooth activation function, so that better information can be allowed to go deep into the neural network, and further better accuracy and generalization are obtained.
Wherein the deconvolution layer Deconv acts as an upsampling.
The output layer is used for outputting the recessive noise data.
Wherein, the recessive noise data is equal to the sum of the random noise data multiplied by the noise expansion coefficient parameter and the data domain deviation parameter; the noise expansion coefficient parameter and the data domain deviation parameter are network parameters of an implicit data enhancement network structure. For example, the implicit noise data is output_noise, the random noise data is input_noise, the noise expansion coefficient parameter is w, and the data domain deviation parameter is b. Output_noise=input_noise_w+b. The w noise expansion coefficient parameter may be a noise expansion coefficient matrix and the b data domain deviation parameter may be a data domain deviation matrix.
Referring to fig. 2 together, in this embodiment, the implicit data enhancement network structure (Implicit data enhancement) for outputting the implicit noise data includes an Input layer Input, a first composite module CBM, a first deconvolution layer Deconv, a second composite module CBM, a second deconvolution layer deconvolution, and an Output layer Output, which are sequentially connected.
Step S103, accessing the implicit data enhancement network structure into a preset neural network model, so that hidden noise data output by the implicit data enhancement network structure and hidden layer characteristic data in the preset neural network model are subjected to matrix addition.
It should be noted that, hidden layer feature data in the neural network model is preset, that is, middle layer features of the neural network model are preset, that is, data output by a hidden layer in the neural network model is preset. Each neuron layer between the input layer and the output layer in the neural network model is commonly called a hidden layer, and the hidden layers of different network models have different structures, and the hidden layers can be a convolution layer, a pooling layer, a full-connection layer and the like. The data output from the hidden layer of the previous stage to the hidden layer of the subsequent stage may be referred to as hidden layer feature data. Hidden layers in a neural network model may be understood as network layers of different parts of the backbone network in the neural network model.
Referring to fig. 2, in this step, the backbone network of the predetermined neural network Model may at least include a first network layer (i.e., model part a shown in fig. 2) and a second network layer (i.e., model part B shown in fig. 2), and the first network layer outputs hidden layer feature data to the second network layer. The first network layer and the second network layer are hidden layers in a preset neural network model, and the first network layer or the second network layer can be a convolution layer, a pooling layer or a full connection layer.
In this step, the output layer of the implicit data enhancement network structure may be connected between the first network layer and the second network layer of the backbone network of the preset neural network model, so that the implicit noise data output by the implicit data enhancement network structure and the implicit layer feature data in the preset neural network model are added in a matrix.
In an alternative embodiment, the implicit data enhancement network structure may be accessed to a preset neural network model, so that the input data of the second network layer is: and the hidden layer characteristic data output by the first network layer and the hidden noise data output by the hidden data enhancement network structure are subjected to matrix addition.
Further, the data obtained by matrix adding the hidden layer feature data and the hidden noise data may be referred to as hidden feature data, where the hidden feature data is input data of the second network layer.
Furthermore, the implicit data enhancement network structure can be enabled to output the implicit noise data corresponding to the implicit characteristic data, so that matrix addition of the implicit noise data and the implicit characteristic data is facilitated. Specifically, the implicit data enhancement network structure may be configured to receive random noise data and convert the random noise data into implicit noise data corresponding to the implicit noise feature data, thereby facilitating matrix addition of the implicit noise data and the implicit noise feature data. The implicit noise data output by the implicit data enhancement network structure is the same as the implicit layer characteristic data in size dimension. For example, if the hidden layer feature data is a feature map (feature map) of 80×80×512, the hidden noise data output by the hidden data enhancement network structure may also have a dimension of 80×80×512, so that matrix addition of the hidden layer feature data and the hidden noise data is ensured.
Step S104, training a preset neural network model accessed with an implicit data enhancement network structure to obtain the preset neural network model accessed with the implicit data enhancement network structure after network parameter updating.
In an alternative embodiment, training a preset neural network model with an implicit data enhancement network structure to obtain a preset neural network model with an implicit data enhancement network structure after network parameter updating, including:
11 Inputting the preset training data set into a preset neural network model, inputting random noise data into an implicit data enhancement network structure, so that hidden layer characteristic data output by a first network layer and hidden noise data output by the implicit data enhancement network structure are subjected to matrix addition in the training process of the preset neural network model, and input into a second network layer.
During each training process, the hidden layer feature data generates a set of random noise for each sample (batch-size) selected from the preset training data set. It can be understood that the hidden noise data and the hidden layer feature data in the preset neural network model are subjected to matrix addition, that is, the hidden noise data which represents the output of the hidden data enhancement network structure adds noise to the input data of the second network layer, so that the model cannot fit all the features of the input data in the training process, and further the effect of preventing overfitting is achieved. Noise is understood to mean, among other things, the disturbance data of a model, i.e. the disturbance of the model by adding disturbance information to the characteristic data in the model. The hidden noise data can cause disturbance to hidden layer characteristic data of a preset neural network model, so that a wider noise space can be manufactured in the training process.
Wherein, the implicit characteristic data (i.e. the data obtained by matrix adding the implicit characteristic data and the implicit noise data) may be represented as Y, the implicit characteristic data may be represented as Y, and the implicit noise data may be represented as output_noise. The implicit characteristic data y=the implicit layer characteristic data y+the implicit noise data output_noise. Wherein the implicit noise data output_noise=input_noise_w+b. Thus, the implicit characteristic data y=y+input_noise_w+b.
To facilitate an understanding of the matrix addition process of the hidden layer feature data and the hidden noise data, the following example is described. Let the feature size of the hidden layer feature data output by the first network layer be 80×80×512. In one embodiment, the constructed implicit data enhancement network structure may include an Input layer Input, a first composite module CBM, a first deconvolution layer Deconv, a second composite module CBM, a second deconvolution layer Deconv, and an Output layer Output, connected in sequence. The Input layer Input receives random noise data z of the one-dimensional vector matrix 400, converts the random noise data z into random noise data z of a two-dimensional matrix 40×40 by using a reshape algorithm, inputs the random noise data z to the first composite module CBM, the characteristic size of data Output after processing by the first composite module CBM is 20×20×128, the characteristic size of data Output after processing by the first deconvolution layer deconvolution is 40×40×128, the characteristic size of data Output after processing by the second composite module CBM is 40×40×512, and the characteristic size of data Output after processing by the second deconvolution layer is 80×80×512, so that the Output layer Output can Output hidden noise data of 80×80×512 corresponding to hidden layer characteristic data Output by the first network layer.
12 Updating the preset neural network model and network parameters in the implicit data enhancement network structure through back propagation to obtain the preset neural network model with the implicit data enhancement network structure after network parameter updating. Since the implicit noise data output_noise=random noise data input_noise, the noise expansion coefficient parameter w+the data domain deviation parameter b. After this step, the noise expansion coefficient parameter w and the data domain deviation parameter b are updated.
It can be understood that the model anti-interference capability of the preset neural network model with the implicit data enhancement network structure is enhanced after the network parameters are updated because the implicit noise data make a wider noise space in the training process of the model.
According to the method, the constructed implicit data enhancement network structure is accessed into the preset neural network model, so that the implicit noise data output by the implicit data enhancement network structure can be subjected to matrix addition with the implicit layer characteristic data in the preset neural network model. That is, the implicit noise data output by the implicit data enhancement network structure adds noise to the input data of the second network layer, so that the model cannot fit all the features of the input data in the training process, and further plays a role in preventing overfitting. Training the preset neural network model with the implicit data enhancement network structure, and obtaining the preset neural network model with the implicit data enhancement network structure after network parameter updating. It can be understood that the hidden noise data can cause disturbance to hidden layer characteristic data of the preset neural network model, so that a wider noise space can be manufactured in the training process, and the model anti-interference capability of the preset neural network model with the hidden data enhancement network structure is enhanced after the network parameters are updated.
In order to facilitate understanding of the technical effects produced by training the neural network model by using the technical scheme of the application, vocal performance is taken as a reference example, and the following metaphor description is made. The neural network model is not trained by utilizing a data enhancement mode and can be regarded as singing. The neural network model is trained using conventional data enhancement modes (rotation, translation, flipping, etc. of the input image) and can be viewed as a vocal dubbing. By utilizing the technical scheme of the application to train the neural network model, the neural network model can be regarded as the on-site singing of the concert. It will be appreciated that conventional data enhancement approaches are only data enhancement made for the input image, such that the model learning scope is constrained to a certain data space (the depth of the input image is limited). According to the model hidden layer feature data enhancement method, disturbance can be caused to hidden layer feature data of the model, the depth of data enhancement is larger, the model learning range is not limited by a certain data space, a wider noise space can be manufactured, and the anti-interference capability of the model is further enhanced.
It should be noted that, after training, the obtained network parameter is updated and is connected to a preset neural network model with an implicit data enhancement network structure, so that the network performance is better. When the preset neural network model with the implicit data enhancement network structure is used or verified after network parameter updating is accessed, the random noise data input_noise needs to be zero, namely the value of the data domain deviation parameter b is the value after model learning updating, and then the implicit characteristic data (namely, the data after matrix addition of the implicit characteristic data and the implicit noise data) is y=the implicit characteristic data y+the data domain deviation parameter b, namely, y=y+b. That is, the finally obtained network parameter updated is connected to a preset neural network model with an implicit data enhancement network structure, and when the network model is put into use, corresponding hidden layer characteristic data in the network model needs to be added with the updated data domain deviation parameter in a matrix mode. Therefore, the output effect of the network model is better, and the anti-interference capability of the network model is greatly improved.
The present application also provides a computer vision network model comprising: the network parameters obtained by the method are updated and then are connected with a preset neural network model of an implicit data enhancement network structure. When the preset neural network model with the implicit data enhancement network structure is put into use, the received random noise data is zero.
Corresponding to the embodiment of the application function implementation method, the application also provides a training device for the neural network model for automatic driving, electronic equipment and corresponding embodiments.
Fig. 3 is a schematic structural view of a training device for an autopilot neural network model according to an embodiment of the present application.
Referring to fig. 3, an embodiment of the present application provides a training apparatus for a neural network model for autopilot, including:
an acquiring module 301, configured to acquire a preset neural network model;
a building module 302, configured to build an implicit data enhancement network structure for outputting implicit noise data;
the access module 303 is configured to access the implicit data enhancement network structure constructed by the construction module 302 to the preset neural network model acquired by the acquisition module 301, so that hidden noise data output by the implicit data enhancement network structure and hidden layer feature data in the preset neural network model are added in a matrix;
the training module 304 is configured to train the preset neural network model with the implicit data enhancement network structure, which is processed by the access module 303, to obtain the preset neural network model with the implicit data enhancement network structure after the network parameters are updated.
According to the embodiment, the hidden noise data can cause disturbance to the hidden layer characteristic data of the preset neural network model, so that a wider noise space can be manufactured in the training process, and the model anti-interference capability of the preset neural network model with the hidden data enhancement network structure is enhanced after the network parameters are updated.
The specific manner in which the respective modules perform the operations in the apparatus of the above embodiments has been described in detail in the embodiments related to the method, and will not be described in detail herein.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Referring to fig. 4, an electronic device 400 includes a memory 410 and a processor 420.
The processor 420 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Memory 410 may include various types of storage units, such as system memory, read Only Memory (ROM), and persistent storage. Where the ROM may store static data or instructions that are required by the processor 420 or other modules of the computer. The persistent storage may be a readable and writable storage. The persistent storage may be a non-volatile memory device that does not lose stored instructions and data even after the computer is powered down. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the persistent storage may be a removable storage device (e.g., diskette, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as dynamic random access memory. The system memory may store instructions and data that are required by some or all of the processors at runtime. Furthermore, memory 410 may include any combination of computer-readable storage media including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks, and/or optical disks may also be employed. In some implementations, memory 410 may include readable and/or writable removable storage devices such as Compact Discs (CDs), digital versatile discs (e.g., DVD-ROMs, dual layer DVD-ROMs), blu-ray discs read only, super-density discs, flash memory cards (e.g., SD cards, min SD cards, micro-SD cards, etc.), magnetic floppy disks, and the like. The computer readable storage medium does not contain a carrier wave or an instantaneous electronic signal transmitted by wireless or wired transmission.
The memory 410 has stored thereon executable code that, when processed by the processor 420, can cause the processor 420 to perform some or all of the methods described above.
Furthermore, the method according to the present application may also be implemented as a computer program or computer program product comprising computer program code instructions for performing part or all of the steps of the above-described method of the present application.
Alternatively, the present application may also be embodied as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) having stored thereon executable code (or a computer program or computer instruction code) which, when executed by a processor of an electronic device (or a server, etc.), causes the processor to perform part or all of the steps of the above-described methods according to the present application.
The embodiments of the present application have been described above, the foregoing description is exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (8)
1. A method of training a neural network model for autopilot, comprising:
acquiring a preset neural network model; the main network of the preset neural network model at least comprises a first network layer and a second network layer, and the first network layer outputs hidden layer characteristic data to the second network layer; the preset neural network model can be a network model for processing computer vision tasks;
constructing an implicit data enhancement network structure for outputting implicit noise data;
accessing the implicit data enhancement network structure into the preset neural network model, so that the implicit noise data output by the implicit data enhancement network structure and hidden layer characteristic data in the preset neural network model are subjected to matrix addition; the implicit data enhancement network structure comprises: an input layer for receiving random noise data; the network structure layer at least comprises a convolution layer and is used for converting random noise data into recessive noise data; the output layer is used for outputting the recessive noise data; the recessive noise data is equal to the sum of random noise data multiplied by a noise expansion coefficient parameter and a data domain deviation parameter; wherein the noise expansion coefficient parameter and the data domain deviation parameter are network parameters of the implicit data enhancement network structure;
training the preset neural network model accessed with the implicit data enhancement network structure to obtain the preset neural network model accessed with the implicit data enhancement network structure after network parameter updating.
2. The method according to claim 1, characterized in that:
the implicit data enhancement network structure is configured to receive random noise data and to convert the random noise data into the implicit noise data corresponding to the implicit noise feature data.
3. The method according to claim 2, characterized in that:
the step of accessing the implicit data enhancement network structure to the preset neural network model so that the implicit noise data output by the implicit data enhancement network structure and the implicit layer characteristic data in the preset neural network model are subjected to matrix addition, comprising the following steps:
accessing the implicit data enhancement network structure into the preset neural network model, so that the input data of the second network layer are as follows: and the hidden layer characteristic data output by the first network layer and the hidden noise data output by the hidden data enhancement network structure are subjected to matrix addition.
4. A method according to claim 3, wherein said training the preset neural network model with the implicit data enhancement network structure, to obtain the preset neural network model with the implicit data enhancement network structure with updated network parameters, comprises:
inputting a preset training data set into the preset neural network model, inputting random noise data into the implicit data enhancement network structure, so that in the training process of the preset neural network model, the implicit noise data output by the first network layer and the implicit noise data output by the implicit data enhancement network structure are subjected to matrix addition, and are input into the second network layer;
updating the preset neural network model and the network parameters in the implicit data enhancement network structure through back propagation to obtain the preset neural network model with the implicit data enhancement network structure after network parameter updating.
5. The method according to claim 1, characterized in that:
the implicit noise data output by the implicit data enhancement network structure is the same as the implicit noise data in size dimension.
6. The method according to claim 1, characterized in that:
the network structure layer at least comprises a composite module and a deconvolution lamination layer connected with the composite module, wherein the composite module comprises a convolution layer, a batch normalization layer and an activation function layer which are sequentially connected.
7. A training device for an autopilot neural network model, comprising:
the acquisition module is used for acquiring a preset neural network model; the main network of the preset neural network model at least comprises a first network layer and a second network layer, and the first network layer outputs hidden layer characteristic data to the second network layer; the preset neural network model can be a network model for processing computer vision tasks;
the construction module is used for constructing an implicit data enhancement network structure for outputting the implicit noise data;
the access module is used for accessing the implicit data enhancement network structure constructed by the construction module into the preset neural network model acquired by the acquisition module, so that the implicit noise data output by the implicit data enhancement network structure and the implicit layer characteristic data in the preset neural network model are subjected to matrix addition; the implicit data enhancement network structure comprises: an input layer for receiving random noise data; the network structure layer at least comprises a convolution layer and is used for converting random noise data into recessive noise data; the output layer is used for outputting the recessive noise data; the recessive noise data is equal to the sum of random noise data multiplied by a noise expansion coefficient parameter and a data domain deviation parameter; wherein the noise expansion coefficient parameter and the data domain deviation parameter are network parameters of the implicit data enhancement network structure;
the training module is used for training the preset neural network model which is processed by the access module and is accessed with the implicit data enhancement network structure, and obtaining the preset neural network model which is updated in network parameters and is accessed with the implicit data enhancement network structure.
8. A neural network model, comprising:
the network parameter updated preset neural network model with the implicit data enhancement network structure obtained by the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111080765.7A CN113762482B (en) | 2021-09-15 | 2021-09-15 | Training method and related device for neural network model for automatic driving |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111080765.7A CN113762482B (en) | 2021-09-15 | 2021-09-15 | Training method and related device for neural network model for automatic driving |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113762482A CN113762482A (en) | 2021-12-07 |
| CN113762482B true CN113762482B (en) | 2024-04-16 |
Family
ID=78795758
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111080765.7A Active CN113762482B (en) | 2021-09-15 | 2021-09-15 | Training method and related device for neural network model for automatic driving |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113762482B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609634A (en) * | 2017-08-21 | 2018-01-19 | 哈尔滨工程大学 | A kind of convolutional neural networks training method based on the very fast study of enhancing |
| CN111209858A (en) * | 2020-01-06 | 2020-05-29 | 电子科技大学 | Real-time license plate detection method based on deep convolutional neural network |
| CN111669514A (en) * | 2020-06-08 | 2020-09-15 | 北京大学 | High dynamic range imaging method and apparatus |
| CN112434213A (en) * | 2020-10-15 | 2021-03-02 | 中国科学院深圳先进技术研究院 | Network model training method, information pushing method and related device |
| CN112990233A (en) * | 2021-04-15 | 2021-06-18 | 中国人民解放军国防科技大学 | Image classification method and device based on channel mixed sample data enhancement |
| CN113298050A (en) * | 2021-07-21 | 2021-08-24 | 智道网联科技(北京)有限公司 | Lane line recognition model training method and device and lane line recognition method and device |
| CN113326371A (en) * | 2021-04-30 | 2021-08-31 | 南京大学 | Event extraction method fusing pre-training language model and anti-noise interference remote monitoring information |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11138469B2 (en) * | 2019-01-15 | 2021-10-05 | Naver Corporation | Training and using a convolutional neural network for person re-identification |
-
2021
- 2021-09-15 CN CN202111080765.7A patent/CN113762482B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609634A (en) * | 2017-08-21 | 2018-01-19 | 哈尔滨工程大学 | A kind of convolutional neural networks training method based on the very fast study of enhancing |
| CN111209858A (en) * | 2020-01-06 | 2020-05-29 | 电子科技大学 | Real-time license plate detection method based on deep convolutional neural network |
| CN111669514A (en) * | 2020-06-08 | 2020-09-15 | 北京大学 | High dynamic range imaging method and apparatus |
| CN112434213A (en) * | 2020-10-15 | 2021-03-02 | 中国科学院深圳先进技术研究院 | Network model training method, information pushing method and related device |
| CN112990233A (en) * | 2021-04-15 | 2021-06-18 | 中国人民解放军国防科技大学 | Image classification method and device based on channel mixed sample data enhancement |
| CN113326371A (en) * | 2021-04-30 | 2021-08-31 | 南京大学 | Event extraction method fusing pre-training language model and anti-noise interference remote monitoring information |
| CN113298050A (en) * | 2021-07-21 | 2021-08-24 | 智道网联科技(北京)有限公司 | Lane line recognition model training method and device and lane line recognition method and device |
Non-Patent Citations (2)
| Title |
|---|
| Implicit Semantic Data Augmentation for Deep Networks;Yulin. Wang, et al.;NeurIPS, 2019.;第10卷(第28期);892-910 * |
| 基于生成式对抗网络的数据增强技术研究;华迪;中国优秀硕士学位论文全文数据库 信息科技辑;I138-1814 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113762482A (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111402257A (en) | Medical image automatic segmentation method based on multi-task collaborative cross-domain migration | |
| US20220092266A1 (en) | Method and device with natural language processing | |
| CN116721334B (en) | Training method, device, equipment and storage medium of image generation model | |
| CN114359073A (en) | Low-illumination image enhancement method, system, device and medium | |
| CN116152087A (en) | Unlimited countermeasure sample generation method, unlimited countermeasure sample generation device, electronic equipment and storage medium | |
| CN113205102B (en) | Vehicle mark identification method based on memristor neural network | |
| CN113762482B (en) | Training method and related device for neural network model for automatic driving | |
| CN111931553B (en) | Method, system, storage medium and application for enhancing generation of remote sensing data into countermeasure network | |
| CN117994570A (en) | Recognition method for improving complex and diverse data distribution based on model-independent adapter | |
| CN110458849B (en) | Image segmentation method based on feature correction | |
| CN111598841B (en) | Example significance detection method based on regularized dense connection feature pyramid | |
| CN112528062A (en) | Cross-modal weapon retrieval method and system | |
| CN116363368A (en) | Image semantic segmentation method and device based on convolutional neural network | |
| CN115358962B (en) | End-to-end visual odometer method and device | |
| CN111368969A (en) | Feature map processing method and device based on residual error neural network and storage medium | |
| CN110866866A (en) | Image color-matching processing method and device, electronic device and storage medium | |
| TWI803243B (en) | Method for expanding images, computer device and storage medium | |
| CN116091765A (en) | RGB-T image semantic segmentation method and device | |
| CN113807354B (en) | Image semantic segmentation method, device, equipment and storage medium | |
| CN116152263A (en) | CM-MLP network-based medical image segmentation method | |
| CN111400484B (en) | Keyword extraction method and system | |
| CN112927250A (en) | Edge detection system and method based on multi-granularity attention hierarchical network | |
| CN111524090A (en) | Depth prediction image-based RGB-D significance detection method | |
| CN117474797B (en) | Image denoising method and device for multi-scale complementary learning | |
| CN116167923B (en) | Sample expansion method and sample expansion device for x-ray image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |