CN109272105A - The construction method and processing unit of network system are passed before multinomial - Google Patents
The construction method and processing unit of network system are passed before multinomial Download PDFInfo
- Publication number
- CN109272105A CN109272105A CN201710588297.1A CN201710588297A CN109272105A CN 109272105 A CN109272105 A CN 109272105A CN 201710588297 A CN201710588297 A CN 201710588297A CN 109272105 A CN109272105 A CN 109272105A
- Authority
- CN
- China
- Prior art keywords
- layer
- layers
- neuron
- construction method
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
It is a kind of it is multinomial before pass the construction method of network, including: network of the building containing multilayer, first layer is input layer, and the last layer is output layer;Each layer is to preceding layer in addition to input layer;The unit of multilayer before at least one at least one layer of unit is connected in addition to input layer.By the way that the result of input layer to be transmitted in subsequent deeper layer, to guarantee to input the whole network that information can be transmitted efficiently, so that network more accurately reacts the feature of input data set, and then precision is improved.
Description
Technical field
This disclosure relates to computer field, further to artificial intelligence field.
Background technique
Neural network algorithm has become state-of-the-art algorithm in many different fields, such as field of image recognition, speech recognition
Field and natural language processing field.Network the most main is mainly deep learning among these.The training method of these networks
Substantially use back-propagation algorithm (Back-propagation, BP).Back-propagation algorithm, which is one, has the study of supervision to instruct
Practice method, error (error between network real output value and desired output) by gradient from output end backpropagation feed back into
End, so as to be modified accordingly according to the gradient of propagation.However now mainstream neural network in easily ten it is several layers of even
Layers up to a hundred, this make the value of the error back propagation of output end it is smaller and smaller (this be referred to as gradient disappearance, gradient
Vanish), the weight near input terminal not can be carried out efficient training substantially, so that the training of whole network is difficult efficiently
It quickly carries out, also the entire training of strong influence obtains the precision of network (error increases, accuracy degradation).
There are following technological deficiencies for the above-mentioned prior art, and in existing network, network path is fixed, Connecting quantity
And also fix, it is inefficient there is no solving the problems, such as to train due to gradient disappearance zone.
Disclosure
According to the one side of the disclosure, provide it is a kind of it is multinomial before pass the construction method of network, including:
The neural network containing multilayer is constructed, each layer includes neuron, and first layer is input layer, and the last layer is defeated
Layer out;
The neuron that each layer of neuron passes through the Synaptic junction containing weight to preceding layer in addition to input layer;
At least one at least one layer of neuron also passes through the Synaptic junction containing weight to the mind of preceding multilayer in addition to input layer
Through member.
In some embodiments, the neural network is BP deep-neural-network.
In some embodiments, each layer of the neuron in addition to input layer is connected to preceding layer and preceding multilayer is all
Neuron, the weighted value connected is consistent, or mutually indepedent, or grouping is independent.
In some embodiments, each layer of the neuron in addition to input layer be connected to all neurons of preceding layer with
And the partial nerve member of preceding multilayer, the weighted value connected are consistent, or mutually indepedent, or grouping is independent.
In some embodiments, a certain layer is set in addition to input layer as L layers, then L layers of neuron export NLMeet
The following conditions:
Wherein, SI, LIt is the weight of i-th layer and L layers of connection cynapse, FiIt is i-th layer and is connected to L layers of required meter
Calculate function, GLIt is then the calculating function of L layers of neuron output, NiFor the output of i-th layer of neuron.
In some embodiments, in L layers neuron calculating function GLFor y=x, i.e.,
In some embodiments, L layers of neuron export NLFormula in calculate function be Fi=ai×Ni×SI, L,
Middle aiCoefficient is passed before being i-th layer to L layers.
In some embodiments, L layers of neuron export NLFormula in calculate function be Fi=ai×Ni*SI, L,
Middle aiCoefficient is passed before being i-th layer to L layers, Ni*SI, LExpression carries out convolution algorithm between the two.
In some embodiments, the output N of L layers of neuronLMeet the following conditions:
In some embodiments, the output N of L layers of neuronLMeet the following conditions:
NL=GL(aL-2×NL-2+FL-1(NL-1, SL-1, L))
Wherein, the L-1 layers of calculating function for being connected to L layers are FL-1, L-2 layers be connected to L layers directly pass through before pass and be
Number aL-2It is superimposed with L-1 layers of calculated result.
It in some embodiments, further include that selection training is carried out to neural network.
In some embodiments, the selection training includes: that one or several samples of selection are trained;It will be selected
Sample input it is multinomial before pass network, determine the output of each sample, and corresponding error is determined according to desired output;By institute institute
It states the layer-by-layer backpropagation of error and returns input layer, and calculate corresponding weighted value knots modification, and network is updated according to weight knots modification
In cynapse weight;
Above step is repeated, the result of setting evaluation criteria is met until obtaining on training dataset.
In some embodiments, selecting one or several samples includes: that the number of samples selected every time is fixed or sample
This number changes, and the mode of variation includes but is not limited to random, increasing or decreasing.
In some embodiments, determine that corresponding error determines error by loss function according to desired output.
In some embodiments, the evaluation criteria is accuracy rate and/or overall error.
In some embodiments, the input of the input layer includes picture, video, audio and/text, the output layer
Output include classification results and/or generate result.
In some embodiments, the output of network is passed before multinomial, result presentation class result generates result.
According to another aspect of the present disclosure, provide it is a kind of it is multinomial before pass network system, including:
Overall network construction unit: for constructing the neural network containing multilayer, each layer includes neuron, first layer
For input layer, the last layer is output layer;
Basic connection unit: for making each layer in addition to input layer of neuron by the Synaptic junction containing weight to previous
The neuron of layer;
Unit is passed before multinomial: for making in addition to input layer at least one at least one layer of neuron also by prominent containing weight
The neuron of multilayer before touching is connected to.
In some embodiments, the neural network is BP neural network.
In some embodiments, each layer of the neuron in addition to input layer is connected to preceding layer and preceding multilayer is all
Neuron, the weighted value connected is consistent, or mutually indepedent, or grouping is independent.
In some embodiments, each layer of the neuron in addition to input layer be connected to all neurons of preceding layer with
And the partial nerve member of preceding multilayer, the weighted value connected are consistent, or mutually indepedent, or grouping is independent.
In some embodiments, a certain layer is set in addition to input layer as L layers, then the output N of L layers of neuronLIt is full
Sufficient the following conditions:
Wherein, SI, LIt is the weight of i-th layer and L layers of connection cynapse, FiIt is i-th layer and is connected to L layers required
Calculate function, GLIt is then the calculating function of L layers of neuron output, NiFor the output of i-th layer of neuron.
In some embodiments, in L layers neuron calculating function GLFor y=x, i.e.,
In some embodiments, L layers of neuron export NLFormula in calculate function be Fi=ai×Ni×SI, L,
Middle aiCoefficient is passed before being i-th layer to L layers.
In some embodiments, L layers of neuron export NLFormula in calculate function be Fi=ai×Ni*SI, L,
Middle aiCoefficient is passed before being i-th layer to L layers, Ni*SI, LExpression carries out convolution algorithm between the two.
In some embodiments, the output M of L layers of neuronLMeet the following conditions:
In some embodiments, the output N of L layers of neuronLMeet the following conditions:
NL=GL(aL-2×NL-2+FL-1(NL-1, SL-1, L))
Wherein, the L-1 layers of calculating function for being connected to L layers are FL-1, L-2 layers be connected to L layers directly pass through before pass and be
Number aL-2With L-1 layers of calculating folded structures.
It in some embodiments, further include training unit: for carrying out selection training to neural network.
In some embodiments, in the training unit, the selection training method includes:
One or several samples are selected to be trained;By selected sample input it is multinomial before pass network, determine each sample
This output, and corresponding error is determined according to desired output;Input layer is returned into institute's layer-by-layer backpropagation of error, and is calculated
Corresponding weighted value knots modification, and according to the weight of the cynapse in weight knots modification update network;Repeat above procedure, Zhi Dao
The result for meeting setting evaluation criteria is obtained on training dataset.
In some embodiments, in the training unit, selecting one or several samples includes: the sample selected every time
Number is fixed or number of samples changes, and the mode of variation includes but is not limited to random, increasing or decreasing.
In some embodiments, described to determine that corresponding error passes through damage according to desired output in the training unit
It loses function and determines error.
In some embodiments, the evaluation criteria is accuracy rate and/or overall error.
In some embodiments, in the overall network construction unit, the input for input layer includes picture, view
Frequently, audio and/text, the output of the output layer include classification results and/or generation result.
In some embodiments, in the overall network construction unit, for passing the output of network, result before multinomial
Presentation class result generates result
According to the another further aspect of the disclosure, a kind of processing unit is provided, comprising:
Storage unit, for storing executable instruction;
Processor carries out method described in any of the above when executing instruction for executing the executable instruction.
According to the another aspect of the disclosure, a kind of electronic device is provided, the electronic device includes process described above
Device, the electronic device include data processing equipment, robot, computer, printer, scanner, tablet computer, intelligently end
End, mobile phone, automobile data recorder, navigator, sensor, camera, cloud server, camera, video camera, projector, wrist-watch, ear
Machine, mobile storage, the wearable device vehicles, household electrical appliance, and/or Medical Devices.
Through the above technical solutions, knowing to pass network system before the disclosure is multinomial and its construction method, processing system are beneficial
Effect is:
(1) network is passed before multinomial, by introducing the input of front layer in current layer, so that the information of front layer be transmitted
To current layer, so that subsequent deeper layer may learn the information of input, to be easy to construct deeper neural network.
(2) by the way that the result of input layer to be transmitted in subsequent deeper layer, to guarantee that input information can be efficient
The whole network of transmitting so that network more accurately reacts the feature of input data set, and then improves precision.
(3) in training process, the preceding presence for passing connection can be reversed the error of output end to be directly propagated back to input layer,
To guarantee that error can be by the utilization of precise and high efficiency, and then modify synaptic weight.
Detailed description of the invention
Fig. 1 is the building flow chart that network is passed before the embodiment of the present disclosure is multinomial.
Fig. 2 is the building flow chart that network is passed before another embodiment of the disclosure is multinomial.
Fig. 3 is the flow chart that network is passed before the training of embodiment of the present disclosure offer is multinomial.
Fig. 4 be the embodiment of the present disclosure provide it is multinomial before pass the module diagram of network system.
Fig. 5 be the embodiment of the present disclosure provide it is multinomial before pass the topological structure schematic diagram of network.
Specific embodiment
For the purposes, technical schemes and advantages of the disclosure are more clearly understood, below in conjunction with specific embodiment, and reference
Attached drawing is described in further detail the disclosure.
According in conjunction with attached drawing to the described in detail below of disclosure exemplary embodiment, other aspects, the advantage of the disclosure
Those skilled in the art will become obvious with prominent features.
In the disclosure, term " includes " and " containing " and its derivative mean including rather than limit.
In present disclosure, following various embodiments for describing disclosure principle only illustrate, should not be with any
Mode is construed to limitation scope of disclosure.Referring to attached drawing the comprehensive understanding described below that is used to help by claim and its equivalent
The exemplary embodiment for the disclosure that object limits.Described below includes a variety of details to help to understand, but these details are answered
Think to be only exemplary.Therefore, it will be appreciated by those of ordinary skill in the art that not departing from the scope of the present disclosure and spirit
In the case where, embodiment described herein can be made various changes and modifications.In addition, for clarity and brevity,
The description of known function and structure is omitted.In addition, running through attached drawing, same reference numerals are used for the same or similar function and behaviour
Make.In addition, those skilled in the art should although the scheme with different characteristic may be described in different embodiments
Recognize: all or part of feature of different embodiments can be combined, does not depart from spirit and scope of the present disclosure to be formed
New embodiment.
According to the one side of the embodiment of the present disclosure, provide it is a kind of it is multinomial before pass the construction method of network.Fig. 1 is disclosure reality
Apply example it is multinomial before pass the building flow chart of network, according to Fig. 1, the construction method of the embodiment of the present disclosure includes:
S101: neural network of the building containing multilayer, each layer includes neuron, and first layer is input layer, last
Layer is output layer;
S102: the neuron that each layer of neuron passes through the Synaptic junction containing weight to preceding layer in addition to input layer;
S103: at least one at least one layer of neuron also passes through the Synaptic junction containing weight to preceding multilayer in addition to input layer
Neuron.
Wherein, by least one neuron at least one layer of in addition to input layer also by the Synaptic junction containing weight to preceding
The neuron of multilayer.I.e. by introducing the input of front layer in current layer, so that the information of front layer is transmitted to current layer,
So that subsequent deeper layer may learn the information of input, to be easy to construct deeper neural network.
In some embodiments, the neural network is BP deep-neural-network.By the way that the result of input layer is transmitted to
In subsequent deeper layer, to guarantee to input the whole network that information can be transmitted efficiently, so that network is more accurate
Reaction input data set feature, and then improve precision.
In some embodiments, each layer of the neuron in addition to input layer is connected to preceding layer and preceding multilayer is all
Neuron, the weighted value connected are consistent, or mutually indepedent, or grouping is independent.The organizational composition of refreshing grade network includes
But it is not limited to full articulamentum, convolutional layer, down-sampled layer, normalization layer, circulation layer, residual error layer, Normalization layers of Batch
Or shot and long term layer.
In some embodiments, each layer of the neuron in addition to input layer be connected to preceding layer all neurons and
The partial nerve member of preceding multilayer, the weighted value connected are consistent, or mutually indepedent, or grouping is independent.
In some embodiments, a certain layer is set in addition to input layer as L layers, then the output N of L layers of neuronLMeet
The following conditions:
Wherein, SI, LIt is the weight of i-th layer and L layers of connection cynapse, FiIt is i-th layer and is connected to L layers required
Calculate function, GLIt is then the calculating function of L layers of neuron output, NiFor the output of i-th layer of neuron.It can be seen that L layers
Exporting has multinomial composition to the output that the 1st layer to L-1 layers of neuron exports all related namely each layer, therefore is referred to as more
Network is passed before.
For above-mentioned formula, the calculating function G of neuron in L layersLFor y=x.
Preferably, L layers of neuron export NLFormula in calculate function be Fi=ai×Ni×SI, L, wherein aiIt is i-th
Layer is to passing coefficient before L layers.
Preferably, L layers of neuron export NLFormula in calculate function be Fi=ai×Ni*SI, L, wherein aiIt is i-th
Layer is to passing coefficient before L layers, Ni*SI, LExpression carries out convolution algorithm between the two.
Preferably, the output N of L layers of neuronLMeet the following conditions:
Preferably, the output N of L layers of neuronLMeet the following conditions:
NL=GL(aL-2×NL-2+FL-1(NL-1, SL-1, L))
Wherein, the L-1 layers of calculating function for being connected to L layers are FL-1, L-2 layers be connected to L layers directly pass through before pass and be
Number aL-2With L-1 layers of calculating folded structures.
It is exemplified below specific example to be explained, it will be appreciated that these examples are only implemented with the explanation disclosure
Example, is not intended to limit the disclosure.
Embodiment 1
NL=a0×N0+a1×N1+…aL-1×NL-1
Wherein a0Be it is multinomial before the coefficient passed, remaining is similar.In this embodiment, every layer of neuron is input to the meter of output
Calculate function GLFor the simple mapping of y=x;Every layer of output of all layers of front is multiplied by one coefficient, and by all result of product
It is cumulative to obtain L layers of output.In the present embodiment, each layer will be delivered in subsequent deeper layer before result, the method for coupling
It is to be input to output in different layers multiplied by different coefficients.Front can be obtained for every layer in network corresponding to the embodiment
All layers of information, and every layer of output can be transferred to subsequent all layers, this makes each layer of processing in network and obtains
Obtaining information can retain and transmit.Especially in training process, the error of backpropagation can be efficiently reflected into input
Layer, to guarantee that the training of network is efficient.
Embodiment 2
NL=GL(a0×N0×S0, L+a1×N1×S1, L+…aL-1×NL-1×SL-1, L)
Wherein a0Be it is multinomial before the coefficient passed, remaining is similar;S0, LIt is the 0th layer to L layers of coefficient matrix, remaining is similar.
In this embodiment, be delivered to before one layer its in subsequent each layer by one it is independent before pass coefficient and an independent connection square
Battle array, the matrix and output complete matrix contraposition and multiply calculating.All layers in front can be obtained for every layer in the corresponding network of the embodiment
Information, and every layer of output can be transferred to subsequent all layers, and contain further to be screened among these
The weight coefficient of weighting, this allows the reservation for different being screened property of layer information in network.In training process, instead
It can be efficiently reflected into input layer to the error of propagation, to guarantee that the training of network is efficient.
Embodiment 3
NL=GL(a0×N0*S0, L+a1×N1*S1, L+…aL-1×NL-1*SL-1, L)
Wherein a0Be it is multinomial before the coefficient passed, remaining is similar;S0, LIt is the 0th layer to L layers of coefficient matrix, remaining is similar.
In this embodiment, be delivered to before one layer its in subsequent each layer by one it is independent before pass coefficient and an independent connection square
Convolutional calculation is completed in battle array, the matrix and output.Every layer corresponding of network can obtain the letter of all layers of front in the embodiment
Breath, and every layer of output can be transferred to subsequent all layers, information pass through convolution operation completion namely before pass letter
Breath carries out screening specific component by the convolution kernel that can train, this allows every layer of being screened property of information in network
Retain.It can be efficiently reflected into input layer in the error of training process, backpropagation, to guarantee that the training of network is high
Effect.
Embodiment 4
4, NL=GL(aL-2×NL-2×SL-2, L+aL-1×NL-1×SL-1, L)
Wherein each variable meaning is as hereinbefore.In this embodiment, the L layers of neuron for only receiving L-1 layers He L layers
Output.L layers and L-1 layers are inputed to that is, passing before L-2 layers.Wherein L-1 layers of, this embodiment consistent with the transmitting of L-2 processing calculating
In be matrix multiplication.Every layer corresponding of network can obtain the information of front fixed quantity layer in the embodiment, and every layer
Output can be transferred to the layer of subsequent fixed quantity, this allows the reservation of every layer of being screened property of information in network,
The layer information level that may be present that hypertelorism can be excluded in certain application scenarios mismatches.In training process,
The error of backpropagation can be efficiently reflected into input layer, to guarantee that the training of network is efficient.
Embodiment 5
5, NL=GL(aL-2×NL-2+FL-1(NL-1, SL-1, L)
Wherein each variable meaning is as hereinbefore.In this embodiment, the L layers of neuron for only receiving L-1 layers He L layers
Output.L layers and L-1 layers are inputed to that is, passing before L-2 layers.Further, wherein L-1 layers and L-2 layers of processing it is inconsistent, wherein
The operation of L-1 is equal to common neural network operation, including but not limited to convolution, will sampling, full connection;Wherein L-2 layers
As a result it is directly superimposed multiplied by a coefficient and L-1 layers of calculated result.Every layer corresponding of network can obtain in the embodiment
The information of front fixed quantity layer, and every layer of output can be transferred to the layer of subsequent fixed quantity, this makes every in network
Layer information being screened property reservation, the layer letter that may be present of hypertelorism can be excluded in certain application scenarios
Abstraction hierarchy is ceased to mismatch.And the operation of information transmitting then can be customized according to different application scenarios, such as image class application
In can be using convolution etc..It can be efficiently reflected into input layer in the error of training process, backpropagation, to guarantee net
The training of network is efficient.
It in some embodiments, further include that selection training is carried out to neural network.Fig. 2 is that another embodiment of the disclosure is multinomial
Before pass the building flow chart of network.As shown in Fig. 2, S201-S203 is corresponding with S101-S103 outer, it further include step S204, to mind
It is trained through network.Wherein the selection training method includes: that one or several samples of selection are trained;It will be selected
Network is passed before sample input is multinomial, determines the output of each sample, and corresponding error is determined according to desired output;It will be described in institute
Input layer is returned in the layer-by-layer backpropagation of error, and calculates corresponding weighted value knots modification, and is updated in network according to weight knots modification
Cynapse weight;And above step is repeated, the result of setting evaluation criteria is met until obtaining on training dataset.
In some embodiments, selecting one or several samples includes: that the number of samples selected every time is fixed or sample
Number changes, and the mode of variation includes but is not limited to random, increasing or decreasing.
Wherein, it can determine that corresponding error determines error by loss function according to desired output.
In some embodiments, the evaluation criteria is accuracy rate and/or overall error.
Fig. 3 be the embodiment of the present disclosure provide it is multinomial before pass the topological structure schematic diagram of network.Referring to shown in Fig. 3, first
Sample set is selected, then calculates positive as a result, medium calculating back-propagation gradient and right value update amount, update according to renewal amount
Judge whether to meet training requirement after weight, restarts to select sample set if being unsatisfactory for, meeting training requirement then terminates
Training.
It is a kind of it is specific it is multinomial before pass the training method example of network and can be divided into following steps:
Firstly, one or several samples that selection training data is concentrated, the selection mode of sample is including but not limited to random,
Order of packets.The number of samples selected every time can be fixed, and can also change, the mode of variation include but is not limited to
Machine is incremented by, successively decreases or according to certain mathematical programming.
Secondly, by the input of selected sample set it is multinomial before pass network, be calculated the output of each sample, and according to
Desired output calculates corresponding error, and the mode of error evaluation includes but is not limited to different loss function.
The layer-by-layer backpropagation of error calculated is returned input layer, and calculates corresponding weight knots modification by third, and according to
Weight knots modification updates the weight of the cynapse in network.
4th, repeat above procedure, until on training dataset obtain meet specific evaluation criteria as a result, the assessment
Standard can be but not limited to recognition accuracy, overall error.
In step s101, the input of the input layer may include picture, video, audio and/text, the output layer
Output include classification results and/or generate result.
Wherein, the output of network is passed before multinomial, result presentation class result generates result.
As shown in figure 4, according to the another aspect of the embodiment of the present disclosure, provide it is a kind of it is multinomial before pass the building system of network,
Including:
Overall network construction unit 401: for constructing the neural network containing multilayer, each layer includes neuron, the
One layer is input layer, and the last layer is output layer;
Basic connection unit 402: for make each layer in addition to input layer of neuron by the Synaptic junction containing weight extremely
The neuron of preceding layer;
Unit 403 is passed before multinomial: for making in addition to input layer at least one at least one layer of neuron also by containing weight
Synaptic junction to preceding multilayer neuron.
In some embodiments, the neural network is BP deep-neural-network.By the way that the result of input layer is transmitted to
In subsequent deeper layer, to guarantee to input the whole network that information can be transmitted efficiently, so that network is more accurate
Reaction input data set feature, and then improve precision.It is shown in Figure 5, be the embodiment of the present disclosure provide it is multinomial before
Pass the topological structure schematic diagram of network, such as input layer N1Result be transmitted to subsequent deeper layer NLIn, to guarantee input letter
The whole network that can efficiently transmit is ceased, so that network more accurately reacts the feature of input data set, Jin Erti
In high precision
In some embodiments, each layer of the neuron in addition to input layer is connected to preceding layer and preceding multilayer is all
Neuron, the weighted value connected are consistent, or mutually indepedent, or grouping is independent.
In some embodiments, each layer of the neuron in addition to input layer be connected to preceding layer all neurons and
The partial nerve member of preceding multilayer, the weighted value connected are consistent, or mutually indepedent, or grouping is independent.
In some embodiments, a certain layer is set in addition to input layer as L layers, then the output N of L layers of neuronLMeet
The following conditions:
Wherein, SI, LIt is the weight of i-th layer and L layers of connection cynapse, FiIt is i-th layer and is connected to L layers required
Calculate function, GLIt is then the calculating function of L layers of neuron output, NiFor the output of i-th layer of neuron.
Function F is calculated in some embodimentsICan be arbitrary form, for example, the contraposition of direct neural member is added, weight and
Neuron is multiplied, weight and neuron carry out convolution operation etc..
For above-mentioned, the calculating function G of neuron in L layersLIt is mapping for y=x.
Preferably, L layers of neuron export NLFormula in calculate function be Fi=ai×Ni×SI, L, wherein aiIt is i-th
Layer is to passing coefficient before L layers.
Preferably, L layers of neuron export NLFormula in calculate function be Fi=ai×Ni*SI, L, wherein aiIt is i-th
Layer is to passing coefficient before L layers, Ni*SI, LExpression carries out convolution algorithm between the two.
Preferably, the output N of L layers of neuronLMeet the following conditions:
Preferably, the output N of L layers of neuronLMeet the following conditions:
NL=GL(aL-2×NL-2+FL-1(NL-1, SL-1, L)
Wherein, the L-1 layers of calculating function for being connected to L layers are FL-1, L-2 layers be connected to L layers directly pass through before pass and be
Number aL-2With L-1 layers of calculating folded structures.
It in some embodiments, further include that selection training is carried out to neural network.Wherein the selection training method includes:
One or several samples are selected to be trained;By selected sample input it is multinomial before pass network, determine the output of each sample,
And corresponding error is determined according to desired output;Input layer is returned into institute's layer-by-layer backpropagation of error, and calculates corresponding power
Weight values knots modification, and according to the weight of the cynapse in weight knots modification update network;And above step is repeated, until in training
The result for meeting setting evaluation criteria is obtained on data set.
In some embodiments, selecting one or several samples includes: that the number of samples selected every time is fixed or sample
Number changes, and the mode of variation includes but is not limited to random, increasing or decreasing.
Wherein, it can determine that corresponding error determines error by loss function according to desired output.
In some embodiments, the evaluation criteria is accuracy rate and/or overall error.
According to the one side of the disclosure, a kind of processing unit is provided, comprising: storage unit, for storing executable instruction;
And processor carries out passing network before any one of the above is multinomial when executing instruction for executing the executable instruction
Construction method.
Wherein, processor can be single processing unit, but also may include two or more processing units.In addition,
Processor can also include general processor (CPU) or graphics processor (GPU);Field programmable logic can also be included in
Gate array (FPGA) or specific integrated circuit (ASIC), to be configured to neural network and operation.Processor can also wrap
Include the on-chip memory (including the memory in processing unit) for caching purposes.
In some embodiments, a kind of chip is disclosed comprising above-mentioned processing unit.
In some embodiments, a kind of chip-packaging structure is disclosed comprising said chip.
In some embodiments, a kind of board is disclosed comprising said chip encapsulating structure.
In some embodiments, a kind of electronic device is disclosed comprising above-mentioned board.
Electronic device include data processing equipment, robot, computer, printer, scanner, tablet computer, intelligent terminal,
Mobile phone, automobile data recorder, navigator, sensor, camera, cloud server, camera, video camera, projector, wrist-watch, earphone,
Mobile storage, wearable device, the vehicles, household electrical appliance, and/or Medical Devices.
The vehicles include aircraft, steamer and/or vehicle;The household electrical appliance include TV, air-conditioning, micro-wave oven,
Refrigerator, electric cooker, humidifier, washing machine, electric light, gas-cooker, kitchen ventilator;The Medical Devices include Nuclear Magnetic Resonance, B ultrasound instrument
And/or electrocardiograph.
It should be appreciated that disclosed relevant apparatus and method, may be implemented in other ways.For example, the above institute
The Installation practice of description is only schematical, for example, the division of the module or unit, only a kind of logic function is drawn
Point, there may be another division manner in actual implementation, such as multiple units or components may be combined or can be integrated into separately
One system, or some features can be ignored or not executed.
By embodiment of the disclosure, provide it is multinomial before pass network system and its construction method, processing unit, Yi Jixin
Piece, chip-packaging structure, board and electronic device.Wherein, by introducing the input of front layer in current layer, thus by front
The information of layer is transmitted to current layer, so that subsequent deeper layer may learn the information of input, to be easy to construct deeper
Secondary neural network.
Particular embodiments described above has carried out further in detail the purpose of the disclosure, technical scheme and beneficial effects
Describe in detail bright, it should be understood that the foregoing is merely the specific embodiment of the disclosure, be not limited to the disclosure, it is all
Within the spirit and principle of the disclosure, any modification, equivalent substitution, improvement and etc. done should be included in the protection of the disclosure
Within the scope of.
Claims (19)
- The construction method of network is passed before 1. one kind is multinomial, including:The neural network containing multilayer is constructed, each layer includes neuron, and first layer is input layer, and the last layer is output Layer;The neuron that each layer of neuron passes through the Synaptic junction containing weight to preceding layer in addition to input layer;At least one at least one layer of neuron also passes through the Synaptic junction containing weight to the neuron of preceding multilayer in addition to input layer.
- 2. construction method according to claim 1, which is characterized in that the neural network is BP deep-neural-network.
- 3. construction method according to claim 1, which is characterized in that each layer of the neuron in addition to input layer is connected to Preceding layer and all neuron of preceding multilayer, the weighted value connected are consistent, or mutually indepedent, or grouping is independent.
- 4. construction method according to claim 1, which is characterized in that each layer of the neuron in addition to input layer is connected to All neurons of preceding layer and the partial nerve member of preceding multilayer, the weighted value connected are consistent, or mutually indepedent, Or grouping is independent.
- 5. construction method according to claim 1, which is characterized in that set in addition to input layer a certain layer as L layers, then L layers of neuron export NLMeet the following conditions:Wherein, SI, LIt is the weight of i-th layer and L layers of connection cynapse, FiIt is i-th layer and is connected to L layers of required calculating letter Number, GLIt is then the calculating function of L layers of neuron output, NiFor the output of i-th layer of neuron.
- 6. construction method according to claim 5, which is characterized in that the calculating function G of neuron in L layersLFor y=x, I.e.
- 7. construction method according to claim 5, which is characterized in that L layers of neuron export NLFormula in calculate function For Fi=ai×Ni×SI, L, wherein aiCoefficient is passed before being i-th layer to L layers.
- 8. construction method according to claim 5, which is characterized in that L layers of neuron export NLFormula in calculate function For Fi=ai×Ni*SI, L, wherein aiCoefficient is passed before being i-th layer to L layers, Ni*SI, LExpression carries out convolution algorithm between the two.
- 9. construction method according to claim 5, which is characterized in that the output N of L layers of neuronLMeet the following conditions:
- 10. construction method according to claim 9, which is characterized in that the output N of L layers of neuronLMeet the following conditions:NL=GL(aL-2×NL-2+FL-1(NL-1, SL-1, L))Wherein, the L-1 layers of calculating function for being connected to L layers are FL-1, L-2 layers be connected to L layers directly pass through before pass coefficient aL-2It is superimposed with L-1 layers of calculated result.
- 11. construction method according to claim 1, which is characterized in that further include carrying out selection training to neural network.
- 12. construction method according to claim 11, which is characterized in that the selection training includes:One or several samples are selected to be trained;By the input of selected sample it is multinomial before pass network, determine the output of each sample, and determine according to desired output corresponding Error;Input layer is returned into institute's layer-by-layer backpropagation of error, and calculates corresponding weighted value knots modification, and change according to weight Amount updates the weight of the cynapse in network;Above step is repeated, the result of setting evaluation criteria is met until obtaining on training dataset.
- 13. construction method according to claim 12, which is characterized in that selecting one or several samples includes: each choosing The number of samples selected is fixed or number of samples changes, and the mode of variation includes but is not limited to random, increasing or decreasing.
- 14. construction method according to claim 12, which is characterized in that determine that corresponding error passes through according to desired output Loss function determines error.
- 15. construction method according to claim 12, which is characterized in that the evaluation criteria is accuracy rate and/or always misses Difference.
- 16. construction method according to claim 1, which is characterized in that the input of the input layer include picture, video, Audio and/text, the output of the output layer include classification results and/or generation result.
- 17. construction method according to claim 1, which is characterized in that pass the output of network before multinomial, result indicates to divide Class result generates result.
- 18. a kind of processing unit, characterized by comprising:Storage unit, for storing executable instruction;Processor carries out any method of claim 1-17 when executing instruction for executing the executable instruction.
- 19. a kind of electronic device, which is characterized in that the electronic device includes processing unit described in claim 18, described Electronic device includes data processing equipment, robot, computer, printer, scanner, tablet computer, intelligent terminal, mobile phone, row Vehicle recorder, navigator, sensor, camera, cloud server, camera, video camera, projector, wrist-watch, earphone, movement are deposited Storage, the wearable device vehicles, household electrical appliance, and/or Medical Devices;The vehicles include aircraft, steamer and/or vehicle;The household electrical appliance include TV, air-conditioning, micro-wave oven, refrigerator, Electric cooker, humidifier, washing machine, electric light, gas-cooker and/or kitchen ventilator;The Medical Devices include Nuclear Magnetic Resonance, B ultrasound instrument And/or electrocardiograph.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710588297.1A CN109272105A (en) | 2017-07-18 | 2017-07-18 | The construction method and processing unit of network system are passed before multinomial |
CN201710620775.2A CN109272106A (en) | 2017-07-18 | 2017-07-18 | Network system is passed before multinomial |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710588297.1A CN109272105A (en) | 2017-07-18 | 2017-07-18 | The construction method and processing unit of network system are passed before multinomial |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710620775.2A Division CN109272106A (en) | 2017-07-18 | 2017-07-18 | Network system is passed before multinomial |
Publications (1)
Publication Number | Publication Date |
---|---|
CN109272105A true CN109272105A (en) | 2019-01-25 |
Family
ID=65148110
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710588297.1A Pending CN109272105A (en) | 2017-07-18 | 2017-07-18 | The construction method and processing unit of network system are passed before multinomial |
CN201710620775.2A Pending CN109272106A (en) | 2017-07-18 | 2017-07-18 | Network system is passed before multinomial |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710620775.2A Pending CN109272106A (en) | 2017-07-18 | 2017-07-18 | Network system is passed before multinomial |
Country Status (1)
Country | Link |
---|---|
CN (2) | CN109272105A (en) |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101019123A (en) * | 2004-02-06 | 2007-08-15 | 科学与工业研究委员会 | Computational method for identifying adhesin and adhesin-like proteins of therapeutic potential |
CN101814158A (en) * | 2009-02-20 | 2010-08-25 | 北京联合大学生物化学工程学院 | Method for analyzing and processing experimental data based on artificial neural network |
JP6521440B2 (en) * | 2015-05-14 | 2019-05-29 | 国立研究開発法人情報通信研究機構 | Neural network and computer program therefor |
-
2017
- 2017-07-18 CN CN201710588297.1A patent/CN109272105A/en active Pending
- 2017-07-18 CN CN201710620775.2A patent/CN109272106A/en active Pending
Also Published As
Publication number | Publication date |
---|---|
CN109272106A (en) | 2019-01-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022077587A1 (en) | Data prediction method and apparatus, and terminal device | |
He et al. | Robust matrix completion via maximum correntropy criterion and half-quadratic optimization | |
CN108510485A (en) | It is a kind of based on convolutional neural networks without reference image method for evaluating quality | |
CN109034365A (en) | The training method and device of deep learning model | |
CN108009594B (en) | A kind of image-recognizing method based on change grouping convolution | |
CN109426553A (en) | Task cutting device and method, Task Processing Unit and method, multi-core processor | |
CN110445126A (en) | A kind of non-intrusion type load decomposition method and system | |
Durmuş et al. | Parameter identification using particle swarm optimization | |
US20190138929A1 (en) | System and method for automatic building of learning machines using learning machines | |
CN113610989B (en) | Method and device for training style migration model and method and device for style migration | |
CN108229536A (en) | Optimization method, device and the terminal device of classification prediction model | |
CN116707331A (en) | Inverter output voltage high-precision adjusting method and system based on model prediction | |
Xu et al. | Distributed no-regret learning in multiagent systems: Challenges and recent developments | |
Nguyen et al. | Fast transient simulation of high-speed channels using recurrent neural network | |
CN113971090B (en) | Layered federal learning method and device of distributed deep neural network | |
CN110007371A (en) | Wind speed forecasting method and device | |
CN109359542A (en) | The determination method and terminal device of vehicle damage rank neural network based | |
CN108509179B (en) | Method for detecting human face and device for generating model | |
CN110146855A (en) | Radar Intermittent AF panel thresholding calculation method and device | |
CN109190749A (en) | A kind of prediction technique and device for the intelligent electric meter service life | |
CN117113087A (en) | Model training method, device, electronic equipment and storage medium | |
CN111210051B (en) | User electricity consumption behavior prediction method and system | |
Urbán et al. | Modelling force-free neutron star magnetospheres using physics-informed neural networks | |
CN109272105A (en) | The construction method and processing unit of network system are passed before multinomial | |
CN109493065A (en) | A kind of fraudulent trading detection method of Behavior-based control incremental update |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190125 |