CN108763360A - A kind of sorting technique and device, computer readable storage medium - Google Patents
A kind of sorting technique and device, computer readable storage medium Download PDFInfo
- Publication number
- CN108763360A CN108763360A CN201810469963.4A CN201810469963A CN108763360A CN 108763360 A CN108763360 A CN 108763360A CN 201810469963 A CN201810469963 A CN 201810469963A CN 108763360 A CN108763360 A CN 108763360A
- Authority
- CN
- China
- Prior art keywords
- layer
- neuron
- parallel
- bias
- weight
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
This application discloses a kind of sorting technique and device, computer readable storage medium, the method includes：Characteristic is inputted to the depth confidence neural network model pre-established；By multi-threaded parallel Gibbs model, according to the probability being mutually activated between sampling results parallel computation hidden layer and aobvious layer, the weight and bias between each neuron are updated parallel；Using between updated each neuron weight and bias as initialization training parameter, to depth confidence neural network model carry out Training；Classification and Identification is carried out using trained depth confidence neural network model.The application is by multi-threaded parallel Gibbs model and updates weight and bias between each neuron parallel, has played the parallel advantage of algorithm, has improved the arithmetic speed of system.
Description
Technical field
The present invention relates to field of computer technology, and in particular to a kind of sorting technique and device, computer-readable storage medium
Matter.
Background technology
With the continuous development of every profession and trade data increased rapidly with database, data analysis technique, for finding in advance
The machine learning and data mining technology of unknown rule and contact have been used to market analysis, industrial production, finance, science are ground
Study carefully, Web information analysis and the fields such as Engineering Diagnosis, and obtains better effects.As main data analysis pattern, classification is calculated
Method is primarily adapted for use in prediction classification designator or centrifugal pump, belongs to monitoring learning problem, and usually first training (is trained with training data
Model generates model parameter) to classify (generate classification results with test data) again, common sorting algorithm includes decision tree, shellfish
Ye Si, support vector machines, neural network etc..Deep learning based on primary data analysis and can find the distributed nature tables of data
Show, advantage is to be substituted with feature learning and layered characteristic extraction highly effective algorithm and obtained feature by hand, is caused in recent years more extensively
Concern.
But existing data mining algorithm is applied to the processing speed and input/output (Input/ of mass data
Output, I/O) bottleneck always exists limitation, it can improve these problems using concurrent operation.Graphics processor (Graphics
Processing Unit, GPU) as a kind of important parallel accelerated mode, just it is widely used in deep learning.
CUDA (Compute Unified Device Architecture) and OpenMP (Open Multi-Processing) by with
In the text detection system based on neural network, multiple data flows can be handled simultaneously under single instruction stream, it is empty to improve GPU storages
Between utilization rate, limitation is that data parallel mode does not play the parallel advantage of algorithm.Domestic and international Internet company is depth
Various GPU clusters have been built in the research of habit, the training of neural network model can be carried out on more GPU servers, but equipment is advised
Mould is larger, and cost is higher.
Invention content
In order to solve the above technical problem, the present invention provides a kind of sorting technique and device, computer-readable storage mediums
Matter can improve the arithmetic speed of system.
In order to reach the object of the invention, what the technical solution of the embodiment of the present invention was realized in：
An embodiment of the present invention provides a kind of sorting techniques, including：
Characteristic is inputted to the depth confidence neural network model pre-established；
By multi-threaded parallel Gibbs model, mutually it is activated according between sampling results parallel computation hidden layer and aobvious layer
Probability, update the weight and bias between each neuron parallel；
Using between updated each neuron weight and bias as initialization training parameter, to depth confidence
Neural network model carries out Training；
Classification and Identification is carried out using trained depth confidence neural network model.
Further, described by multi-threaded parallel Gibbs model, according to sampling results parallel computation hidden layer and aobvious layer
Between the probability that is mutually activated, update the weight and bias between each neuron parallel, including：
Parallel Gibbs model, while the value for showing each neuron on layer is extracted, each hidden neuron is calculated by aobvious layer
The probability value of neuronal activation：
Wherein i ∈ [0, M], j ∈ [0, N], v are aobvious layer data value, and h is hidden layer data value, and M is aobvious layer neuron number, N
For hidden neuron number；W is weight, and b is aobvious layer bias, and c is hidden layer bias；
Pass through hidden layer h(0)Reconstruct shows layer v(1), parallel Gibbs model, calculates and shows layer neuron by hidden neuron again
The probability value of activation：
By showing layer v(1)Reconstruct hidden layer h(1), parallel Gibbs model, calculates hidden neuron by aobvious layer neuron again
The probability value of activation：
The weight and bias between each neuron are updated parallel：
Wherein, η is preset learning rate.
Further, the weight and bias using between updated each neuron is as initialization training ginseng
Number carries out Training to depth confidence neural network model, including：
According to back-propagation algorithm, prison has been carried out to the depth B P neural networks in the depth confidence neural network model
Superintend and direct tuning training；
The weight and bias that each layer is updated using gradient descent algorithm iteration, until the weight and bias of each layer reach
It optimizes.
Further, further include before the method：
The data of acquisition are obtained characteristic by the data for obtaining sensor acquisition by signal processing.
Sorting technique according to claim 4, which is characterized in that the data of described pair of acquisition are obtained by signal processing
To characteristic, including：
By the data of acquisition, chronologically adding window is divided into multiple data blocks；
Concurrent Feature extraction, and the feature that each thread is extracted are carried out to the multiple data blocks marked off by multithreading
Category merges, the characteristic after being merged.
Further, when executing the multithreading by multiple multiprocessors (MP), multithreading inside identical MP it
Between by shared drive into row data communication；By global memory into row data communication between different MP.
The embodiment of the present invention additionally provides a kind of computer readable storage medium, which is characterized in that described computer-readable
One or more program is stored on storage medium, one or more of programs can be held by one or more processor
Row, the step of to realize sorting technique as described in any of the above item.
The embodiment of the present invention additionally provides a kind of sorter, including input unit, training unit and taxon,
In：
Input unit, for characteristic to be inputted the depth confidence neural network model pre-established；
Training unit, for passing through multi-threaded parallel Gibbs model, according to sampling results parallel computation hidden layer and aobvious layer
Between the probability that is mutually activated, update the weight and bias between each neuron parallel；By updated each god
Through the weight and bias conduct initialization training parameter between member, supervision instruction has been carried out to depth confidence neural network model
Practice；
Taxon, for being classified using trained depth confidence neural network model.
Further, the training unit by multi-threaded parallel Gibbs model, according to sampling results parallel computation
The probability being mutually activated between hidden layer and aobvious layer updates weight and bias between each neuron parallel, including：
Parallel Gibbs model, while the value for showing each neuron on layer is extracted, each hidden neuron is calculated by aobvious layer
The probability value of neuronal activation：
Wherein i ∈ [0, M], j ∈ [0, N], v are aobvious layer data value, and h is hidden layer data value, and M is aobvious layer neuron number, N
For hidden neuron number；W is weight, and b is aobvious layer bias, and c is hidden layer bias；
Pass through hidden layer h(0)Reconstruct shows layer v(1), parallel Gibbs model, calculates and shows layer neuron by hidden neuron again
The probability value of activation：
By showing layer v(1)Reconstruct hidden layer h(1), parallel Gibbs model, calculates hidden neuron by aobvious layer neuron again
The probability value of activation：
The weight and bias between each neuron are updated parallel：
Wherein, η is preset learning rate.
Further, the training unit using between updated each neuron weight and bias as just
Beginningization training parameter carries out Training to depth confidence neural network model, including：
According to back-propagation algorithm, supervision is carried out to the depth B P neural networks in the depth confidence neural network and has been adjusted
Excellent training；
The weight and bias that each layer is updated using gradient descent algorithm iteration, until the weight and bias of each layer reach
It optimizes.
Technical scheme of the present invention has the advantages that：
Sorting technique provided by the invention and device, computer readable storage medium, are taken out by multi-threaded parallel gibbs
Sample and weight and bias between each neuron are updated parallel, played the parallel advantage of algorithm, improved system
Arithmetic speed.
Description of the drawings
Attached drawing described herein is used to provide further understanding of the present invention, and is constituted part of this application, this hair
Bright illustrative embodiments and their description are not constituted improper limitations of the present invention for explaining the present invention.In the accompanying drawings：
Fig. 1 is a kind of flow diagram of sorting technique of the embodiment of the present invention；
Fig. 2 is a kind of open parallel computation isomery framework PHM system hierarchy schematic diagrames of the embodiment of the present invention；
Fig. 3 is a kind of feature extraction concurrent operation flow diagram of the embodiment of the present invention；
Fig. 4 is a kind of feature extraction parallel computation GPU method schematic diagrams of the embodiment of the present invention；
Fig. 5 is in the related technology by notch hereby graceful machine simple structure schematic diagram；
Fig. 6 is depth confidence network (Deep Belief Nets, DBN) concurrent operation flow chart of the embodiment of the present invention；
Fig. 7 is the DBN principle schematic diagrams of the embodiment of the present invention；
Fig. 8 is a kind of structural schematic diagram of sorter of first embodiment of the invention；
Fig. 9 is a kind of structural schematic diagram of sorter of second embodiment of the invention.
Specific implementation mode
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing to the present invention
Embodiment be described in detail.It should be noted that in the absence of conflict, in the embodiment and embodiment in the application
Feature mutually can arbitrarily combine.
As shown in Figure 1, a kind of sorting technique according to the present invention, includes the following steps：
Step 101：Characteristic is inputted to the depth confidence neural network model pre-established；
In the present embodiment, further include before the method：The data for obtaining sensor acquisition pass through letter to the data of acquisition
Number processing obtains characteristic.
Further, characteristic is obtained by signal processing to the data of acquisition, including：
By the data of acquisition, chronologically adding window is divided into multiple data blocks；
Concurrent Feature extraction, and the feature that each thread is extracted are carried out to the multiple data blocks marked off by multithreading
Category merges, the characteristic after being merged.
It should be noted that the sorting technique in the present embodiment uses the method for parallel training to depth confidence neural network
Model is trained.The method that the present invention can utilize serial or parallel passes through signal processing to the data of acquisition, obtains feature
Data.Chronologically adding window of the present invention refers to carrying out piecemeal to data in time domain.The collected data of sensor are on time
Sequence processing adding window is divided into the data block of multiple equal sizes, and data block is stored in the shared drive of processor in each window,
It is used for processor multithreading is shared.
On equipment the hardware of PHM systems can by central processing unit (Central Processing Unit, CPU)+
Two kinds of system platform knots of GPU or CPU+ field programmable gate arrays (Field-Programmable Gate Array, FPGA)
Structure forms, and CPU runs primary control program, and GPU needs the algorithm routine of concurrent operation, answer as data parallel equipment, operation
With general purpose GPU (General Purpose GPU, GPGPU) technologies and open operation language (Open Computing
Language, OpenCL) technology completion general-purpose computations.FPGA system is realized by algorithm, c program (hardware c program) to FPGA mesh
Mark the concurrent operation solution of data file.PHM programs realize diagnosis, prediction and health state evaluation, using parallel computation
Algorithm bottom hardware is driven by the parallel computation framework of standard so that diagnosis prediction program realizes independently of bottom hardware,
The embedded computing system using GPU or FPGA can be common to.
Fig. 2 is a kind of the health control plateform system hierarchical structure of concurrent operation to be supported to illustrate according to what the present invention realized
Figure, the bottom is hardware configuration, is made of CPU, GPU or FPGA, the primary control program that CPU operation orders execute, GPU (or FPGA)
The algorithm routine of application acceleration technology concurrent operation is run, completes to accelerate general purpose operation.Middle layer is operating system and simultaneously
Row operational software framework, operating system complete the functions such as program scheduler, memory management, human-computer interaction, parallel on CPU
Operational software framework mainly applies GPGPU and OpenCL technologies to realize from health control algorithm routine to GPU concurrent operation programs
Realization.Top layer is the algorithm routine applied to health control, include mainly data processing features extraction (root mean square operation,
The flexure factor, kurtosis factor etc.), health evaluating (K mean values (K-means) cluster, hidden Markov model etc.), fault diagnosis
(depth confidence neural network, support vector machines etc.), predicting residual useful life (hidden Markov model, neural network etc.).
The present invention mainly applies the numerous processing cores of GPU and higher bandwidth of memory and stronger processing highly-parallel
Data-intensive computing ability and open, compatible, the free standard towards the general purpose multiple programming of heterogeneous system
OpenCL opens operation language, and concurrent operation is realized in CPU, GPU, FPGA heterogeneous system.By using open parallel computation frame
Structure, the present invention provides a kind of health control platforms for supporting concurrent operation, improve the upper PHM system operations speed of equipment and letter
Breath analysis ability has effectively ensured the safe operation of equipment to realize that real-time fault diagnosis and repair suggest generating.
In PHM systems, sensor (measuring each system component status signal) data on equipment are transferred to PHM systems,
Data processing and information analysis main operational, which are feature extraction processing and health control analysis, the present invention, realizes these cores
The concurrent operation platform of operation.In our CPU+GPU heterogeneous systems, CPU is responsible for the logic of the whole process of serial process
Control and serial task therein are organized organization data and are copied in the memory of GPU, then GPU called to generate multiple threads
And be mapped on the computing array of GPU, kernel (Kernel) function is executed parallel with OpenCL technical controlling GPU many-cores on GPU,
Concurrent operation is carried out, GPU is finished, and data are copied go back to the ends CPU by serial program, complete once-through operation.The operation list on GPU
Data communication between member passes through global memory by Sharing Memory Realization in GPU blocks between GPU blocks.Hereinafter, being directed to feature respectively
For extraction and health evaluating clustering method, illustrate concurrent operation software desk Implementation method.
Further, identical when executing the multithreading by multiple multiprocessors (Multi Processor, MP)
By shared drive into row data communication between multithreading inside MP；Data are carried out between different MP by global memory to lead to
Letter.
In multiprocessor, start multithreading simultaneously with each MP on OpenCL technical controllings GPU, it is parallel to execute kernel in MP
Function, each kernel functions (time domain, time and frequency zone signal characteristic abstraction function) call the data in same shared drive, warp
The data crossed after data processing are deposited into the global store of GPU.Inventive novelty herein is due to PHM feature extractions
Sequential processing, shared data is with uniformity to the data of multithreading operation in MP, and at the same time carrying out the multichannel feature of signal
Operation.Wherein kernel functions are signal characteristic abstraction function, they are the root mean square of time domain, the flexure factor, the kurtosis factor, wave
The peak factor, the nargin factor, shape factor, pulse factor operation function and time-frequency domain empirical mode decomposition eigen mode letter
Number.After feature extraction and calculation, uniformity signal characteristic is grouped together in global memory.
Signal characteristic abstraction concurrent operation method is as shown in figure 3, multiple characteristic operations execute parallel.
Its GPU implementation method is as shown in Figure 4, wherein Rms is root mean square operation, and Cf is crest factor operation, and If is pulse
Factor operation, Mf are nargin factor operations, and Sk is flexure factor operation, and Ku is kurtosis factor operation, and Sf is shape factor operation,
Emd is empirical mode decomposition operation.
For the fault diagnosis of PHM systems, carried out using DBN (the depth confidence neural network based on deep neural network)
Automatic classification to characteristic, to carry out fault diagnosis.Deep neural network is using RBM (by notch hereby graceful machine), parallel
Operation design is as follows：
RBM by notch hereby graceful machine simple structure schematic diagram as shown in figure 5, wherein each limited Boltzmann machine is a kind of god
Through perceptron, it is made of with hidden layer an aobvious layer.It is two-way between hidden layer H and aobvious layer V neurons to connect.
RBM algorithm principles：
The probability that hidden neuron is shown layer neuronal activation is：P(Hj| V)=sigmoid (bj+∑iwi,j*xi)
Show layer neuron is by the probability that hidden neuron activates：
P(V|Hj)=sigmoid (cj+∑jwi,j*Hi)
Wherein w is bonding strength, and b is the biasing coefficient to showing layer, and c is the biasing coefficient to hidden layer.
xiTo show layer neuron wherein i=1,2,3..NV, HjFor hidden neuron wherein j=1,2,3 ... NH。
The probability that each hidden neuron is activated is calculated, and threshold value u is set, and the value range of u is 0-1, if more than
Threshold value is then activated, and is not otherwise activated.The activation probability for similarly calculating each aobvious layer neuron, judges whether it is activated.
First, data are passed into aobvious layer, calculates the probability P (H that each neuron of hidden layer is activated1|V1).From the general of calculating
Gibbs model is taken to extract a sample, H in rate distribution1~P (H1|V1)
Use H1It rebuilds and shows layer, aobvious layer is pushed away by the way that hidden layer is counter.
In next step, the probability P (V for showing that each neuron is activated in layer is calculated2|H1), similarly taken out using Gibbs model
Take a sample, V2~P (V2|H1)
In next step, the probability that each hidden neuron is activated is calculated, distribution P (H are obtained2|V2)
In next step, weight is updated.W=w+ η * (P (H1|V1)*V1-P(H2|V2)*V2)
B=b+ η * (V1-V2)
C=c+ η * (H1-H2)
Wherein η is learning rate.
Collateral learning gathers better than the part that existing learning process is multichannel Non-surveillance clustering while handling small data set
Class, to accelerate the speed of overall data clustering.
Step 102：By multi-threaded parallel Gibbs model, according to phase between sampling results parallel computation hidden layer and aobvious layer
The probability being mutually activated updates weight and bias between each neuron parallel；
As shown in fig. 6, in RBM concurrent operations design, include the following steps：
First, each neuron of parallel computation hidden layer is activated probability,
In next step, it according to Gibbs model, is connected as sample calculating in hidden layer paralleling abstracting neuron and is each shown
Layer neuron is activated probability, pushes away aobvious layer neuron using hidden neuron is counter and is activated probability；
It should be noted that parallel Gibbs model herein is each neuron while being extracted, and calculate
Aobvious layer neuron is activated probability, and Gibbs model in the prior art is sequentially to extract neuron one by one and calculate its aobvious layer
Neuron is activated probability.
In next step, layer neuron is shown using parallel Gibbs model, calculates the aobvious anti-hidden neuron that pushes away of layer neuron and is swashed
Probability living
Finally, weight is updated parallel.
Parallel update weight herein, newer is the connection weight and bias between hidden neuron and aobvious layer
(and being to update simultaneously), and update weight in the prior art is to update the company of hidden neuron and aobvious layer neuron one by one
Connect weight and bias.
DBN principle schematic diagrams are as shown in Figure 7：
Depth confidence neural network is made of multiple limited Boltzmann machines (RBM) with depth B P neural networks.Depth
The training of confidence neural network model is mainly following two steps：
1) each layer of unsupervised training is limited Boltzmann's network (RBM)
Training characteristics data input, and initialize weighted value w between the hidden layer of each RBM and aobvious layer and each other neuron
Between bias b, c.It calculates between hidden layer and aobvious layer, be mutually activated probability.
RBM networks in DBN neural networks can make the initialization weighting parameter of deep layer BP neural network, make entire DBN
Neural network compensates for because training time for being brought because random parameter initializes of BP neural network is long, and is easily absorbed in office
The shortcomings of portion is optimal.
2) training parameter obtained after the unsupervised training of RBM trains ginseng as the initialization of depth B P neural networks
Number carries out Training
In order to reach the desired value of trained label, according to back-propagation algorithm, carry out for the depth in entire neural network
Spend training weights in BP neural network, the small parameter perturbations of bias, and the training weighted value of each layer for being optimal, biasing
Value.
Parallel DBN algorithm principles：
First, characteristic is input to aobvious layer.Random initializtion weight w is b to aobvious layer bias, to hidden layer bias c
Parameter.
It calculates aobvious layer data value v and hidden layer data value h and passes through formula：
H=w*v+c；
V=wT*h+b；
Wherein, v is the matrix of M*1, and h is the matrix of N*1, and M is aobvious layer neuron number, and N is hidden neuron number.W
For the matrix of M*N, b is the matrix of M*1, and c is the matrix of N*1.
In next step, parallel Gibbs model extracts each neuron shown on layer simultaneously, calculated with its corresponding value
Each hidden neuron is shown the probability value of layer neuronal activation, and the probability that hidden neuron is shown layer neuronal activation is：
Wherein i ∈ [0, M], j ∈ [0, N]
In next step, pass through hidden layer h(0)Reconstruct shows layer v(1), parallel Gibbs model, calculates and shows what layer was activated by hidden layer again
Probability value, aobvious layer neuron are by the probability that hidden neuron activates：
In next step, by showing layer v(1)Reconstruct hidden layer h(1), parallel Gibbs model, calculates hidden layer and is shown layer activation again
Probability value, the probability that hidden neuron is shown layer neuronal activation are：
In next step, the weight between update neuron and biasing parallel
Wherein, η is learning rate.
Step 103：Using between updated each neuron weight and bias as initialization training parameter, it is right
Depth confidence neural network model carries out Training；
In next step, according to trained w, b, hidden layer output is calculated using formula：hl=sigmoid (wl*v(2)+bl)；
H herein represents the output of hidden layer, and l is the index number of the hidden layer of BP neural network, v(2)As BP nerve nets
The input layer of network.
In next step, the output valve y of output layer is calculated：Y=sigmoid (h), wherein y is the matrix of M*1.
In next step, the model parameter of network is updated using the back-propagation algorithm of least-mean-square-error criterion, has carried out prison
The tuning training superintended and directed.Input expected matrix (training label matrix) y '.Calculate cost function：
Wherein, y ' is the matrix of M*1, and E is the matrix of M*1.
In next step, using gradient descent algorithm, iteration updates weight and the biasing of each layer：
Until E → 0, stop update weight and bias
Step 104：Classification and Identification is carried out using trained depth confidence neural network model.
Specifically, it according to trained each layer weighted value and bias, calculates and exports final classification results.
The embodiment of the present invention additionally provides a kind of computer readable storage medium, is deposited on the computer readable storage medium
One or more program is contained, one or more of programs can be executed by one or more processor, to realize such as
The step of sorting technique described in any of the above item.
As shown in figure 8, an embodiment of the present invention provides a kind of sorter, including input unit 801, training unit 802
With taxon 803, wherein：
Input unit 801, for characteristic to be inputted the depth confidence neural network model pre-established；
Training unit 802, for by multi-threaded parallel Gibbs model, according to sampling results parallel computation hidden layer and showing
The probability being mutually activated between layer, updates the weight and bias between each neuron parallel；It will be updated each
Weight and bias between neuron have carried out supervision as initialization training parameter to depth confidence neural network model
Training；
Taxon 803, for carrying out Classification and Identification using trained depth confidence neural network model.
In the present embodiment, as shown in figure 9, the sorter further includes feature extraction unit 804, wherein：
Feature extraction unit 804, the data for obtaining sensor acquisition, the data of acquisition are obtained by signal processing
Characteristic.
Further, the feature extraction unit 804 obtains characteristic to the data of acquisition by signal processing, packet
It includes：
By the data of acquisition, chronologically adding window is divided into multiple data blocks；
Concurrent Feature extraction, and the feature that each thread is extracted are carried out to the multiple data blocks marked off by multithreading
Category merges, the characteristic after being merged.
Further, when feature extraction unit 804 executes the multithreading by multiple multiprocessors (MP), identical MP
By shared drive into row data communication between internal multithreading；By global memory into row data communication between different MP.
Specifically, the collected data of sensor chronologically handle the data block that adding window is divided into multiple equal sizes, often
Data block is stored in the shared drive of processor in a window, is used for processor multithreading is shared；Passing through feature extraction meter
After calculation, uniformity signal characteristic is grouped together in global memory.
Further, feature extraction unit 804 extract feature include the root mean square of time domain, the flexure factor, the kurtosis factor,
Crest factor, the nargin factor, shape factor, pulse factor operation function and time-frequency domain empirical mode decomposition eigen mode letter
Number.
Further, the training unit 802 by multi-threaded parallel Gibbs model, counted parallel according to sampling results
The probability being mutually activated between hidden layer and aobvious layer is calculated, updates the weight and bias between each neuron parallel, including：
Random initializtion weight w is b to aobvious layer bias, to hidden layer bias c；Aobvious layer data value v and hidden layer data value
H calculation formula are：
H=w*v+c
V=wT*h+b
Wherein, v is the matrix of M*1, and h is the matrix of N*1, and M is aobvious layer neuron number, and N is hidden neuron number；W
For the matrix of M*N, b is the matrix of M*1, and c is the matrix of N*1；
Parallel Gibbs model, while the value for showing each neuron on layer is extracted, each hidden neuron is calculated by aobvious layer
The probability value of neuronal activation：
Wherein i ∈ [0, M], j ∈ [0, N]；
Pass through hidden layer h(0)Reconstruct shows layer v(1), parallel Gibbs model, calculates and shows layer neuron by hidden neuron again
The probability value of activation：
By showing layer v(1)Reconstruct hidden layer h(1), parallel Gibbs model, calculates hidden neuron by aobvious layer neuron again
The probability value of activation：
The weight and bias between each neuron are updated parallel：
Wherein, η is preset learning rate.
Further, the training unit 802 using between updated each neuron weight and bias as
Training parameter is initialized, Training is carried out to depth confidence neural network model, including：
According to back-propagation algorithm, prison has been carried out to the depth B P neural networks in the depth confidence neural network model
Superintend and direct tuning training；
The weight and bias that each layer is updated using gradient descent algorithm iteration, until the weight and bias of each layer reach
It optimizes.
The present invention realizes the universal classification parallel computing architecture framework of cross-platform isomery, is realized independently of bottom hardware,
It can support CPU, GPU and FPGA various processor platform；Using general purpose GPU and parallel computation architecture technology is opened,
Effectively improve the ability and real-time of genealogical classification operation so that can handle more complicated sorting algorithm on equipment.
One of ordinary skill in the art will appreciate that all or part of step in the above method can be instructed by program
Related hardware is completed, and described program can be stored in computer readable storage medium, such as read-only memory, disk or CD
Deng.Optionally, all or part of step of above-described embodiment can also be realized using one or more integrated circuits, accordingly
Ground, the form that hardware may be used in each module/unit in above-described embodiment are realized, the shape of software function module can also be used
Formula is realized.The present invention is not limited to the combinations of the hardware and software of any particular form.
Although describing the invention in detail above, the foregoing is merely the preferred embodiment of the present invention
, it is not intended to restrict the invention, for those skilled in the art, the invention may be variously modified and varied.It is all
Within the spirit and principles in the present invention, any modification, equivalent replacement, improvement and so on should be included in the guarantor of the present invention
Within the scope of shield.
Claims (10)
1. a kind of sorting technique, which is characterized in that including：
Characteristic is inputted to the depth confidence neural network model pre-established；
It is general according to what is be mutually activated between sampling results parallel computation hidden layer and aobvious layer by multi-threaded parallel Gibbs model
Rate updates weight and bias between each neuron parallel；
Using between updated each neuron weight and bias as initialization training parameter, to depth confidence nerve
Network model carries out Training；
Classification and Identification is carried out using trained depth confidence neural network model.
2. sorting technique according to claim 1, which is characterized in that described to pass through multi-threaded parallel Gibbs model, root
According to the probability being mutually activated between sampling results parallel computation hidden layer and aobvious layer, the weight between each neuron is updated parallel
And bias, including：
Parallel Gibbs model, while the value for showing each neuron on layer is extracted, each hidden neuron is calculated by aobvious layer nerve
The probability value of member activation：
Wherein i ∈ [0, M], j ∈ [0, N], v are aobvious layer data value, and h is hidden layer data value, and M is aobvious layer neuron number, and N is hidden
Layer neuron number；W is weight, and b is aobvious layer bias, and c is hidden layer bias；
Pass through hidden layer h(0)Reconstruct shows layer v(1), parallel Gibbs model, calculates aobvious layer neuron and is activated by hidden neuron again
Probability value：
By showing layer v(1)Reconstruct hidden layer h(1), parallel Gibbs model, calculates hidden neuron by aobvious layer neuronal activation again
Probability value：
The weight and bias between each neuron are updated parallel：
Wherein, η is preset learning rate.
3. sorting technique according to claim 1, which is characterized in that the power by between updated each neuron
Weight and bias carry out Training as initialization training parameter to depth confidence neural network model, including：
According to back-propagation algorithm, supervision is carried out to the depth B P neural networks in the depth confidence neural network model and has been adjusted
Excellent training；
The weight and bias that each layer is updated using gradient descent algorithm iteration, until the weight and bias of each layer are optimal
Change.
4. sorting technique according to claim 1, which is characterized in that further include before the method：
The data of acquisition are obtained characteristic by the data for obtaining sensor acquisition by signal processing.
5. sorting technique according to claim 4, which is characterized in that the data of described pair of acquisition are obtained by signal processing
Characteristic, including：
By the data of acquisition, chronologically adding window is divided into multiple data blocks；
Concurrent Feature extraction is carried out to the multiple data blocks marked off by multithreading, and the feature that each thread is extracted is by class
It does not merge, the characteristic after being merged.
6. sorting technique according to claim 5, which is characterized in that described more when being executed by multiple multiprocessors (MP)
When thread, by shared drive into row data communication between the multithreading inside identical MP；Pass through global memory between different MP
Into row data communication.
7. a kind of computer readable storage medium, which is characterized in that on the computer readable storage medium storage there are one or
The multiple programs of person, one or more of programs can be executed by one or more processor, with realize as claim 1 to
The step of sorting technique described in any one of 6.
8. a kind of sorter, which is characterized in that including input unit, training unit and taxon, wherein：
Input unit, for characteristic to be inputted the depth confidence neural network model pre-established；
Training unit, for passing through multi-threaded parallel Gibbs model, according between sampling results parallel computation hidden layer and aobvious layer
The probability being mutually activated updates weight and bias between each neuron parallel；By updated each neuron
Between weight and bias as initialization training parameter, to depth confidence neural network model carry out Training；
Taxon, for carrying out Classification and Identification using trained depth confidence neural network model.
9. sorter according to claim 8, which is characterized in that the training unit passes through multi-threaded parallel Ji cloth
This sampling, according to the probability being mutually activated between sampling results parallel computation hidden layer and aobvious layer, updates each neuron parallel
Between weight and bias, including：
Parallel Gibbs model, while the value for showing each neuron on layer is extracted, each hidden neuron is calculated by aobvious layer nerve
The probability value of member activation：
Wherein i ∈ [0, M], j ∈ [0, N], v are aobvious layer data value, and h is hidden layer data value, and M is aobvious layer neuron number, and N is hidden
Layer neuron number；W is weight, and b is aobvious layer bias, and c is hidden layer bias；
Pass through hidden layer h(0)Reconstruct shows layer v(1), parallel Gibbs model, calculates aobvious layer neuron and is activated by hidden neuron again
Probability value：
By showing layer v(1)Reconstruct hidden layer h(1), parallel Gibbs model, calculates hidden neuron by aobvious layer neuronal activation again
Probability value：
The weight and bias between each neuron are updated parallel：
Wherein, η is preset learning rate.
10. sorter according to claim 8, which is characterized in that the training unit by updated each god
Through the weight and bias conduct initialization training parameter between member, supervision instruction has been carried out to depth confidence neural network model
Practice, including：
According to back-propagation algorithm, supervision is carried out to the depth B P neural networks in the depth confidence neural network model and has been adjusted
Excellent training；
The weight and bias that each layer is updated using gradient descent algorithm iteration, until the weight and bias of each layer are optimal
Change.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810469963.4A CN108763360A (en) | 2018-05-16 | 2018-05-16 | A kind of sorting technique and device, computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810469963.4A CN108763360A (en) | 2018-05-16 | 2018-05-16 | A kind of sorting technique and device, computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108763360A true CN108763360A (en) | 2018-11-06 |
Family
ID=64008252
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810469963.4A Pending CN108763360A (en) | 2018-05-16 | 2018-05-16 | A kind of sorting technique and device, computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108763360A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109711543A (en) * | 2018-12-30 | 2019-05-03 | 南京大学 | A kind of restructural depth confidence network implementations system |
| CN109740527A (en) * | 2018-12-30 | 2019-05-10 | 杭州铭智云教育科技有限公司 | Image processing method in a kind of video frame |
| CN109739979A (en) * | 2018-12-11 | 2019-05-10 | 中科恒运股份有限公司 | Tuning method, tuning device and the terminal of neural network |
| CN109901869A (en) * | 2019-01-25 | 2019-06-18 | 中国电子科技集团公司第三十研究所 | A kind of computer program classification method based on bag of words |
| CN111552563A (en) * | 2020-04-20 | 2020-08-18 | 南昌嘉研科技有限公司 | Multithreading data architecture, multithreading message transmission method and system |
| CN111638034A (en) * | 2020-06-09 | 2020-09-08 | 重庆大学 | Strain balance temperature gradient error compensation method and system based on deep learning |
| CN112734048A (en) * | 2021-01-26 | 2021-04-30 | 天津飞腾信息技术有限公司 | Reinforced learning method |
| CN113065642A (en) * | 2021-04-09 | 2021-07-02 | 中电科数字科技(集团)有限公司 | Artificial intelligence acceleration method and system based on heterogeneous computing |
| CN113420785A (en) * | 2021-05-31 | 2021-09-21 | 北京联合大学 | Method and device for classifying written corpus types, storage medium and electronic equipment |
| WO2021255569A1 (en) * | 2020-06-18 | 2021-12-23 | International Business Machines Corporation | Drift regularization to counteract variation in drift coefficients for analog accelerators |
| CN113933652A (en) * | 2021-10-15 | 2022-01-14 | 南方电网科学研究院有限责任公司 | Method and system for upgrading algorithm of high-resistance grounding identification device of power distribution network |
| CN114826921A (en) * | 2022-05-05 | 2022-07-29 | 苏州大学应用技术学院 | Network resource dynamic allocation method, system and medium based on sampling subgraph |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436811A (en) * | 2010-09-21 | 2012-05-02 | 微软公司 | Full-sequence training of deep structures for speech recognition |
| CN102737278A (en) * | 2011-03-31 | 2012-10-17 | 微软公司 | Deep convex network with joint use of nonlinear random projection, restricted boltzmann machine and batch-based parallelizable optimization |
| CN106769048A (en) * | 2017-01-17 | 2017-05-31 | 苏州大学 | Adaptive deep confidence network bearing fault diagnosis method based on Nesterov momentum method |
| CN107316078A (en) * | 2016-04-27 | 2017-11-03 | 北京中科寒武纪科技有限公司 | Apparatus and method for performing artificial neural network self study computing |
| CN107993012A (en) * | 2017-12-04 | 2018-05-04 | 国网湖南省电力有限公司娄底供电分公司 | A kind of adaptive electric system on-line transient stability appraisal procedure of time |
-
2018
- 2018-05-16 CN CN201810469963.4A patent/CN108763360A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436811A (en) * | 2010-09-21 | 2012-05-02 | 微软公司 | Full-sequence training of deep structures for speech recognition |
| CN102737278A (en) * | 2011-03-31 | 2012-10-17 | 微软公司 | Deep convex network with joint use of nonlinear random projection, restricted boltzmann machine and batch-based parallelizable optimization |
| CN107316078A (en) * | 2016-04-27 | 2017-11-03 | 北京中科寒武纪科技有限公司 | Apparatus and method for performing artificial neural network self study computing |
| CN106769048A (en) * | 2017-01-17 | 2017-05-31 | 苏州大学 | Adaptive deep confidence network bearing fault diagnosis method based on Nesterov momentum method |
| CN107993012A (en) * | 2017-12-04 | 2018-05-04 | 国网湖南省电力有限公司娄底供电分公司 | A kind of adaptive electric system on-line transient stability appraisal procedure of time |
Non-Patent Citations (2)
| Title |
|---|
| 石祥滨等: "基于对称神经网络的图像分类", 《小型微型计算机系统》 * |
| 雷振山: "《LabVIEW高级编程与虚拟仪器工程应用》", 31 December 2013 * |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109739979A (en) * | 2018-12-11 | 2019-05-10 | 中科恒运股份有限公司 | Tuning method, tuning device and the terminal of neural network |
| CN109740527A (en) * | 2018-12-30 | 2019-05-10 | 杭州铭智云教育科技有限公司 | Image processing method in a kind of video frame |
| CN109711543A (en) * | 2018-12-30 | 2019-05-03 | 南京大学 | A kind of restructural depth confidence network implementations system |
| CN109901869B (en) * | 2019-01-25 | 2022-03-18 | 中国电子科技集团公司第三十研究所 | Bag-of-words model-based computer program classification method |
| CN109901869A (en) * | 2019-01-25 | 2019-06-18 | 中国电子科技集团公司第三十研究所 | A kind of computer program classification method based on bag of words |
| CN111552563A (en) * | 2020-04-20 | 2020-08-18 | 南昌嘉研科技有限公司 | Multithreading data architecture, multithreading message transmission method and system |
| CN111638034B (en) * | 2020-06-09 | 2021-07-09 | 重庆大学 | Strain balance temperature gradient error compensation method and system based on deep learning |
| CN111638034A (en) * | 2020-06-09 | 2020-09-08 | 重庆大学 | Strain balance temperature gradient error compensation method and system based on deep learning |
| WO2021255569A1 (en) * | 2020-06-18 | 2021-12-23 | International Business Machines Corporation | Drift regularization to counteract variation in drift coefficients for analog accelerators |
| GB2611681A (en) * | 2020-06-18 | 2023-04-12 | Ibm | Drift regularization to counteract variation in drift coefficients for analog accelerators |
| CN112734048A (en) * | 2021-01-26 | 2021-04-30 | 天津飞腾信息技术有限公司 | Reinforced learning method |
| CN113065642A (en) * | 2021-04-09 | 2021-07-02 | 中电科数字科技(集团)有限公司 | Artificial intelligence acceleration method and system based on heterogeneous computing |
| CN113065642B (en) * | 2021-04-09 | 2023-04-07 | 中电科数字科技(集团)有限公司 | Artificial intelligence acceleration method and system based on heterogeneous computing |
| CN113420785A (en) * | 2021-05-31 | 2021-09-21 | 北京联合大学 | Method and device for classifying written corpus types, storage medium and electronic equipment |
| CN113420785B (en) * | 2021-05-31 | 2023-12-19 | 北京联合大学 | Method and device for classifying written language types, storage medium and electronic equipment |
| CN113933652A (en) * | 2021-10-15 | 2022-01-14 | 南方电网科学研究院有限责任公司 | Method and system for upgrading algorithm of high-resistance grounding identification device of power distribution network |
| CN114826921A (en) * | 2022-05-05 | 2022-07-29 | 苏州大学应用技术学院 | Network resource dynamic allocation method, system and medium based on sampling subgraph |
| CN114826921B (en) * | 2022-05-05 | 2024-05-17 | 苏州大学应用技术学院 | Dynamic network resource allocation method, system and medium based on sampling subgraph |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108763360A (en) | A kind of sorting technique and device, computer readable storage medium | |
| Nabil et al. | Prediction of students’ academic performance based on courses’ grades using deep neural networks | |
| Peng et al. | Building program vector representations for deep learning | |
| Swain et al. | An approach for iris plant classification using neural network | |
| Lopes et al. | Towards adaptive learning with improved convergence of deep belief networks on graphics processing units | |
| Sindhwani et al. | Performance analysis of deep neural networks using computer vision | |
| Artemenko et al. | The formation of a set of informative features based on the functional relationships between the data structure field observations | |
| Mou et al. | Building program vector representations for deep learning | |
| Kasfi et al. | Convolutional neural network for time series cattle behaviour classification | |
| CN110363090A (en) | Intelligent heart disease detection method, device and computer readable storage medium | |
| Niimi | Deep learning for credit card data analysis | |
| Chicho et al. | A comprehensive survey of deep learning models based on Keras framework | |
| Khalil et al. | Diagnosis of blood cells using deep learning | |
| CN108710958A (en) | A kind of prediction health control method and device, computer readable storage medium | |
| Bharadi | Qlattice environment and Feyn QGraph models—A new perspective toward deep learning | |
| Benmeziane | Comparison of deep learning frameworks and compilers | |
| Apolloni et al. | Training a network of mobile neurons | |
| WO2022169954A1 (en) | Deep neural network learning with controllable rules | |
| Kumari et al. | Accuracy testing of data classification using tensor flow a python framework in ANN designing | |
| Rajendran et al. | Vortex detection on unsteady CFD simulations using recurrent neural networks | |
| Qiao et al. | SRS-DNN: a deep neural network with strengthening response sparsity | |
| Štancel et al. | An introduction to image classification and object detection using YOLO detector | |
| Manoharan et al. | Ensemble Model for Educational Data Mining Based on Synthetic Minority Oversampling Technique | |
| Shankar et al. | Software Defect Prediction using ANN Algorithm | |
| Lacey | Deep learning on FPGAs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181106 |
|
| RJ01 | Rejection of invention patent application after publication |