SG10201904549QA - System And Method For Training Neural Networks - Google Patents

System And Method For Training Neural Networks

Info

Publication number
SG10201904549QA
SG10201904549QA SG10201904549QA SG10201904549QA SG10201904549QA SG 10201904549Q A SG10201904549Q A SG 10201904549QA SG 10201904549Q A SG10201904549Q A SG 10201904549QA SG 10201904549Q A SG10201904549Q A SG 10201904549QA SG 10201904549Q A SG10201904549Q A SG 10201904549QA
Authority
SG
Singapore
Prior art keywords
nodes
hidden layers
training
layer
neural network
Prior art date
Application number
SG10201904549QA
Inventor
Jianshu Li
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to SG10201904549QA priority Critical patent/SG10201904549QA/en
Publication of SG10201904549QA publication Critical patent/SG10201904549QA/en
Priority to MYPI2019007148A priority patent/MY195878A/en
Priority to PH12019000467A priority patent/PH12019000467A1/en
Priority to US16/791,749 priority patent/US11250323B2/en
Priority to CN202010244417.8A priority patent/CN111461305A/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G06V40/173—Classification, e.g. identification face re-identification, e.g. recognising unknown faces across different face tracks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Abstract

SYSTEM AND METHOD FOR TRAINING NEURAL NETWORKS A method comprising: training a pre -trained neural network that comprises: an input layer; a plurality of hidden layers, wherein each of the plurality of hidden layers has one or more nodes, wherein each of said one or more nodes has an associated weight trained based on data from a source domain; and an output layer. Training the pre - trained neural network comprises: introducing at least one additional layer to the plurality of hidden layers, wherein said additional layer has one or more nodes having associated weights; keeping weights of the nodes in the plurality of hidden layers of the pre-trained neural network unchanged; inputting data from a target domain to the input layer; and adjusting weights of the one or more nodes in the at least one additional layer based on features obtained at the output layer. Figure 4
SG10201904549QA 2019-05-21 2019-05-21 System And Method For Training Neural Networks SG10201904549QA (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
SG10201904549QA SG10201904549QA (en) 2019-05-21 2019-05-21 System And Method For Training Neural Networks
MYPI2019007148A MY195878A (en) 2019-05-21 2019-12-03 System And Method For Training Neural Networks
PH12019000467A PH12019000467A1 (en) 2019-05-21 2019-12-06 System and method for training neural networks
US16/791,749 US11250323B2 (en) 2019-05-21 2020-02-14 System and method for training neural networks
CN202010244417.8A CN111461305A (en) 2019-05-21 2020-03-31 Neural network training method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
SG10201904549QA SG10201904549QA (en) 2019-05-21 2019-05-21 System And Method For Training Neural Networks

Publications (1)

Publication Number Publication Date
SG10201904549QA true SG10201904549QA (en) 2019-09-27

Family

ID=68062734

Family Applications (1)

Application Number Title Priority Date Filing Date
SG10201904549QA SG10201904549QA (en) 2019-05-21 2019-05-21 System And Method For Training Neural Networks

Country Status (5)

Country Link
US (1) US11250323B2 (en)
CN (1) CN111461305A (en)
MY (1) MY195878A (en)
PH (1) PH12019000467A1 (en)
SG (1) SG10201904549QA (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10672663B2 (en) 2016-10-07 2020-06-02 Xcelsis Corporation 3D chip sharing power circuit
US10672745B2 (en) 2016-10-07 2020-06-02 Xcelsis Corporation 3D processor
US10580757B2 (en) 2016-10-07 2020-03-03 Xcelsis Corporation Face-to-face mounted IC dies with orthogonal top interconnect layers
US10762420B2 (en) * 2017-08-03 2020-09-01 Xcelsis Corporation Self repairing neural network
US10580735B2 (en) 2016-10-07 2020-03-03 Xcelsis Corporation Stacked IC structure with system level wiring on multiple sides of the IC die
KR102512017B1 (en) 2016-10-07 2023-03-17 엑셀시스 코포레이션 Direct-bonded native interconnects and active base die
CN111428748B (en) * 2020-02-20 2023-06-27 重庆大学 HOG feature and SVM-based infrared image insulator identification detection method
US20220094713A1 (en) * 2020-09-21 2022-03-24 Sophos Limited Malicious message detection
CN113657651A (en) * 2021-07-27 2021-11-16 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) Diesel vehicle emission prediction method, medium and equipment based on deep migration learning
CN116910449A (en) * 2023-07-12 2023-10-20 上海虹港数据信息有限公司 Operation application platform of artificial intelligence power

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7917596B2 (en) 2009-01-07 2011-03-29 Oracle International Corporation Super master
US9235799B2 (en) * 2011-11-26 2016-01-12 Microsoft Technology Licensing, Llc Discriminative pretraining of deep neural networks
US9329950B2 (en) 2014-01-01 2016-05-03 International Business Machines Corporation Efficient fail-over in replicated systems
US10068171B2 (en) * 2015-11-12 2018-09-04 Conduent Business Services, Llc Multi-layer fusion in a convolutional neural network for image classification
US10410108B2 (en) 2016-08-08 2019-09-10 EyeEm Mobile GmbH Systems, methods, and computer program products for searching and sorting images by aesthetic quality personalized to users or segments
US11106974B2 (en) * 2017-07-05 2021-08-31 International Business Machines Corporation Pre-training of neural network by parameter decomposition
US10679346B2 (en) * 2018-01-30 2020-06-09 General Electric Company Systems and methods for capturing deep learning training data from imaging systems
CN108881660B (en) * 2018-05-02 2021-03-02 北京大学 Method for compressing and calculating hologram by adopting quantum neural network for optimizing initial weight
US10600005B2 (en) * 2018-06-01 2020-03-24 Sas Institute Inc. System for automatic, simultaneous feature selection and hyperparameter tuning for a machine learning model
CN109376773A (en) 2018-09-30 2019-02-22 福州大学 Crack detecting method based on deep learning
CN109409520B (en) 2018-10-17 2021-10-29 深圳市微埃智能科技有限公司 Welding process parameter recommendation method and device based on transfer learning and robot

Also Published As

Publication number Publication date
US11250323B2 (en) 2022-02-15
MY195878A (en) 2023-02-27
CN111461305A (en) 2020-07-28
US20200372345A1 (en) 2020-11-26
PH12019000467A1 (en) 2021-01-11

Similar Documents

Publication Publication Date Title
SG10201904549QA (en) System And Method For Training Neural Networks
MX2016011438A (en) Neural network and method of neural network training.
Ota et al. A comparative study on optimization techniques for solving multi-objective geometric programming problems
IL274424B1 (en) Meta-learning for multi-task learning for neural networks
WO2002023224A3 (en) Neural net prediction of seismic streamer shape
WO2015011688A3 (en) Method of training a neural network
BR112017019821B8 (en) Relevance score assignment for artificial neural networks
EP4030348A4 (en) Neural network training method, data processing method, and related apparatuses
CN109657055A (en) Title party article detection method and federal learning strategy based on level hybrid network
JP2013020638A5 (en)
WO2010042256A3 (en) Methods of improved learning in simultaneous recurrent neural networks
WO2007050622A3 (en) Weighted pattern learning for neural networks
GB2600871A (en) Machine learning hardware having reduced precision parameter components for efficient parameter update
SG10201610116PA (en) Method and system for machine failure prediction
AU2020202422B2 (en) Method of secure classification of input data by means of a convolutional neural network
Otair et al. Speeding up back-propagation neural networks
CN108168577A (en) MEMS gyro random error compensation method based on BP neural network
CN106355346A (en) Method and system for predicting box-office performance of movies on basis of neural network algorithms
WO2020041026A8 (en) Efficiently building deep neural networks
EP4148629A4 (en) Neural network training method via auto-encoder and multiple instance learning, and computing system for performing same
WO2015020815A3 (en) Implementing delays between neurons in an artificial nervous system
MX2021011131A (en) Low displacement rank based deep neural network compression.
KR960705277A (en) LEARNING NEURAL NETWORK AND METHODS THEREOF
WO2020174458A3 (en) Partial activation of multiple pathways in neural networks
WO2021221977A3 (en) Neural network training technique