CN115398419A - 用于基于目标的超参数调优的方法和系统 - Google Patents

用于基于目标的超参数调优的方法和系统 Download PDF

Info

Publication number
CN115398419A
CN115398419A CN202180025698.0A CN202180025698A CN115398419A CN 115398419 A CN115398419 A CN 115398419A CN 202180025698 A CN202180025698 A CN 202180025698A CN 115398419 A CN115398419 A CN 115398419A
Authority
CN
China
Prior art keywords
machine learning
learning model
metric
score
loss function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202180025698.0A
Other languages
English (en)
Chinese (zh)
Inventor
M·E·约翰逊
T·L·董
V·韦氏诺一
S·维纳科塔
T·帕姆
C·D·V·黄
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Oracle International Corp
Original Assignee
Oracle International Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oracle International Corp filed Critical Oracle International Corp
Publication of CN115398419A publication Critical patent/CN115398419A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G06F18/2178—Validation; Performance evaluation; Active pattern learning techniques based on feedback of a supervisor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0985—Hyperparameter optimisation; Meta-learning; Learning-to-learn
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/02—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/12—Computing arrangements based on biological models using genetic models
    • G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Medical Informatics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Machine Translation (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
CN202180025698.0A 2020-03-30 2021-03-30 用于基于目标的超参数调优的方法和系统 Pending CN115398419A (zh)

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
US202063002159P 2020-03-30 2020-03-30
US63/002,159 2020-03-30
US202063119577P 2020-11-30 2020-11-30
US63/119,577 2020-11-30
US17/216,498 US20210304074A1 (en) 2020-03-30 2021-03-29 Method and system for target based hyper-parameter tuning
US17/216,498 2021-03-29
US17/216,496 2021-03-29
US17/216,496 US12405975B2 (en) 2020-03-30 2021-03-29 Method and system for constraint based hyperparameter tuning
PCT/US2021/024953 WO2021202576A1 (en) 2020-03-30 2021-03-30 Method and system for target based hyper-parameter tuning

Publications (1)

Publication Number Publication Date
CN115398419A true CN115398419A (zh) 2022-11-25

Family

ID=77856190

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202180025698.0A Pending CN115398419A (zh) 2020-03-30 2021-03-30 用于基于目标的超参数调优的方法和系统
CN202180025672.6A Pending CN115398418A (zh) 2020-03-30 2021-03-30 用于基于约束的超参数调优的方法和系统

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN202180025672.6A Pending CN115398418A (zh) 2020-03-30 2021-03-30 用于基于约束的超参数调优的方法和系统

Country Status (5)

Country Link
US (2) US20210304074A1 (https=)
EP (2) EP4127963A1 (https=)
JP (2) JP7692432B2 (https=)
CN (2) CN115398419A (https=)
WO (2) WO2021202576A1 (https=)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115827171A (zh) * 2023-01-31 2023-03-21 阿里巴巴达摩院(杭州)科技有限公司 云端调参系统、调参方法及调参系统
CN116991429A (zh) * 2023-09-28 2023-11-03 之江实验室 计算机程序的编译调优方法、装置和存储介质
CN118626918A (zh) * 2024-08-14 2024-09-10 杭州迪普科技股份有限公司 一种基于人工智能的数据分类分级方法
CN119536835A (zh) * 2023-08-30 2025-02-28 华为技术有限公司 一种参数调优方法、装置以及设备

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2569335B (en) * 2017-12-13 2022-07-27 Sage Global Services Ltd Chatbot system
US11463386B2 (en) * 2020-09-23 2022-10-04 Capital One Services, Llc Systems and methods for generating conversational responses using machine learning models
US20230034136A1 (en) * 2021-07-30 2023-02-02 Kabushiki Kaisha Toshiba System and method for scheduling communication within a distributed learning and deployment framework
US11870651B2 (en) 2021-11-29 2024-01-09 Sap Se Landscape model verification system
US12567034B2 (en) * 2022-04-05 2026-03-03 Thrive Technologies, Inc. System and processes for optimizing inventory
CN114897617A (zh) * 2022-05-18 2022-08-12 北京百度网讯科技有限公司 金融风控场景的模型评估方法、装置、设备以及存储介质
US12026254B2 (en) * 2022-06-17 2024-07-02 Optum, Inc. Prediction model selection for cyber security
CN115277073B (zh) * 2022-06-20 2024-02-06 北京邮电大学 信道传输的方法、装置、电子设备及介质
US12517159B2 (en) 2022-08-04 2026-01-06 Viasat, Inc. Machine learning based tuning of radio frequency apparatuses
CN115511186A (zh) * 2022-09-29 2022-12-23 苏州浪潮智能科技有限公司 一种深度学习训练时长的预测管理方法、装置及设备
US12277396B2 (en) * 2022-09-30 2025-04-15 Tenyx, Inc. Assessing and improving the deployment of large language models in specific domains
KR20240103576A (ko) * 2022-12-27 2024-07-04 주식회사 모빌린트 딥러닝 컴파일러 최적화 성능 모니터링 장치 및 방법
GB202302321D0 (en) * 2023-02-17 2023-04-05 Samsung Electronics Co Ltd Methods and apparatus for Ai/Ml model monitoring
CN118821957A (zh) * 2023-04-18 2024-10-22 自动化机器学习有限公司 用于运行预测引擎的方法和装置
US12008409B1 (en) * 2023-05-02 2024-06-11 The Strategic Coach Inc. Apparatus and a method for determining resource distribution
CN116301282B (zh) * 2023-05-16 2023-08-01 中诚华隆计算机技术有限公司 一种多核处理器芯片的低功耗控制方法和装置
US12231378B2 (en) * 2023-06-08 2025-02-18 Sap Se Realtime conversation AI insights and deployment
CN116738239B (zh) * 2023-08-11 2023-11-24 浙江菜鸟供应链管理有限公司 模型训练方法、资源调度方法及装置、系统、设备及介质
US20250103908A1 (en) * 2023-09-21 2025-03-27 International Business Machines Corporation Dynamic Selection of AI Computer Models to Reduce Costs and Maximize User Experience
US12293265B1 (en) 2024-01-10 2025-05-06 The Strategic Coach Inc. Apparatus and method for model optimization
US12400149B1 (en) 2024-04-22 2025-08-26 Sas Institute Inc. Systems and methods for parallel exploration of a hyperparameter search space
US12367342B1 (en) * 2025-01-15 2025-07-22 Conversational AI Ltd Automated analysis of computerized conversational agent conversational data

Family Cites Families (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9152926B2 (en) * 2012-02-02 2015-10-06 Arizona Board Of Regents On Behalf Of Arizona State University Systems, methods, and media for updating a classifier
US9330362B2 (en) * 2013-05-15 2016-05-03 Microsoft Technology Licensing, Llc Tuning hyper-parameters of a computer-executable learning algorithm
US10558935B2 (en) 2013-11-22 2020-02-11 California Institute Of Technology Weight benefit evaluator for training data
JP2016042322A (ja) 2014-08-19 2016-03-31 日本電気株式会社 データ分析装置、分析方法とそのプログラム
US20160132787A1 (en) 2014-11-11 2016-05-12 Massachusetts Institute Of Technology Distributed, multi-model, self-learning platform for machine learning
US11070525B2 (en) 2016-04-15 2021-07-20 Adris Chakraborty Method and system of privacy enablement in a family networking computing platform
US10417566B2 (en) 2016-05-22 2019-09-17 Microsoft Technology Licensing, Llc Self-learning technique for training a PDA component and a simulated user component
US10572823B1 (en) * 2016-12-13 2020-02-25 Ca, Inc. Optimizing a malware detection model using hyperparameters
EP4700664A3 (en) 2017-05-17 2026-04-22 Intel Corporation Systems and methods implementing an intelligent optimization platform
US20190019108A1 (en) 2017-07-13 2019-01-17 General Electric Company Systems and methods for a validation tree
US11227188B2 (en) * 2017-08-04 2022-01-18 Fair Ip, Llc Computer system for building, training and productionizing machine learning models
US11270228B2 (en) 2017-11-17 2022-03-08 Panasonic Intellectual Property Management Co., Ltd. Information processing method and information processing system
JP7065368B2 (ja) 2017-11-17 2022-05-12 パナソニックＩｐマネジメント株式会社 情報処理方法および情報処理システム
US20190236487A1 (en) 2018-01-30 2019-08-01 Microsoft Technology Licensing, Llc Machine learning hyperparameter tuning tool
US10860629B1 (en) * 2018-04-02 2020-12-08 Amazon Technologies, Inc. Task-oriented dialog systems utilizing combined supervised and reinforcement learning
US10832002B2 (en) * 2018-05-08 2020-11-10 International Business Machines Corporation System and method for scoring performance of chatbots
US10635939B2 (en) * 2018-07-06 2020-04-28 Capital One Services, Llc System, method, and computer-accessible medium for evaluating multi-dimensional synthetic data using integrated variants analysis
US10558934B1 (en) 2018-08-23 2020-02-11 SigOpt, Inc. Systems and methods for implementing an intelligent machine learning optimization platform for multiple tuning criteria
EP3620996A1 (en) 2018-09-04 2020-03-11 Siemens Aktiengesellschaft Transfer learning of a machine-learning model using a hyperparameter response model
US11783917B2 (en) 2019-03-21 2023-10-10 Illumina, Inc. Artificial intelligence-based base calling
US12299541B2 (en) * 2019-05-22 2025-05-13 Adobe Inc. Model insights framework for providing insight based on model evaluations to optimize machine learning models
WO2021112822A1 (en) * 2019-12-03 2021-06-10 Hewlett-Packard Development Company, L.P. Intent addition for a chatbot
US11444893B1 (en) * 2019-12-13 2022-09-13 Wells Fargo Bank, N.A. Enhanced chatbot responses during conversations with unknown users based on maturity metrics determined from history of chatbot interactions
US11556826B2 (en) 2020-03-20 2023-01-17 Adobe Inc. Generating hyper-parameters for machine learning models using modified Bayesian optimization based on accuracy and training efficiency
US12106197B2 (en) * 2020-03-25 2024-10-01 International Business Machines Corporation Learning parameter sampling configuration for automated machine learning

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115827171A (zh) * 2023-01-31 2023-03-21 阿里巴巴达摩院(杭州)科技有限公司 云端调参系统、调参方法及调参系统
CN119536835A (zh) * 2023-08-30 2025-02-28 华为技术有限公司 一种参数调优方法、装置以及设备
CN116991429A (zh) * 2023-09-28 2023-11-03 之江实验室 计算机程序的编译调优方法、装置和存储介质
CN116991429B (zh) * 2023-09-28 2024-01-16 之江实验室 计算机程序的编译调优方法、装置和存储介质
CN118626918A (zh) * 2024-08-14 2024-09-10 杭州迪普科技股份有限公司 一种基于人工智能的数据分类分级方法
CN118626918B (zh) * 2024-08-14 2024-11-08 杭州迪普科技股份有限公司 一种基于人工智能的数据分类分级方法

Also Published As

Publication number Publication date
US12405975B2 (en) 2025-09-02
WO2021202573A1 (en) 2021-10-07
EP4127963A1 (en) 2023-02-08
JP2023520415A (ja) 2023-05-17
JP2023520425A (ja) 2023-05-17
WO2021202576A1 (en) 2021-10-07
CN115398418A (zh) 2022-11-25
JP7674384B2 (ja) 2025-05-09
US20210304003A1 (en) 2021-09-30
EP4127962A1 (en) 2023-02-08
JP7692432B2 (ja) 2025-06-13
US20210304074A1 (en) 2021-09-30

Similar Documents

Publication Publication Date Title
US12405975B2 (en) Method and system for constraint based hyperparameter tuning
US12249314B2 (en) Routing for chatbots
JP7851913B2 (ja) テキスト分類についての説明を与えるための技術
CN115485690A (zh) 用于处置聊天机器人的不平衡训练数据的分批技术
JP2024503517A (ja) 自然言語処理のための多因子モデリング
US12518129B2 (en) Method and system for over-prediction in neural networks
EP4281880A1 (en) Multi-feature balancing for natural language processors
KR102821062B1 (ko) 사전-트레이닝된 언어 모델들에 대한 긴 텍스트를 핸들링하기 위한 시스템 및 기술들
US20240086767A1 (en) Continuous hyper-parameter tuning with automatic domain weight adjustment based on periodic performance checkpoints
US20240095584A1 (en) Objective function optimization in target based hyperparameter tuning
CN116235164B (zh) 聊天机器人的范围外自动转变
WO2023091436A1 (en) System and techniques for handling long text for pre-trained language models
CN116235164A (zh) 聊天机器人的范围外自动转变

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination