JP7674384B2 - ターゲットに基づくハイパーパラメータチューニングのための方法およびシステム - Google Patents

ターゲットに基づくハイパーパラメータチューニングのための方法およびシステム Download PDF

Info

Publication number
JP7674384B2
JP7674384B2 JP2022559629A JP2022559629A JP7674384B2 JP 7674384 B2 JP7674384 B2 JP 7674384B2 JP 2022559629 A JP2022559629 A JP 2022559629A JP 2022559629 A JP2022559629 A JP 2022559629A JP 7674384 B2 JP7674384 B2 JP 7674384B2
Authority
JP
Japan
Prior art keywords
machine learning
learning model
metric
score
hyperparameters
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022559629A
Other languages
English (en)
Japanese (ja)
Other versions
JP2023520415A5 (cg-RX-API-DMAC7.html
JP2023520415A (ja
Inventor
ジョンソン,マーク・エドワード
ドゥオング,タン・ロング
ビシュノイ,ビシャル
ビナコタ,シュリニバス
ファム,トーマス
ホアン,コン・ズイ・ブー
Original Assignee
オラクル・インターナショナル・コーポレイション
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オラクル・インターナショナル・コーポレイション filed Critical オラクル・インターナショナル・コーポレイション
Publication of JP2023520415A publication Critical patent/JP2023520415A/ja
Publication of JP2023520415A5 publication Critical patent/JP2023520415A5/ja
Application granted granted Critical
Publication of JP7674384B2 publication Critical patent/JP7674384B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G06F18/2178—Validation; Performance evaluation; Active pattern learning techniques based on feedback of a supervisor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0985—Hyperparameter optimisation; Meta-learning; Learning-to-learn
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/02—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/12—Computing arrangements based on biological models using genetic models
    • G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Medical Informatics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Machine Translation (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
JP2022559629A 2020-03-30 2021-03-30 ターゲットに基づくハイパーパラメータチューニングのための方法およびシステム Active JP7674384B2 (ja)

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
US202063002159P 2020-03-30 2020-03-30
US63/002,159 2020-03-30
US202063119577P 2020-11-30 2020-11-30
US63/119,577 2020-11-30
US17/216,496 US12405975B2 (en) 2020-03-30 2021-03-29 Method and system for constraint based hyperparameter tuning
US17/216,498 US20210304074A1 (en) 2020-03-30 2021-03-29 Method and system for target based hyper-parameter tuning
US17/216,498 2021-03-29
US17/216,496 2021-03-29
PCT/US2021/024953 WO2021202576A1 (en) 2020-03-30 2021-03-30 Method and system for target based hyper-parameter tuning

Publications (3)

Publication Number Publication Date
JP2023520415A JP2023520415A (ja) 2023-05-17
JP2023520415A5 JP2023520415A5 (cg-RX-API-DMAC7.html) 2024-03-27
JP7674384B2 true JP7674384B2 (ja) 2025-05-09

Family

ID=77856190

Family Applications (2)

Application Number Title Priority Date Filing Date
JP2022559629A Active JP7674384B2 (ja) 2020-03-30 2021-03-30 ターゲットに基づくハイパーパラメータチューニングのための方法およびシステム
JP2022559647A Active JP7692432B2 (ja) 2020-03-30 2021-03-30 制約に基づくハイパーパラメータチューニングのための方法およびシステム

Family Applications After (1)

Application Number Title Priority Date Filing Date
JP2022559647A Active JP7692432B2 (ja) 2020-03-30 2021-03-30 制約に基づくハイパーパラメータチューニングのための方法およびシステム

Country Status (5)

Country Link
US (2) US12405975B2 (cg-RX-API-DMAC7.html)
EP (2) EP4127963A1 (cg-RX-API-DMAC7.html)
JP (2) JP7674384B2 (cg-RX-API-DMAC7.html)
CN (2) CN115398419A (cg-RX-API-DMAC7.html)
WO (2) WO2021202573A1 (cg-RX-API-DMAC7.html)

Families Citing this family (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2569335B (en) * 2017-12-13 2022-07-27 Sage Global Services Ltd Chatbot system
US11463386B2 (en) * 2020-09-23 2022-10-04 Capital One Services, Llc Systems and methods for generating conversational responses using machine learning models
US12664359B1 (en) * 2021-04-14 2026-06-23 Wells Fargo Bank, N.A. Conversation orchestration system with natural language processing and response handling
US20230034136A1 (en) * 2021-07-30 2023-02-02 Kabushiki Kaisha Toshiba System and method for scheduling communication within a distributed learning and deployment framework
US12705430B1 (en) * 2021-10-15 2026-08-11 NTT DATA Services, LLC Intelligent process automation for omnichannel user interaction
US11870651B2 (en) * 2021-11-29 2024-01-09 Sap Se Landscape model verification system
CA3247360A1 (en) * 2022-04-05 2023-10-12 Thrive Technologies, Inc INVENTORY OPTIMIZATION SYSTEMS AND METHODS
CN114897617A (zh) * 2022-05-18 2022-08-12 北京百度网讯科技有限公司 金融风控场景的模型评估方法、装置、设备以及存储介质
US12705501B2 (en) * 2022-05-19 2026-08-11 Google Llc Determining hyperparameters using sequence generation neural networks
US12026254B2 (en) * 2022-06-17 2024-07-02 Optum, Inc. Prediction model selection for cyber security
CN115277073B (zh) * 2022-06-20 2024-02-06 北京邮电大学 信道传输的方法、装置、电子设备及介质
US12517159B2 (en) * 2022-08-04 2026-01-06 Viasat, Inc. Machine learning based tuning of radio frequency apparatuses
CN115511186A (zh) * 2022-09-29 2022-12-23 苏州浪潮智能科技有限公司 一种深度学习训练时长的预测管理方法、装置及设备
US12277396B2 (en) * 2022-09-30 2025-04-15 Tenyx, Inc. Assessing and improving the deployment of large language models in specific domains
CN117035003B (zh) * 2022-09-30 2026-08-11 腾讯科技(深圳)有限公司 超参数优化方法、装置、计算设备、存储介质及程序产品
KR20240103576A (ko) * 2022-12-27 2024-07-04 주식회사 모빌린트 딥러닝 컴파일러 최적화 성능 모니터링 장치 및 방법
CN115827171B (zh) * 2023-01-31 2023-05-23 阿里巴巴达摩院(杭州)科技有限公司 云端调参系统、调参方法及调参系统
GB202302321D0 (en) * 2023-02-17 2023-04-05 Samsung Electronics Co Ltd Methods and apparatus for Ai/Ml model monitoring
CN118821957A (zh) * 2023-04-18 2024-10-22 自动化机器学习有限公司 用于运行预测引擎的方法和装置
US20240362487A1 (en) * 2023-04-26 2024-10-31 Samsung Electronics Co., Ltd. Systems and methods for tuning parameters of a machine learning model for federated learning
US12008409B1 (en) * 2023-05-02 2024-06-11 The Strategic Coach Inc. Apparatus and a method for determining resource distribution
CN116301282B (zh) * 2023-05-16 2023-08-01 中诚华隆计算机技术有限公司 一种多核处理器芯片的低功耗控制方法和装置
US12231378B2 (en) * 2023-06-08 2025-02-18 Sap Se Realtime conversation AI insights and deployment
CN116738239B (zh) * 2023-08-11 2023-11-24 浙江菜鸟供应链管理有限公司 模型训练方法、资源调度方法及装置、系统、设备及介质
US12694946B2 (en) * 2023-08-30 2026-07-28 International Business Machines Corporation Machine learning model distillation for protein design
CN117348942A (zh) * 2023-08-30 2024-01-05 华为技术有限公司 一种参数调优方法、装置以及设备
US20250103908A1 (en) * 2023-09-21 2025-03-27 International Business Machines Corporation Dynamic Selection of AI Computer Models to Reduce Costs and Maximize User Experience
CN116991429B (zh) * 2023-09-28 2024-01-16 之江实验室 计算机程序的编译调优方法、装置和存储介质
US12293265B1 (en) 2024-01-10 2025-05-06 The Strategic Coach Inc. Apparatus and method for model optimization
US12400149B1 (en) 2024-04-22 2025-08-26 Sas Institute Inc. Systems and methods for parallel exploration of a hyperparameter search space
KR20250165031A (ko) * 2024-05-17 2025-11-25 주식회사 Lg 경영개발원 복수의 모델을 포함하는 인공지능 모델을 응용한 llm 기반의 ai 에이전트 제공 방법 및 그 시스템
US20250383653A1 (en) * 2024-06-17 2025-12-18 Rockwell Automation Technologies, Inc. Hyperparameter tuning for generative artificial intelligence prompt engineering
CN118626918B (zh) * 2024-08-14 2024-11-08 杭州迪普科技股份有限公司 一种基于人工智能的数据分类分级方法
US12367342B1 (en) * 2025-01-15 2025-07-22 Conversational AI Ltd Automated analysis of computerized conversational agent conversational data

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016042322A (ja) 2014-08-19 2016-03-31 日本電気株式会社 データ分析装置、分析方法とそのプログラム
US20200065705A1 (en) 2018-08-23 2020-02-27 SigOpt, Inc. Systems and methods for implementing an intelligent machine learning optimization platform for multiple tuning criteria

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9152926B2 (en) * 2012-02-02 2015-10-06 Arizona Board Of Regents On Behalf Of Arizona State University Systems, methods, and media for updating a classifier
US9330362B2 (en) * 2013-05-15 2016-05-03 Microsoft Technology Licensing, Llc Tuning hyper-parameters of a computer-executable learning algorithm
US10558935B2 (en) 2013-11-22 2020-02-11 California Institute Of Technology Weight benefit evaluator for training data
US20160132787A1 (en) * 2014-11-11 2016-05-12 Massachusetts Institute Of Technology Distributed, multi-model, self-learning platform for machine learning
US11070525B2 (en) 2016-04-15 2021-07-20 Adris Chakraborty Method and system of privacy enablement in a family networking computing platform
US10417566B2 (en) 2016-05-22 2019-09-17 Microsoft Technology Licensing, Llc Self-learning technique for training a PDA component and a simulated user component
CN109791631A (zh) * 2016-08-25 2019-05-21 谷歌有限责任公司 奖励增强模型训练
US10572823B1 (en) * 2016-12-13 2020-02-25 Ca, Inc. Optimizing a malware detection model using hyperparameters
EP4700664A3 (en) 2017-05-17 2026-04-22 Intel Corporation Systems and methods implementing an intelligent optimization platform
US20190019108A1 (en) 2017-07-13 2019-01-17 General Electric Company Systems and methods for a validation tree
WO2019028468A1 (en) 2017-08-04 2019-02-07 Fair Ip, Llc COMPUTER SYSTEM FOR CONSTRUCTING, TRAINING AND PRODUCING AUTOMATIC LEARNING MODELS
JP7065368B2 (ja) 2017-11-17 2022-05-12 パナソニックIpマネジメント株式会社 情報処理方法および情報処理システム
US11270228B2 (en) 2017-11-17 2022-03-08 Panasonic Intellectual Property Management Co., Ltd. Information processing method and information processing system
US20190236487A1 (en) 2018-01-30 2019-08-01 Microsoft Technology Licensing, Llc Machine learning hyperparameter tuning tool
US10860629B1 (en) 2018-04-02 2020-12-08 Amazon Technologies, Inc. Task-oriented dialog systems utilizing combined supervised and reinforcement learning
US10832002B2 (en) * 2018-05-08 2020-11-10 International Business Machines Corporation System and method for scoring performance of chatbots
US10635939B2 (en) * 2018-07-06 2020-04-28 Capital One Services, Llc System, method, and computer-accessible medium for evaluating multi-dimensional synthetic data using integrated variants analysis
EP3620996A1 (en) 2018-09-04 2020-03-11 Siemens Aktiengesellschaft Transfer learning of a machine-learning model using a hyperparameter response model
CN109657041B (zh) * 2018-12-04 2023-09-29 南京理工大学 基于深度学习的问题自动生成方法
US11347965B2 (en) 2019-03-21 2022-05-31 Illumina, Inc. Training data generation for artificial intelligence-based sequencing
US12299541B2 (en) * 2019-05-22 2025-05-13 Adobe Inc. Model insights framework for providing insight based on model evaluations to optimize machine learning models
CN110298436B (zh) * 2019-06-28 2023-05-09 乐山金蜜工业卫士服务股份有限公司 基于指针生成网络的数据到文本的生成模型
WO2021112822A1 (en) * 2019-12-03 2021-06-10 Hewlett-Packard Development Company, L.P. Intent addition for a chatbot
US11444893B1 (en) * 2019-12-13 2022-09-13 Wells Fargo Bank, N.A. Enhanced chatbot responses during conversations with unknown users based on maturity metrics determined from history of chatbot interactions
US11556826B2 (en) * 2020-03-20 2023-01-17 Adobe Inc. Generating hyper-parameters for machine learning models using modified Bayesian optimization based on accuracy and training efficiency
US12106197B2 (en) * 2020-03-25 2024-10-01 International Business Machines Corporation Learning parameter sampling configuration for automated machine learning

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016042322A (ja) 2014-08-19 2016-03-31 日本電気株式会社 データ分析装置、分析方法とそのプログラム
US20200065705A1 (en) 2018-08-23 2020-02-27 SigOpt, Inc. Systems and methods for implementing an intelligent machine learning optimization platform for multiple tuning criteria

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
伊藤 健太郎 ほか,サポートベクトル回帰におけるハイパーパラメータの最適化法,電子情報通信学会技術研究報告 IEICE Technical Report,日本,社団法人電子情報通信学会,2003年01月16日,Vol.102,No.508,pp.7-12

Also Published As

Publication number Publication date
CN115398418A (zh) 2022-11-25
JP7692432B2 (ja) 2025-06-13
US20210304074A1 (en) 2021-09-30
JP2023520415A (ja) 2023-05-17
WO2021202573A1 (en) 2021-10-07
EP4127962A1 (en) 2023-02-08
EP4127963A1 (en) 2023-02-08
JP2023520425A (ja) 2023-05-17
US20210304003A1 (en) 2021-09-30
WO2021202576A1 (en) 2021-10-07
US12405975B2 (en) 2025-09-02
CN115398419A (zh) 2022-11-25

Similar Documents

Publication Publication Date Title
JP7692432B2 (ja) 制約に基づくハイパーパラメータチューニングのための方法およびシステム
US12236321B2 (en) Batching techniques for handling unbalanced training data for a chatbot
US12249314B2 (en) Routing for chatbots
JP7851913B2 (ja) テキスト分類についての説明を与えるための技術
US12288550B2 (en) Framework for focused training of language models and techniques for end-to-end hypertuning of the framework
JP7692482B2 (ja) ニューラルネットワークにおける過剰予測のための方法およびシステム
JP7771196B2 (ja) 自然言語プロセッサのための複数特徴均衡化
JP2024503517A (ja) 自然言語処理のための多因子モデリング
JP2023544328A (ja) チャットボットの自動スコープ外遷移
US20210319360A1 (en) Fast and scalable multi-tenant serve pool for chatbots
JP2024541762A (ja) 事前トレーニングされた言語モデルのための長いテキストを処理するシステムおよび技術
US20240086767A1 (en) Continuous hyper-parameter tuning with automatic domain weight adjustment based on periodic performance checkpoints
US12112560B2 (en) Usage based resource utilization of training pool for chatbots
US20240095584A1 (en) Objective function optimization in target based hyperparameter tuning

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240314

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20240314

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20250129

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20250204

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250312

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20250401

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20250424

R150 Certificate of patent or registration of utility model

Ref document number: 7674384

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150