JP7571167B2 - 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク - Google Patents

自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク Download PDF

Info

Publication number
JP7571167B2
JP7571167B2 JP2023010303A JP2023010303A JP7571167B2 JP 7571167 B2 JP7571167 B2 JP 7571167B2 JP 2023010303 A JP2023010303 A JP 2023010303A JP 2023010303 A JP2023010303 A JP 2023010303A JP 7571167 B2 JP7571167 B2 JP 7571167B2
Authority
JP
Japan
Prior art keywords
test
network
neural network
test case
inputs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2023010303A
Other languages
English (en)
Japanese (ja)
Other versions
JP2023109726A (ja
Inventor
イーサン・ホセアン・ペレス
サフロン・シャン・フアン
ナサニエル・ジョン・マッカリース-パーク
ジェフリー・アーヴィング
Original Assignee
ディープマインド テクノロジーズ リミテッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ディープマインド テクノロジーズ リミテッド filed Critical ディープマインド テクノロジーズ リミテッド
Publication of JP2023109726A publication Critical patent/JP2023109726A/ja
Priority to JP2024177329A priority Critical patent/JP2025020137A/ja
Application granted granted Critical
Publication of JP7571167B2 publication Critical patent/JP7571167B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G06V30/1916—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Prevention of errors by analysis, debugging or testing of software
    • G06F11/3668—Testing of software
    • G06F11/3672—Test management
    • G06F11/3684—Test management for test design, e.g. generating new test cases
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Prevention of errors by analysis, debugging or testing of software
    • G06F11/3668—Testing of software
    • G06F11/3672—Test management
    • G06F11/3688—Test management for test execution, e.g. scheduling of test suites
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/092—Reinforcement learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/1895—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for short real-time information, e.g. alarms, notifications, alerts, updates
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/02—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Machine Translation (AREA)
  • Debugging And Monitoring (AREA)
JP2023010303A 2022-01-27 2023-01-26 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク Active JP7571167B2 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2024177329A JP2025020137A (ja) 2022-01-27 2024-10-09 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202263303958P 2022-01-27 2022-01-27
US63/303,958 2022-01-27

Related Child Applications (1)

Application Number Title Priority Date Filing Date
JP2024177329A Division JP2025020137A (ja) 2022-01-27 2024-10-09 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク

Publications (2)

Publication Number Publication Date
JP2023109726A JP2023109726A (ja) 2023-08-08
JP7571167B2 true JP7571167B2 (ja) 2024-10-22

Family

ID=86114565

Family Applications (2)

Application Number Title Priority Date Filing Date
JP2023010303A Active JP7571167B2 (ja) 2022-01-27 2023-01-26 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク
JP2024177329A Pending JP2025020137A (ja) 2022-01-27 2024-10-09 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク

Family Applications After (1)

Application Number Title Priority Date Filing Date
JP2024177329A Pending JP2025020137A (ja) 2022-01-27 2024-10-09 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク

Country Status (4)

Country Link
US (2) US12327421B2 (cg-RX-API-DMAC7.html)
JP (2) JP7571167B2 (cg-RX-API-DMAC7.html)
KR (1) KR20230115917A (cg-RX-API-DMAC7.html)
CN (2) CN116049003B (cg-RX-API-DMAC7.html)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12596890B2 (en) * 2023-03-30 2026-04-07 Salesforce, Inc. Systems and methods for cross-lingual transfer learning
US12346243B2 (en) * 2023-06-13 2025-07-01 Sap Se Intelligent digital assistant for software testing automation
US20250004910A1 (en) * 2023-06-27 2025-01-02 Microsoft Technology Licensing, Llc Automated software testing using natural language-based form completion
US20250077552A1 (en) * 2023-08-30 2025-03-06 Htc Corporation Data classification method for filtering outlier text data
US12499241B2 (en) 2023-09-06 2025-12-16 The Toronto-Dominion Bank Correcting security vulnerabilities with generative artificial intelligence
US12517812B2 (en) 2023-09-06 2026-01-06 The Toronto-Dominion Bank Security testing based on generative artificial intelligence
GB2634926A (en) * 2023-10-26 2025-04-30 Univ Nanyang Tech Method and system for diagnosing machine learning model underperformance or failure
US20250147869A1 (en) 2023-11-03 2025-05-08 Ropes AI Inc Automated computer code generation
US20250259073A1 (en) 2024-02-14 2025-08-14 Deepmind Technologies Limited Reinforcement learning through preference feedback
CN119201748B (zh) * 2024-11-20 2025-03-14 长江证券股份有限公司 测试用例生成方法、装置、设备及可读存储介质
EP4790571A1 (en) 2025-02-05 2026-08-12 Fujitsu Limited Computer program, response-generation support method, and information processing apparatus

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018063504A (ja) 2016-10-12 2018-04-19 株式会社リコー 生成モデル学習方法、装置及びプログラム
JP2019125014A (ja) 2018-01-12 2019-07-25 コニカミノルタ株式会社 学習装置、学習方法、および学習プログラム
US20200073788A1 (en) 2018-08-30 2020-03-05 International Business Machines Corporation Automated test case generation for deep neural networks and other model-based artificial intelligence systems
JP2020112915A (ja) 2019-01-09 2020-07-27 株式会社Nttドコモ データ生成装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10338993B1 (en) * 2018-04-22 2019-07-02 Sas Institute Inc. Analysis of failures in combinatorial test suite
US11182566B2 (en) * 2016-10-03 2021-11-23 Google Llc Processing text sequences using neural networks
CN107861880B (zh) * 2017-11-28 2021-06-22 曲明成 基于递归在线算法的进程任务块可疑依赖关系的发现方法
CN113056743B (zh) * 2018-09-20 2024-06-28 辉达公司 训练神经网络以用于车辆重新识别
US11138469B2 (en) * 2019-01-15 2021-10-05 Naver Corporation Training and using a convolutional neural network for person re-identification
US11544498B2 (en) * 2020-03-05 2023-01-03 Google Llc Training neural networks using consistency measures
EP4094198A1 (en) * 2020-06-05 2022-11-30 Google LLC Classifying out-of-distribution data using a contrastive loss
US12481891B2 (en) * 2020-10-26 2025-11-25 Google Llc Training neural networks with label differential privacy
CN113836027B (zh) * 2021-09-28 2023-07-21 重庆大学 一种使用生成式网络进行失败测试用例生成的方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018063504A (ja) 2016-10-12 2018-04-19 株式会社リコー 生成モデル学習方法、装置及びプログラム
JP2019125014A (ja) 2018-01-12 2019-07-25 コニカミノルタ株式会社 学習装置、学習方法、および学習プログラム
US20200073788A1 (en) 2018-08-30 2020-03-05 International Business Machines Corporation Automated test case generation for deep neural networks and other model-based artificial intelligence systems
JP2020112915A (ja) 2019-01-09 2020-07-27 株式会社Nttドコモ データ生成装置

Also Published As

Publication number Publication date
CN122526979A (zh) 2026-08-07
KR20230115917A (ko) 2023-08-03
JP2025020137A (ja) 2025-02-12
CN116049003A (zh) 2023-05-02
US20250273001A1 (en) 2025-08-28
CN116049003B (zh) 2026-05-01
US12327421B2 (en) 2025-06-10
US20230237826A1 (en) 2023-07-27
JP2023109726A (ja) 2023-08-08

Similar Documents

Publication Publication Date Title
US12327421B2 (en) Using automatically uncovered failure cases to improve the performance of neural networks
US12423295B2 (en) Text to question-answer model system
US20210150355A1 (en) Training machine learning models using task selection policies to increase learning progress
US20230259707A1 (en) Systems and methods for natural language processing (nlp) model robustness determination
US10083157B2 (en) Text classification and transformation based on author
CN113297366B (zh) 多轮对话的情绪识别模型训练方法、装置、设备及介质
US11847423B2 (en) Dynamic intent classification based on environment variables
US12475330B2 (en) Method for identifying noise samples, electronic device, and storage medium
EP3520036B1 (en) Processing text sequences using neural networks
CN113555006B (zh) 一种语音信息识别方法、装置、电子设备及存储介质
EP3586276A1 (en) Sequence processing using online attention
US20250131915A1 (en) Enhanced speech-to-text performance with mixed languages
US11481609B2 (en) Computationally efficient expressive output layers for neural networks
CN114625874B (zh) 应用于文本识别的模型训练方法、文本识别方法和装置
CN112395857B (zh) 基于对话系统的语音文本处理方法、装置、设备及介质
JP2024519265A (ja) フィードフォワード空間変換ユニットを備えたニューラルネットワーク
CN118095272A (zh) 文本的识别方法、装置、电子设备及存储介质
CN113919354A (zh) 一种文本对抗的自然语言增强处理方法及装置
US20250307546A1 (en) Systems and methods for large language model optimization using prompt structuring
US20250094865A1 (en) Ensuring that language models follow instructions indicated in prompts
Smajić et al. Application of Natural Language Processing Algorithms for Chatbots
US12050875B2 (en) System and method for determining context changes in text
CN115049018A (zh) 一种情绪分析模型的训练方法、装置、设备及介质
US20260105102A1 (en) Accelerated LLM Usage
TW202503578A (zh) 加速生成式人工智慧模型的推理

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20230327

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230327

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20240214

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240311

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240610

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240909

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20241009

R150 Certificate of patent or registration of utility model

Ref document number: 7571167

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313113

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350