CN116049003B - 使用自动发现的失败用例提高神经网络的性能 - Google Patents

使用自动发现的失败用例提高神经网络的性能

Info

Publication number
CN116049003B
CN116049003B CN202310088005.3A CN202310088005A CN116049003B CN 116049003 B CN116049003 B CN 116049003B CN 202310088005 A CN202310088005 A CN 202310088005A CN 116049003 B CN116049003 B CN 116049003B
Authority
CN
China
Prior art keywords
test
network
neural network
test case
input
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310088005.3A
Other languages
English (en)
Chinese (zh)
Other versions
CN116049003A (zh
Inventor
伊桑·约瑟安·佩雷斯
萨弗朗·山·黄
纳塔涅尔·约翰·麦卡利斯-帕克
杰弗里·伊尔温
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
GDM Holding LLC
Original Assignee
DeepMind Technologies Ltd
GDM Holding LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by DeepMind Technologies Ltd, GDM Holding LLC filed Critical DeepMind Technologies Ltd
Priority to CN202610461919.3A priority Critical patent/CN122526979A/zh
Publication of CN116049003A publication Critical patent/CN116049003A/zh
Application granted granted Critical
Publication of CN116049003B publication Critical patent/CN116049003B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G06V30/1916—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Prevention of errors by analysis, debugging or testing of software
    • G06F11/3668—Testing of software
    • G06F11/3672—Test management
    • G06F11/3684—Test management for test design, e.g. generating new test cases
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Prevention of errors by analysis, debugging or testing of software
    • G06F11/3668—Testing of software
    • G06F11/3672—Test management
    • G06F11/3688—Test management for test execution, e.g. scheduling of test suites
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/092—Reinforcement learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/1895—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for short real-time information, e.g. alarms, notifications, alerts, updates
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/02—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Machine Translation (AREA)
  • Debugging And Monitoring (AREA)
CN202310088005.3A 2022-01-27 2023-01-28 使用自动发现的失败用例提高神经网络的性能 Active CN116049003B (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202610461919.3A CN122526979A (zh) 2022-01-27 2023-01-28 使用自动发现的失败用例提高神经网络的性能

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202263303958P 2022-01-27 2022-01-27
US63/303,958 2022-01-27

Related Child Applications (1)

Application Number Title Priority Date Filing Date
CN202610461919.3A Division CN122526979A (zh) 2022-01-27 2023-01-28 使用自动发现的失败用例提高神经网络的性能

Publications (2)

Publication Number Publication Date
CN116049003A CN116049003A (zh) 2023-05-02
CN116049003B true CN116049003B (zh) 2026-05-01

Family

ID=86114565

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202310088005.3A Active CN116049003B (zh) 2022-01-27 2023-01-28 使用自动发现的失败用例提高神经网络的性能
CN202610461919.3A Pending CN122526979A (zh) 2022-01-27 2023-01-28 使用自动发现的失败用例提高神经网络的性能

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN202610461919.3A Pending CN122526979A (zh) 2022-01-27 2023-01-28 使用自动发现的失败用例提高神经网络的性能

Country Status (4)

Country Link
US (2) US12327421B2 (cg-RX-API-DMAC7.html)
JP (2) JP7571167B2 (cg-RX-API-DMAC7.html)
KR (1) KR20230115917A (cg-RX-API-DMAC7.html)
CN (2) CN116049003B (cg-RX-API-DMAC7.html)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12596890B2 (en) * 2023-03-30 2026-04-07 Salesforce, Inc. Systems and methods for cross-lingual transfer learning
US12346243B2 (en) * 2023-06-13 2025-07-01 Sap Se Intelligent digital assistant for software testing automation
US20250004910A1 (en) * 2023-06-27 2025-01-02 Microsoft Technology Licensing, Llc Automated software testing using natural language-based form completion
US20250077552A1 (en) * 2023-08-30 2025-03-06 Htc Corporation Data classification method for filtering outlier text data
US12499241B2 (en) 2023-09-06 2025-12-16 The Toronto-Dominion Bank Correcting security vulnerabilities with generative artificial intelligence
US12517812B2 (en) 2023-09-06 2026-01-06 The Toronto-Dominion Bank Security testing based on generative artificial intelligence
GB2634926A (en) * 2023-10-26 2025-04-30 Univ Nanyang Tech Method and system for diagnosing machine learning model underperformance or failure
US20250147869A1 (en) 2023-11-03 2025-05-08 Ropes AI Inc Automated computer code generation
US20250259073A1 (en) 2024-02-14 2025-08-14 Deepmind Technologies Limited Reinforcement learning through preference feedback
CN119201748B (zh) * 2024-11-20 2025-03-14 长江证券股份有限公司 测试用例生成方法、装置、设备及可读存储介质
EP4790571A1 (en) 2025-02-05 2026-08-12 Fujitsu Limited Computer program, response-generation support method, and information processing apparatus

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109891437A (zh) * 2016-10-03 2019-06-14 谷歌有限责任公司 使用神经网络处理文本序列

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10338993B1 (en) * 2018-04-22 2019-07-02 Sas Institute Inc. Analysis of failures in combinatorial test suite
JP2018063504A (ja) 2016-10-12 2018-04-19 株式会社リコー 生成モデル学習方法、装置及びプログラム
CN107861880B (zh) * 2017-11-28 2021-06-22 曲明成 基于递归在线算法的进程任务块可疑依赖关系的发现方法
JP2019125014A (ja) 2018-01-12 2019-07-25 コニカミノルタ株式会社 学習装置、学習方法、および学習プログラム
US10956310B2 (en) 2018-08-30 2021-03-23 International Business Machines Corporation Automated test case generation for deep neural networks and other model-based artificial intelligence systems
CN113056743B (zh) * 2018-09-20 2024-06-28 辉达公司 训练神经网络以用于车辆重新识别
JP7103957B2 (ja) 2019-01-09 2022-07-20 株式会社Nttドコモ データ生成装置
US11138469B2 (en) * 2019-01-15 2021-10-05 Naver Corporation Training and using a convolutional neural network for person re-identification
US11544498B2 (en) * 2020-03-05 2023-01-03 Google Llc Training neural networks using consistency measures
EP4094198A1 (en) * 2020-06-05 2022-11-30 Google LLC Classifying out-of-distribution data using a contrastive loss
US12481891B2 (en) * 2020-10-26 2025-11-25 Google Llc Training neural networks with label differential privacy
CN113836027B (zh) * 2021-09-28 2023-07-21 重庆大学 一种使用生成式网络进行失败测试用例生成的方法

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109891437A (zh) * 2016-10-03 2019-06-14 谷歌有限责任公司 使用神经网络处理文本序列

Also Published As

Publication number Publication date
CN122526979A (zh) 2026-08-07
KR20230115917A (ko) 2023-08-03
JP2025020137A (ja) 2025-02-12
CN116049003A (zh) 2023-05-02
US20250273001A1 (en) 2025-08-28
US12327421B2 (en) 2025-06-10
US20230237826A1 (en) 2023-07-27
JP2023109726A (ja) 2023-08-08
JP7571167B2 (ja) 2024-10-22

Similar Documents

Publication Publication Date Title
US12327421B2 (en) Using automatically uncovered failure cases to improve the performance of neural networks
US11080589B2 (en) Sequence processing using online attention
CN113723105B (zh) 语义特征提取模型的训练方法、装置、设备及存储介质
US20250190524A1 (en) Training machine learning models using unsupervised data augmentation
US20220129621A1 (en) Bert-based machine-learning tool for predicting emotional response to text
US9818409B2 (en) Context-dependent modeling of phonemes
US11636272B2 (en) Hybrid natural language understanding
US11847423B2 (en) Dynamic intent classification based on environment variables
US20210271810A1 (en) Proficiency and native language-adapted grammatical error correction
US11182566B2 (en) Processing text sequences using neural networks
US12608616B2 (en) Attention neural networks with linear units
US11481609B2 (en) Computationally efficient expressive output layers for neural networks
CN113555006B (zh) 一种语音信息识别方法、装置、电子设备及存储介质
CN118974739A (zh) 具有并行注意力层和前馈层的注意力神经网络
CN120476406A (zh) 大型神经网络的改进的训练
CN119358539B (zh) 口语化文本生成方法、装置、设备及存储介质
CN119226485B (zh) 一种基于大语言模型的可控式模拟谈话方法和系统
CN120764484A (zh) 对话大模型流式输出内容安全修正方法及相关装置
US20220108174A1 (en) Training neural networks using auxiliary task update decomposition
CN119808942A (zh) 对话内容的生成方法、装置、设备、存储介质及程序产品

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20251215

Address after: U.S.A.

Applicant after: GDM Holdings LLC

Country or region after: U.S.A.

Address before: Britain

Applicant before: DEEPMIND TECHNOLOGIES Ltd.

Country or region before: United Kingdom

GR01 Patent grant
GR01 Patent grant