JP7571167B2 - 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク - Google Patents

自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク Download PDF

Info

Publication number
JP7571167B2
JP7571167B2 JP2023010303A JP2023010303A JP7571167B2 JP 7571167 B2 JP7571167 B2 JP 7571167B2 JP 2023010303 A JP2023010303 A JP 2023010303A JP 2023010303 A JP2023010303 A JP 2023010303A JP 7571167 B2 JP7571167 B2 JP 7571167B2
Authority
JP
Japan
Prior art keywords
test
network
neural network
test case
inputs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2023010303A
Other languages
English (en)
Japanese (ja)
Other versions
JP2023109726A (ja
Inventor
イーサン・ホセアン・ペレス
サフロン・シャン・フアン
ナサニエル・ジョン・マッカリース－パーク
ジェフリー・アーヴィング
Original Assignee
ディープマインド テクノロジーズ リミテッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ディープマインド テクノロジーズ リミテッド filed Critical ディープマインド テクノロジーズ リミテッド
Publication of JP2023109726A publication Critical patent/JP2023109726A/ja
Priority to JP2024177329A priority Critical patent/JP2025020137A/ja
Application granted granted Critical
Publication of JP7571167B2 publication Critical patent/JP7571167B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G06V30/1916—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Prevention of errors by analysis, debugging or testing of software
    • G06F11/3668—Testing of software
    • G06F11/3672—Test management
    • G06F11/3684—Test management for test design, e.g. generating new test cases
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Prevention of errors by analysis, debugging or testing of software
    • G06F11/3668—Testing of software
    • G06F11/3672—Test management
    • G06F11/3688—Test management for test execution, e.g. scheduling of test suites
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/092—Reinforcement learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/1895—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for short real-time information, e.g. alarms, notifications, alerts, updates
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/02—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Machine Translation (AREA)
  • Debugging And Monitoring (AREA)
JP2023010303A 2022-01-27 2023-01-26 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク Active JP7571167B2 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2024177329A JP2025020137A (ja) 2022-01-27 2024-10-09 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202263303958P 2022-01-27 2022-01-27
US63/303,958 2022-01-27

Related Child Applications (1)

Application Number Title Priority Date Filing Date
JP2024177329A Division JP2025020137A (ja) 2022-01-27 2024-10-09 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク

Publications (2)

Publication Number Publication Date
JP2023109726A JP2023109726A (ja) 2023-08-08
JP7571167B2 true JP7571167B2 (ja) 2024-10-22

Family

ID=86114565

Family Applications (2)

Application Number Title Priority Date Filing Date
JP2023010303A Active JP7571167B2 (ja) 2022-01-27 2023-01-26 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク
JP2024177329A Pending JP2025020137A (ja) 2022-01-27 2024-10-09 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク

Family Applications After (1)

Application Number Title Priority Date Filing Date
JP2024177329A Pending JP2025020137A (ja) 2022-01-27 2024-10-09 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク

Country Status (3)

Country Link
US (2) US12327421B2 (https=)
JP (2) JP7571167B2 (https=)
KR (1) KR20230115917A (https=)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12596890B2 (en) * 2023-03-30 2026-04-07 Salesforce, Inc. Systems and methods for cross-lingual transfer learning
US12346243B2 (en) * 2023-06-13 2025-07-01 Sap Se Intelligent digital assistant for software testing automation
US20250004910A1 (en) * 2023-06-27 2025-01-02 Microsoft Technology Licensing, Llc Automated software testing using natural language-based form completion
JP2025036355A (ja) * 2023-08-30 2025-03-14 宏達國際電子股▲ふん▼有限公司 外れた文字データをスクリーニングするためのデータ分類方法
US12517812B2 (en) 2023-09-06 2026-01-06 The Toronto-Dominion Bank Security testing based on generative artificial intelligence
US12499241B2 (en) 2023-09-06 2025-12-16 The Toronto-Dominion Bank Correcting security vulnerabilities with generative artificial intelligence
GB2634926A (en) * 2023-10-26 2025-04-30 Univ Nanyang Tech Method and system for diagnosing machine learning model underperformance or failure
US12596628B2 (en) 2023-11-03 2026-04-07 Ropes AI Inc Automated computer code timeline generation
US20250259073A1 (en) * 2024-02-14 2025-08-14 Deepmind Technologies Limited Reinforcement learning through preference feedback
CN119201748B (zh) * 2024-11-20 2025-03-14 长江证券股份有限公司 测试用例生成方法、装置、设备及可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018063504A (ja) 2016-10-12 2018-04-19 株式会社リコー 生成モデル学習方法、装置及びプログラム
JP2019125014A (ja) 2018-01-12 2019-07-25 コニカミノルタ株式会社 学習装置、学習方法、および学習プログラム
US20200073788A1 (en) 2018-08-30 2020-03-05 International Business Machines Corporation Automated test case generation for deep neural networks and other model-based artificial intelligence systems
JP2020112915A (ja) 2019-01-09 2020-07-27 株式会社Ｎｔｔドコモ データ生成装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11455807B2 (en) * 2018-09-20 2022-09-27 Nvidia Corporation Training neural networks for vehicle re-identification
US11138469B2 (en) * 2019-01-15 2021-10-05 Naver Corporation Training and using a convolutional neural network for person re-identification
US12481891B2 (en) * 2020-10-26 2025-11-25 Google Llc Training neural networks with label differential privacy

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018063504A (ja) 2016-10-12 2018-04-19 株式会社リコー 生成モデル学習方法、装置及びプログラム
JP2019125014A (ja) 2018-01-12 2019-07-25 コニカミノルタ株式会社 学習装置、学習方法、および学習プログラム
US20200073788A1 (en) 2018-08-30 2020-03-05 International Business Machines Corporation Automated test case generation for deep neural networks and other model-based artificial intelligence systems
JP2020112915A (ja) 2019-01-09 2020-07-27 株式会社Ｎｔｔドコモ データ生成装置

Also Published As

Publication number Publication date
US20250273001A1 (en) 2025-08-28
CN116049003A (zh) 2023-05-02
US20230237826A1 (en) 2023-07-27
JP2025020137A (ja) 2025-02-12
KR20230115917A (ko) 2023-08-03
US12327421B2 (en) 2025-06-10
JP2023109726A (ja) 2023-08-08

Similar Documents

Publication Publication Date Title
JP7571167B2 (ja) 自動的に発見される不合格ケースを使用した、性能の向上したニューラルネットワーク
US20210150355A1 (en) Training machine learning models using task selection policies to increase learning progress
US12423295B2 (en) Text to question-answer model system
US10083157B2 (en) Text classification and transformation based on author
CN113297366B (zh) 多轮对话的情绪识别模型训练方法、装置、设备及介质
US20230259707A1 (en) Systems and methods for natural language processing (nlp) model robustness determination
US11847423B2 (en) Dynamic intent classification based on environment variables
US12475330B2 (en) Method for identifying noise samples, electronic device, and storage medium
EP3586276A1 (en) Sequence processing using online attention
US11481609B2 (en) Computationally efficient expressive output layers for neural networks
US20250131915A1 (en) Enhanced speech-to-text performance with mixed languages
CN112395857B (zh) 基于对话系统的语音文本处理方法、装置、设备及介质
KR20190074508A (ko) 챗봇을 위한 대화 모델의 데이터 크라우드소싱 방법
JP7596559B2 (ja) フィードフォワード空間変換ユニットを備えたニューラルネットワーク
CN113919354A (zh) 一种文本对抗的自然语言增强处理方法及装置
US20250094865A1 (en) Ensuring that language models follow instructions indicated in prompts
CN116049003B (zh) 使用自动发现的失败用例提高神经网络的性能
Smajić et al. Application of Natural Language Processing Algorithms for Chatbots
US20230267276A1 (en) System and method for determining context changes in text
CN115049018A (zh) 一种情绪分析模型的训练方法、装置、设备及介质
CN114625874A (zh) 应用于文本识别的模型训练方法、文本识别方法和装置
US20260105102A1 (en) Accelerated LLM Usage
US20250307546A1 (en) Systems and methods for large language model optimization using prompt structuring
Harrison et al. Testing Topological NLP Transformers on Text Message Data to Determine the Best Online Phishing Detection Technique
TW202503578A (zh) 加速生成式人工智慧模型的推理

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20230327

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230327

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20240214

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240311

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240610

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240909

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20241009

R150 Certificate of patent or registration of utility model

Ref document number: 7571167

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313113

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350