KR20230115917A - 자동으로 발견된 실패 케이스들을 사용하여 신경 네트워크의 성능 개선하기 - Google Patents

자동으로 발견된 실패 케이스들을 사용하여 신경 네트워크의 성능 개선하기 Download PDF

Info

Publication number
KR20230115917A
KR20230115917A KR1020230010019A KR20230010019A KR20230115917A KR 20230115917 A KR20230115917 A KR 20230115917A KR 1020230010019 A KR1020230010019 A KR 1020230010019A KR 20230010019 A KR20230010019 A KR 20230010019A KR 20230115917 A KR20230115917 A KR 20230115917A
Authority
KR
South Korea
Prior art keywords
test
network
neural network
test case
inputs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
KR1020230010019A
Other languages
English (en)
Korean (ko)
Inventor
이단 조신 페레즈
사프론 샨 후앙
파크 나다니엘 존 매칼리스
제오프레이 어빙
Original Assignee
딥마인드 테크놀로지스 리미티드
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 딥마인드 테크놀로지스 리미티드 filed Critical 딥마인드 테크놀로지스 리미티드
Publication of KR20230115917A publication Critical patent/KR20230115917A/ko
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G06V30/1916—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Prevention of errors by analysis, debugging or testing of software
    • G06F11/3668—Testing of software
    • G06F11/3672—Test management
    • G06F11/3684—Test management for test design, e.g. generating new test cases
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Prevention of errors by analysis, debugging or testing of software
    • G06F11/3668—Testing of software
    • G06F11/3672—Test management
    • G06F11/3688—Test management for test execution, e.g. scheduling of test suites
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/092—Reinforcement learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/1895—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for short real-time information, e.g. alarms, notifications, alerts, updates
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/02—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Machine Translation (AREA)
  • Debugging And Monitoring (AREA)
KR1020230010019A 2022-01-27 2023-01-26 자동으로 발견된 실패 케이스들을 사용하여 신경 네트워크의 성능 개선하기 Pending KR20230115917A (ko)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202263303958P 2022-01-27 2022-01-27
US63/303,958 2022-01-27

Publications (1)

Publication Number Publication Date
KR20230115917A true KR20230115917A (ko) 2023-08-03

Family

ID=86114565

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020230010019A Pending KR20230115917A (ko) 2022-01-27 2023-01-26 자동으로 발견된 실패 케이스들을 사용하여 신경 네트워크의 성능 개선하기

Country Status (3)

Country Link
US (2) US12327421B2 (https=)
JP (2) JP7571167B2 (https=)
KR (1) KR20230115917A (https=)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12596890B2 (en) * 2023-03-30 2026-04-07 Salesforce, Inc. Systems and methods for cross-lingual transfer learning
US12346243B2 (en) * 2023-06-13 2025-07-01 Sap Se Intelligent digital assistant for software testing automation
US20250004910A1 (en) * 2023-06-27 2025-01-02 Microsoft Technology Licensing, Llc Automated software testing using natural language-based form completion
JP2025036355A (ja) * 2023-08-30 2025-03-14 宏達國際電子股▲ふん▼有限公司 外れた文字データをスクリーニングするためのデータ分類方法
US12517812B2 (en) 2023-09-06 2026-01-06 The Toronto-Dominion Bank Security testing based on generative artificial intelligence
US12499241B2 (en) 2023-09-06 2025-12-16 The Toronto-Dominion Bank Correcting security vulnerabilities with generative artificial intelligence
GB2634926A (en) * 2023-10-26 2025-04-30 Univ Nanyang Tech Method and system for diagnosing machine learning model underperformance or failure
US12596628B2 (en) 2023-11-03 2026-04-07 Ropes AI Inc Automated computer code timeline generation
US20250259073A1 (en) * 2024-02-14 2025-08-14 Deepmind Technologies Limited Reinforcement learning through preference feedback
CN119201748B (zh) * 2024-11-20 2025-03-14 长江证券股份有限公司 测试用例生成方法、装置、设备及可读存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018063504A (ja) * 2016-10-12 2018-04-19 株式会社リコー 生成モデル学習方法、装置及びプログラム
JP2019125014A (ja) * 2018-01-12 2019-07-25 コニカミノルタ株式会社 学習装置、学習方法、および学習プログラム
US10956310B2 (en) * 2018-08-30 2021-03-23 International Business Machines Corporation Automated test case generation for deep neural networks and other model-based artificial intelligence systems
US11455807B2 (en) * 2018-09-20 2022-09-27 Nvidia Corporation Training neural networks for vehicle re-identification
JP7103957B2 (ja) * 2019-01-09 2022-07-20 株式会社Ｎｔｔドコモ データ生成装置
US11138469B2 (en) * 2019-01-15 2021-10-05 Naver Corporation Training and using a convolutional neural network for person re-identification
US12481891B2 (en) * 2020-10-26 2025-11-25 Google Llc Training neural networks with label differential privacy

Also Published As

Publication number Publication date
US20250273001A1 (en) 2025-08-28
JP7571167B2 (ja) 2024-10-22
CN116049003A (zh) 2023-05-02
US20230237826A1 (en) 2023-07-27
JP2025020137A (ja) 2025-02-12
US12327421B2 (en) 2025-06-10
JP2023109726A (ja) 2023-08-08

Similar Documents

Publication Publication Date Title
KR20230115917A (ko) 자동으로 발견된 실패 케이스들을 사용하여 신경 네트워크의 성능 개선하기
US12217180B2 (en) Training text summarization neural networks with an extracted segments prediction objective
US12136037B2 (en) Non-transitory computer-readable storage medium and system for generating an abstractive text summary of a document
US11003863B2 (en) Interactive dialog training and communication system using artificial intelligence
US20220129621A1 (en) Bert-based machine-learning tool for predicting emotional response to text
US12106197B2 (en) Learning parameter sampling configuration for automated machine learning
CN113723105B (zh) 语义特征提取模型的训练方法、装置、设备及存储介质
US20200226324A1 (en) System for providing intelligent part of speech processing of complex natural language
Can et al. “It sounds like...”: A natural language processing approach to detecting counselor reflections in motivational interviewing.
US12254411B2 (en) Attention neural networks with linear units
US20240256964A1 (en) Pretraining Already-Pretrained Models for Diverse Downstream Tasks
US20250131915A1 (en) Enhanced speech-to-text performance with mixed languages
US12367342B1 (en) Automated analysis of computerized conversational agent conversational data
US20210350088A1 (en) Systems and methods for digital document generation using natural language interaction
Perdana et al. Multi-task learning for named entity recognition and intent classification in natural language understanding applications
US20240412039A1 (en) Generating parallel synthetic training data for a machine learning model to predict compliance with rulesets
CN120764484A (zh) 对话大模型流式输出内容安全修正方法及相关装置
CN113849634A (zh) 用于提升深度模型推荐方案可解释性的方法
CN116049003B (zh) 使用自动发现的失败用例提高神经网络的性能
Smajić et al. Application of Natural Language Processing Algorithms for Chatbots
Cooray et al. Machine Learning Approach for Detection of Harassment and Hate Speech in Sinhala Text Messages on Telegram
US20250291929A1 (en) Identifying and handling vulnerable information in log files using large language models
US20250307546A1 (en) Systems and methods for large language model optimization using prompt structuring
Gunnu et al. Coreference Resolution and Text Entailment
WO2026072881A1 (en) Devices, systems, and methods for evaluating an artificial intelligence model

Legal Events

Date Code Title Description
E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

PA0109 Patent application

St.27 status event code: A-0-1-A10-A12-nap-PA0109

PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

P22-X000 Classification modified

St.27 status event code: A-2-2-P10-P22-nap-X000

D21 Rejection of application intended

Free format text: ST27 STATUS EVENT CODE: A-1-2-D10-D21-EXM-PE0902 (AS PROVIDED BY THE NATIONAL OFFICE)

PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902

P11 Amendment of application requested

Free format text: ST27 STATUS EVENT CODE: A-2-2-P10-P11-NAP-X000 (AS PROVIDED BY THE NATIONAL OFFICE)

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

PN2301 Change of applicant

St.27 status event code: A-3-3-R10-R11-asn-PN2301

R11 Change to the name of applicant or owner or transfer of ownership requested

Free format text: ST27 STATUS EVENT CODE: A-3-3-R10-R11-ASN-PN2301 (AS PROVIDED BY THE NATIONAL OFFICE)