KR102592677B1 - 어텐션-기반의 시퀀스 변환 신경망 - Google Patents

어텐션-기반의 시퀀스 변환 신경망 Download PDF

Info

Publication number
KR102592677B1
KR102592677B1 KR1020227033030A KR20227033030A KR102592677B1 KR 102592677 B1 KR102592677 B1 KR 102592677B1 KR 1020227033030 A KR1020227033030 A KR 1020227033030A KR 20227033030 A KR20227033030 A KR 20227033030A KR 102592677 B1 KR102592677 B1 KR 102592677B1
Authority
KR
South Korea
Prior art keywords
output
attention
self
location
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020227033030A
Other languages
English (en)
Korean (ko)
Other versions
KR20220133323A (ko
Inventor
노암 엠. 샤즈엘
에이던 니콜라스 고메즈
루커즈 미치슬라프 카이저
야콥 디. 우슈코리츠
리온 오웬 존스
니키 제이. 파마
일리아 폴로슈킨
아쉬시 테쿠 바스와니
Original Assignee
구글 엘엘씨
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 구글 엘엘씨 filed Critical 구글 엘엘씨
Priority to KR1020237035690A priority Critical patent/KR20230151047A/ko
Publication of KR20220133323A publication Critical patent/KR20220133323A/ko
Application granted granted Critical
Publication of KR102592677B1 publication Critical patent/KR102592677B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
  • Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)
  • Machine Translation (AREA)
  • Color Image Communication Systems (AREA)
  • Facsimile Image Signal Circuits (AREA)
  • Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
KR1020227033030A 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망 Active KR102592677B1 (ko)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020237035690A KR20230151047A (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201762510256P 2017-05-23 2017-05-23
US62/510,256 2017-05-23
US201762541594P 2017-08-04 2017-08-04
US62/541,594 2017-08-04
KR1020207032481A KR102448389B1 (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망
PCT/US2018/034224 WO2018217948A1 (en) 2017-05-23 2018-05-23 Attention-based sequence transduction neural networks

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
KR1020207032481A Division KR102448389B1 (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망

Related Child Applications (1)

Application Number Title Priority Date Filing Date
KR1020237035690A Division KR20230151047A (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망

Publications (2)

Publication Number Publication Date
KR20220133323A KR20220133323A (ko) 2022-10-04
KR102592677B1 true KR102592677B1 (ko) 2023-10-23

Family

ID=62873574

Family Applications (5)

Application Number Title Priority Date Filing Date
KR1020227033030A Active KR102592677B1 (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망
KR1020237035690A Ceased KR20230151047A (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망
KR1020207032481A Active KR102448389B1 (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망
KR1020207032482A Active KR102486348B1 (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망
KR1020197019186A Active KR102180002B1 (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망

Family Applications After (4)

Application Number Title Priority Date Filing Date
KR1020237035690A Ceased KR20230151047A (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망
KR1020207032481A Active KR102448389B1 (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망
KR1020207032482A Active KR102486348B1 (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망
KR1020197019186A Active KR102180002B1 (ko) 2017-05-23 2018-05-23 어텐션-기반의 시퀀스 변환 신경망

Country Status (12)

Country Link
US (9) US10452978B2 (https=)
EP (7) EP4156033A1 (https=)
JP (6) JP6884871B2 (https=)
KR (5) KR102592677B1 (https=)
CN (1) CN110192206A (https=)
AU (5) AU2018271931B2 (https=)
BR (1) BR112019014822B1 (https=)
CA (3) CA3144674C (https=)
ES (1) ES2934313T3 (https=)
PL (1) PL3542316T3 (https=)
RU (2) RU2749945C1 (https=)
WO (1) WO2018217948A1 (https=)

Families Citing this family (180)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10510018B2 (en) 2013-09-30 2019-12-17 Manyworlds, Inc. Method, system, and apparatus for selecting syntactical elements from information as a focus of attention and performing actions to reduce uncertainty
US12293270B1 (en) 2015-08-03 2025-05-06 Steven D. Flinn Iterative attention-based neural network training and processing
US20200151816A1 (en) 2016-09-15 2020-05-14 Erik M Simpson Transportation and freight capacity units
US12579588B2 (en) 2016-09-15 2026-03-17 Circlesx Llc Method and system utilizing one or more virtual power plant capacity units
US12517892B2 (en) 2017-04-10 2026-01-06 Circlesx Llc Systems, methods, and devices to map to a distributed ledger using a search term
US12493831B2 (en) 2016-09-15 2025-12-09 Circlesx Llc Market layer price queue map routing in transportation capacity and space
US11861527B2 (en) 2018-11-07 2024-01-02 Circlesx Llc Financial swap payment structure method and system on transportation capacity unit assets
US11880883B2 (en) 2016-09-15 2024-01-23 Circlesx Llc Systems and methods for geolocation portfolio exchanges
US11157852B2 (en) 2016-09-15 2021-10-26 Simpsx Technologies Llc Tool appliance community objects with price-time priority queues for transformed tool appliance units
US11790382B2 (en) 2016-09-15 2023-10-17 Circlesx Llc Method to transmit geolocation exchange based markets
US12124976B2 (en) 2018-01-23 2024-10-22 Circlesx Llc Market exchange for transportation capacity in transportation vehicles
US12487095B2 (en) 2016-09-15 2025-12-02 Circlesx Llc Curb community objects with price-time priority queues for transformed curb capacity units
US11138661B2 (en) 2016-09-15 2021-10-05 Simpsx Technologies Llc Agriculture community objects with price-time priority queues for transformed agriculture units
US20190272589A1 (en) 2016-09-15 2019-09-05 Erik M. Simpson Securitization of transportation units
WO2018057667A1 (en) 2016-09-20 2018-03-29 Paradromics, Inc. Systems and methods for detecting corrupt or inaccurate sensory representations
KR102592677B1 (ko) 2017-05-23 2023-10-23 구글 엘엘씨 어텐션-기반의 시퀀스 변환 신경망
US11205121B2 (en) * 2018-06-20 2021-12-21 Disney Enterprises, Inc. Efficient encoding and decoding sequences using variational autoencoders
US11138392B2 (en) 2018-07-26 2021-10-05 Google Llc Machine translation using neural network models
US11295739B2 (en) * 2018-08-23 2022-04-05 Google Llc Key phrase spotting
KR102744304B1 (ko) 2018-12-04 2024-12-18 삼성전자주식회사 뉴럴 네트워크를 구동하기 위한 메모리 공간을 할당하는 방법 및 장치
CN111401081B (zh) * 2018-12-14 2025-07-01 波音公司 神经网络机器翻译方法、模型及模型形成方法
CN109558605B (zh) * 2018-12-17 2022-06-10 北京百度网讯科技有限公司 用于翻译语句的方法和装置
KR102699828B1 (ko) * 2018-12-18 2024-08-29 삼성전자주식회사 어텐션 모델과 하이퍼네트워크를 이용한 기계 번역 방법 및 장치
CN109740169B (zh) * 2019-01-09 2020-10-13 北京邮电大学 一种基于词典和seq2seq预训练机制的中医古籍翻译方法
SG11202107620QA (en) * 2019-01-17 2021-08-30 Visa Int Service Ass A deep learning model for learning program embeddings
WO2020154538A1 (en) * 2019-01-23 2020-07-30 Google Llc Generating neural network outputs using insertion operations
CN109919358B (zh) * 2019-01-31 2021-03-02 中国科学院软件研究所 一种基于神经网络时空注意力机制的实时站点流量预测方法
EP3690752A1 (en) 2019-01-31 2020-08-05 Avatar Cognition Barcelona, SL Fractal cognitive computing node and computer-implemented method for learning procedures
KR102254300B1 (ko) * 2019-04-19 2021-05-21 한국과학기술원 토론 상황 시 발화된 주장에 대한 근거 문장 제공 엔진
CN110083770B (zh) * 2019-04-29 2023-01-13 苏州市职业大学 一种基于特征层面更深的自注意力网络的序列推荐方法
CN112037776A (zh) * 2019-05-16 2020-12-04 武汉Tcl集团工业研究院有限公司 一种语音识别方法、语音识别装置及终端设备
WO2020237188A1 (en) * 2019-05-23 2020-11-26 Google Llc Fully attentional computer vision
US11657277B2 (en) 2019-05-23 2023-05-23 Google Llc Generating neural network outputs using insertion commands
CN110175338B (zh) * 2019-05-31 2023-09-26 北京金山数字娱乐科技有限公司 一种数据处理方法及装置
US11908457B2 (en) * 2019-07-03 2024-02-20 Qualcomm Incorporated Orthogonally constrained multi-head attention for speech tasks
CN110321961A (zh) * 2019-07-09 2019-10-11 北京金山数字娱乐科技有限公司 一种数据处理方法及装置
JP7290507B2 (ja) * 2019-08-06 2023-06-13 本田技研工業株式会社 情報処理装置、情報処理方法、認識モデルならびにプログラム
US11600067B2 (en) * 2019-09-12 2023-03-07 Nec Corporation Action recognition with high-order interaction through spatial-temporal object tracking
IL291346B2 (en) * 2019-09-25 2026-01-01 Deepmind Tech Ltd Augmenting attention-based neural networks for selective processing of past inputs
US11663444B2 (en) 2019-09-27 2023-05-30 Microsoft Technology Licensing, Llc Pipelined neural network processing with continuous and asynchronous updates
KR20210043995A (ko) * 2019-10-14 2021-04-22 삼성전자주식회사 모델 학습 방법 및 장치, 및 시퀀스 인식 방법
KR20210044056A (ko) 2019-10-14 2021-04-22 삼성전자주식회사 중복 토큰 임베딩을 이용한 자연어 처리 방법 및 장치
CN110737764B (zh) * 2019-10-24 2023-07-07 西北工业大学 一种个性化对话内容生成方法
CN112751686B (zh) * 2019-10-29 2022-10-18 中国移动通信集团浙江有限公司 局数据脚本生成方法、装置、计算设备及计算机存储介质
US11246173B2 (en) 2019-11-08 2022-02-08 Huawei Technologies Co. Ltd. Systems and methods for multi-user pairing in wireless communication networks
CN114846523A (zh) * 2019-11-15 2022-08-02 伟摩有限责任公司 使用记忆注意力的多对象跟踪
KR102430918B1 (ko) * 2019-11-15 2022-08-10 고려대학교 산학협력단 한국어 맞춤법 교정장치 및 방법
US11455656B2 (en) * 2019-11-18 2022-09-27 Walmart Apollo, Llc Methods and apparatus for electronically providing item advertisement recommendations
US11392984B2 (en) 2019-11-20 2022-07-19 Walmart Apollo, Llc Methods and apparatus for automatically providing item advertisement recommendations
KR102439165B1 (ko) * 2019-11-26 2022-09-01 한국과학기술원 상식 지식과 언어학적 패턴을 이용한 서술문 신뢰도 평가 시스템 및 그 방법
KR102556096B1 (ko) 2019-11-29 2023-07-18 한국전자통신연구원 이전 프레임의 정보를 사용한 오디오 신호 부호화/복호화 장치 및 방법
CN111222343B (zh) * 2019-12-06 2023-12-29 深圳市优必选科技股份有限公司 一种意图识别方法和意图识别装置
KR102785402B1 (ko) * 2019-12-06 2025-03-21 삼성전자주식회사 뉴럴 네트워크의 행렬 곱셈 연산을 수행하는 장치 및 방법
KR102436900B1 (ko) * 2019-12-12 2022-08-26 서울대학교산학협력단 양방향 언어 모델을 이용한 문장 평가 방법 및 장치
CN111079450B (zh) 2019-12-20 2021-01-22 北京百度网讯科技有限公司 基于顺句驱动的语言转换方法和装置
CN111078825A (zh) * 2019-12-20 2020-04-28 北京百度网讯科技有限公司 结构化处理方法、装置、计算机设备及介质
CN111222562B (zh) * 2020-01-02 2022-04-08 南京邮电大学 基于空间自注意力机制的目标检测方法
CN114981804A (zh) * 2020-01-14 2022-08-30 谷歌有限责任公司 由客户端设备对计算机资产进行活动预测、预取和预加载的方法和系统
US12346793B2 (en) * 2020-02-07 2025-07-01 Google Llc Sorting attention neural networks
EP4085429B1 (en) * 2020-02-07 2025-12-10 GDM Holding LLC Autoregressive neural networks for the generation of polygonal meshes of 3d objects
US12254411B2 (en) * 2020-02-12 2025-03-18 Google Llc Attention neural networks with linear units
US11386885B2 (en) 2020-02-17 2022-07-12 Wipro Limited Method and system for detecting intent as an ordered sequence from a user query
US20210279576A1 (en) * 2020-03-03 2021-09-09 Google Llc Attention neural networks with talking heads attention
CN111427932B (zh) * 2020-04-02 2022-10-04 南方科技大学 出行预测方法、装置、设备和存储介质
CN113627135B (zh) 2020-05-08 2023-09-29 百度在线网络技术（北京）有限公司 一种招聘岗位描述文本的生成方法、装置、设备及介质
US10909461B1 (en) * 2020-05-08 2021-02-02 Google Llc Attention neural networks with locality-sensitive hashing
CN113627199B (zh) * 2020-05-09 2025-10-17 阿里巴巴集团控股有限公司 一种机器翻译方法及其装置
US11806631B2 (en) * 2020-05-11 2023-11-07 Rovi Guides, Inc. Gaming content recommendation for a video game
KR20210145490A (ko) 2020-05-25 2021-12-02 삼성전자주식회사 어텐션 기반 시퀀스 투 시퀀스 모델의 성능 향상 방법 및 장치
US12423575B2 (en) 2020-05-29 2025-09-23 Samsung Electronics Co., Ltd. Method and apparatus for training retrosynthesis prediction model
CN111859927B (zh) * 2020-06-01 2024-03-15 北京先声智能科技有限公司 一种基于注意力共享Transformer的语法改错模型
US20210390410A1 (en) * 2020-06-12 2021-12-16 Google Llc Local self-attention computer vision neural networks
CN111460126B (zh) * 2020-06-12 2020-09-25 支付宝(杭州)信息技术有限公司 一种用于人机对话系统的回复生成方法、装置及电子设备
CN115516461A (zh) * 2020-06-30 2022-12-23 谷歌有限责任公司 具有条件计算的注意力神经网络
US12541292B2 (en) 2020-07-15 2026-02-03 Circlesx Llc User device utilizing one or more multidimensional object databases
JP7357291B2 (ja) * 2020-07-16 2023-10-06 日本電信電話株式会社 翻訳装置、翻訳方法及びプログラム
CN111652357B (zh) * 2020-08-10 2021-01-15 浙江大学 一种利用基于图的特定目标网络解决视频问答问题的方法及其系统
CN112016245A (zh) * 2020-08-13 2020-12-01 五邑大学 基于自注意力变形网络的磁暴预测方法、装置及存储介质
US11983619B2 (en) 2020-08-14 2024-05-14 Micron Technology, Inc. Transformer neural network in memory
US20220058489A1 (en) * 2020-08-19 2022-02-24 The Toronto-Dominion Bank Two-headed attention fused autoencoder for context-aware recommendation
JP7536574B2 (ja) * 2020-09-16 2024-08-20 キオクシア株式会社 演算デバイス、計算機システム、及び演算方法
EP4154185A2 (en) * 2020-09-16 2023-03-29 Google LLC Modeling dependencies with global self-attention neural networks
US12125247B2 (en) * 2020-10-02 2024-10-22 Google Llc Processing images using self-attention based neural networks
JP7618201B2 (ja) 2020-10-20 2025-01-21 国立研究開発法人情報通信研究機構 回答特定用テキスト分類器及び背景知識表現ジェネレータ及びその訓練装置、並びにコンピュータプログラム
CN112347104B (zh) * 2020-11-06 2023-09-29 中国人民大学 一种基于深度强化学习的列存储布局优化方法
US11270124B1 (en) * 2020-11-16 2022-03-08 Branded Entertainment Network, Inc. Temporal bottleneck attention architecture for video action recognition
CN112580822B (zh) 2020-12-16 2023-10-17 北京百度网讯科技有限公司 机器学习模型的对抗训练方法装置、电子设备和介质
CN112508625B (zh) * 2020-12-18 2022-10-21 国网河南省电力公司经济技术研究院 基于多分枝残差注意力网络的智能稽查建模方法
CN112580361A (zh) * 2020-12-18 2021-03-30 蓝舰信息科技南京有限公司 基于统一注意力机制的公式及文字识别模型方法
KR102747605B1 (ko) * 2020-12-31 2024-12-27 팽현석 이미지 생성 방법 및 장치
US11741967B2 (en) 2021-01-04 2023-08-29 Kwai Inc. Systems and methods for automatic speech recognition based on graphics processing units
EP4030355A1 (en) 2021-01-14 2022-07-20 Naver Corporation Neural reasoning path retrieval for multi-hop text comprehension
KR102628947B1 (ko) * 2021-02-02 2024-01-23 숙명여자대학교산학협력단 반응 데이터를 예측하는 시스템 및 그 제어방법
JP7596549B2 (ja) * 2021-02-05 2024-12-09 ディープマインド テクノロジーズ リミテッド 自己注意動作および相互注意動作を使用して潜在的埋め込みを豊かにすることによってニューラルネットワーク出力を生成すること
WO2022167657A2 (en) * 2021-02-05 2022-08-11 Deepmind Technologies Limited Attention neural networks with short-term memory units
US11755973B2 (en) * 2021-02-12 2023-09-12 Accenture Global Solutions Limited System and method for intelligent contract guidance
US20220284267A1 (en) * 2021-03-05 2022-09-08 Qualcomm Incorporated Architectures for temporal processing associated with wireless transmission of encoded data
CN112966626B (zh) * 2021-03-16 2024-10-29 三星（中国）半导体有限公司 人脸识别方法和装置
US12494083B2 (en) 2021-03-16 2025-12-09 Samsung Electronics Co., Ltd. Method and apparatus with face recognition
KR20220130450A (ko) 2021-03-18 2022-09-27 삼성전자주식회사 음성 인식을 위한 인공 신경망에서의 디코딩 방법 및 디코딩 장치
US12462794B2 (en) 2021-03-25 2025-11-04 Beijing Transtreams Technology Co. Ltd. Methods and devices for structured pruning for automatic speech recognition
US12002453B2 (en) 2021-03-25 2024-06-04 Beijing Transtreams Technology Co. Ltd. Methods and devices for irregular pruning for automatic speech recognition
CN117157635A (zh) 2021-04-14 2023-12-01 三菱电机株式会社 学习装置、推理装置、程序、学习方法和推理方法
CN113095431B (zh) * 2021-04-27 2023-08-18 中山大学 一种基于注意力机制的图像描述方法、系统及装置
WO2022241320A1 (en) * 2021-05-14 2022-11-17 Google Llc Neural networks with feedforward spatial transformation units
KR102589164B1 (ko) * 2021-05-25 2023-10-13 한국과학기술원 협업 작업에서 시간적 관계를 활용한 트랜스포머 기반 활동 예측 방법
WO2022250063A1 (ja) 2021-05-26 2022-12-01 キヤノン株式会社 顔認証を行う画像処理装置および画像処理方法
WO2022248727A1 (en) * 2021-05-28 2022-12-01 Deepmind Technologies Limited Generating neural network outputs by cross attention of query embeddings over a set of latent embeddings
EP4298556A1 (en) 2021-05-28 2024-01-03 Google LLC Granular neural network architecture search over low-level primitives
CN113393025A (zh) * 2021-06-07 2021-09-14 浙江大学 一种基于Informer模型编码结构的非侵入式负荷分解方法
EP4102405A1 (en) 2021-06-10 2022-12-14 Naver Corporation Demonstration-conditioned reinforcement learning for few-shot imitation
CN113705323B (zh) * 2021-06-15 2022-09-09 腾讯医疗健康(深圳)有限公司 图像识别方法、装置、设备以及存储介质
US12354402B2 (en) * 2021-06-16 2025-07-08 Sony Group Corporation Landmark detection using deep neural network with multi-frequency self-attention
US11669331B2 (en) * 2021-06-17 2023-06-06 International Business Machines Corporation Neural network processing assist instruction
US20220402122A1 (en) * 2021-06-18 2022-12-22 Naver Labs Corporation Robotic demonstration retrieval systems and methods
US20230028226A1 (en) * 2021-07-15 2023-01-26 Samsung Electronics Co., Ltd. Method for sparsification of feature maps in self-attention mechanisms
CN113379164B (zh) * 2021-07-16 2024-03-26 国网江苏省电力有限公司苏州供电分公司 基于深度自注意力网络的负荷预测方法及系统
KR20230022005A (ko) * 2021-08-06 2023-02-14 주식회사 제이엘케이바이오 화합물 정보 추출을 위한 장치 및 방법
KR102388599B1 (ko) * 2021-08-24 2022-04-21 (주)제로엑스플로우 텍스트 및 이미지를 활용한 문장 교정 장치 및 방법
CN113688640B (zh) * 2021-08-30 2023-01-20 深译信息科技(珠海)有限公司 一种用于医学文档翻译的神经网络机器翻译方法
US12524630B2 (en) 2021-08-31 2026-01-13 Naver Corporation Adversarial generation method for training a neural model
US20230082485A1 (en) * 2021-09-10 2023-03-16 Optum, Inc. Machine learning techniques for denoising input sequences
KR102793723B1 (ko) * 2021-10-08 2025-04-09 네이버 주식회사 세분성 자각 어텐션 기반의 신경망
US20250104717A9 (en) * 2021-10-27 2025-03-27 Mitsubishi Electric Research Laboratories, Inc. End-to-End Speech Recognition Adapted for Multi-Speaker Applications
CN114021736B (zh) * 2021-11-01 2025-07-18 上海交通大学 基于双层索引嵌入层的推荐模型分布式训练方法及gpu
KR102610431B1 (ko) * 2021-11-04 2023-12-06 연세대학교 산학협력단 인공지능 분석 기반 프로그램 소스코드의 요약문 생성 장치 및 방법
KR20230069607A (ko) 2021-11-12 2023-05-19 삼성전자주식회사 셀프 어텐션 기반의 영상 인식 방법 및 장치
KR20230073630A (ko) * 2021-11-19 2023-05-26 주식회사 제이엘케이바이오 화합물 최적화를 위한 장치 및 방법
KR102479817B1 (ko) * 2021-11-25 2022-12-21 인하대학교 산학협력단 소규모 데이터세트를 위한 비전 트랜스포머 장치 및 그 동작 방법
KR102475316B1 (ko) * 2021-12-03 2022-12-08 (주)대교씨엔에스 복수의 은닉층들로 구성되는 딥러닝 기반의 인공지능을 이용한 학습량 추천 방법 및 장치
KR102405832B1 (ko) * 2021-12-03 2022-06-07 주식회사 대교씨엔에스 통계적 분석과 딥러닝 기반의 인공지능을 동시에 활용한 학습량 추천 방법 및 장치
KR102405828B1 (ko) * 2021-12-03 2022-06-07 주식회사 대교씨엔에스 가우시안 혼합 모델을 이용한 클러스터링과 인공지능을 동시에 활용한 학습량 추천 방법 및 장치
CN114529234A (zh) * 2022-01-18 2022-05-24 北京旷视机器人技术有限公司 出库排序方法、电子设备和计算机可读介质
US20250173821A1 (en) 2022-03-23 2025-05-29 Google Llc Multi-resolution Transformer for Video Quality Assessment
US12147776B2 (en) * 2022-04-11 2024-11-19 AtomLeap GmbH Method for extracting information from an unstructured data source
WO2023211735A1 (en) 2022-04-27 2023-11-02 Google Llc Rendering videos with novel views from near-duplicate photos
WO2023211446A1 (en) 2022-04-28 2023-11-02 Google Llc Modeling ambiguity in neural machine translation
EP4468232A4 (en) 2022-05-09 2025-04-09 Samsung Electronics Co., Ltd Image processing device and method for operating same
WO2023219275A1 (ko) * 2022-05-09 2023-11-16 삼성전자 주식회사 영상 처리 장치 및 그 동작 방법
US12137073B2 (en) 2022-05-16 2024-11-05 Google Llc Email summary and completion check
CN114758032B (zh) * 2022-06-15 2022-09-16 之江实验室 基于时空注意力模型的多相期ct图像分类系统及构建方法
US20240020477A1 (en) * 2022-07-13 2024-01-18 Samsung Electronics Co., Ltd. System and method for using residual transformers in natural language processing
KR20240014374A (ko) 2022-07-25 2024-02-01 삼성전자주식회사 트랜스포머 모델을 경량화하는 컴퓨터 시스템 및 그것의 양자화 학습 방법
JP2024032167A (ja) 2022-08-29 2024-03-12 キヤノン株式会社 画像処理装置、画像処理方法及びコンピュータプログラム
KR102573643B1 (ko) 2022-11-14 2023-09-01 주식회사 마키나락스 인공지능 기반 제어 시뮬레이션 방법
CN115760863A (zh) * 2022-11-15 2023-03-07 济南大学 一种适用于水培根茎类禾草根系的图像分割分析方法
US12304512B2 (en) 2022-11-21 2025-05-20 GM Global Technology Operations LLC Detecting and resolving information contradictions for a vehicle
US20240169715A1 (en) * 2022-11-23 2024-05-23 Google Llc Training large-scale vision transformer neural networks with variable patch sizes
US12333835B2 (en) * 2022-11-29 2025-06-17 Bloomberg L.P. Method and apparatus for document analysis and outcome determination
WO2024123310A1 (en) 2022-12-06 2024-06-13 Google Llc Universal sound event detector using multi-layered conditioning
KR102865156B1 (ko) * 2022-12-19 2025-09-29 한국전자통신연구원 인공지능 연산 시스템 및 방법
WO2024156887A1 (en) * 2023-01-26 2024-08-02 Deepmind Technologies Limited Neural networks with intention layers
KR102539679B1 (ko) * 2023-02-01 2023-06-02 (주)피플리 사용자의 경로를 이용하여 사용자 맞춤형 장소 추천 방법, 장치 및 시스템
CN115994668B (zh) * 2023-02-16 2023-06-20 浙江非线数联科技股份有限公司 智慧社区资源管理系统
US20250371843A1 (en) 2023-02-23 2025-12-04 Ceske Vysoke Uceni Technicke V Praze Systems and methods for few-shot new action recognition
CN116312039A (zh) * 2023-03-23 2023-06-23 Oppo广东移动通信有限公司 停车场寻车方法及装置、存储介质及电子设备
CN116070676B (zh) * 2023-03-28 2023-06-30 南京气象科技创新研究院 基于注意力机制和自编码器的高速公路路面温度预报方法
CN116030078B (zh) * 2023-03-29 2023-06-30 之江实验室 多任务学习框架下结合注意力的肺叶分割方法及系统
CN120958494A (zh) 2023-03-31 2025-11-14 谷歌有限责任公司 在基于Transformer的动作识别中使用外部对象检测
CN120936463A (zh) 2023-04-20 2025-11-11 株式会社安川电机 机器人系统、机器人控制方法、机器人控制程序以及程序生成系统
US20240362460A1 (en) 2023-04-25 2024-10-31 Google Llc Train-once-for-all personalization
WO2024233088A1 (en) 2023-05-09 2024-11-14 Google Llc Multimodal multitask alternating gradient descent
KR102644779B1 (ko) * 2023-07-10 2024-03-07 주식회사 스토리컨셉스튜디오 온라인 쇼핑몰의 컨셉에 맞는 상품의 추천 방법
US20250021800A1 (en) * 2023-07-14 2025-01-16 Google Llc Memory efficient sequence generation neural networks
WO2025048508A1 (ko) * 2023-08-30 2025-03-06 하인텔 주식회사 학습 효율 향상을 위한 트랜스포머의 학습 방법 및 이를 위한 장치
CN117059081B (zh) * 2023-08-30 2024-08-09 易方信息科技股份有限公司 一种轻量化语音识别方法、计算机设备及可读存储介质
WO2025075756A1 (en) 2023-10-04 2025-04-10 X Development Llc Large language models for predictive modeling and inverse design
US12608565B2 (en) 2023-10-19 2026-04-21 Naver Corporation Multimodal text-to-text neural machine translation using noise and domain adapters and training noise adapters while a domain adapter is frozen
KR102884459B1 (ko) * 2023-12-13 2025-11-11 동국대학교 산학협력단 트랜스포머를 이용한 배터리 soh 추정 장치 및 추정 방법
EP4576016A1 (en) 2023-12-20 2025-06-25 Naver Corporation Methods and systems for generating 3d representations of scenes from a plurality of images using pointmaps
EP4597364A1 (en) 2024-02-05 2025-08-06 NXAI GmbH Extended long short-term memory neural networks
KR102775392B1 (ko) 2024-02-22 2025-03-05 주식회사 알세미 인공지능 연산의 효율화를 위한 방법, 장치 및 이에 기반한 모델
WO2026005639A1 (ru) * 2024-06-26 2026-01-02 Акционерное Общество "Софит" Способ сопоставления структур топологии слоя микроэлектронного компонента
KR102786623B1 (ko) * 2024-07-24 2025-03-27 렉스이노베이션 주식회사 수급에 따른 마이크로그리드의 동적 가격 조정 방법 및 장치
CN119739995B (zh) * 2024-12-10 2025-09-16 西北工业大学 一种基于自注意力机制的卫星指令自动生成方法
CN119785296A (zh) * 2025-01-13 2025-04-08 济南幼儿师范高等专科学校 一种基于推理增强的校园监控行为识别方法及系统
US12335379B1 (en) 2025-01-17 2025-06-17 K2 Network Labs, Inc. Privacy-preserving transformer model with encrypted dimensionality reduction
US12316753B1 (en) 2025-02-03 2025-05-27 K2 Network Labs, Inc. Secure multi-agent system for privacy-preserving distributed computation
KR102913263B1 (ko) * 2025-02-27 2026-01-15 주식회사 스피어에이엑스 시각 언어 모델에 대한 학습을 위한 학습 데이터를 구축하기 위한 장치 및 이를 위한 방법
US12437113B1 (en) 2025-05-10 2025-10-07 K2 Network Labs, Inc. Data processing orchestrator utilizing semantic type inference and privacy preservation

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2001247627A1 (en) * 2000-03-22 2001-10-03 3-Dimensional Pharmaceuticals, Inc. System, method, and computer program product for representing object relationships in a multidimensional space
EP2036079B1 (en) * 2006-04-27 2011-01-12 Mobiter Dicta Oy A method, a system and a device for converting speech
US9536528B2 (en) * 2012-07-03 2017-01-03 Google Inc. Determining hotword suitability
US10181098B2 (en) * 2014-06-06 2019-01-15 Google Llc Generating representations of input sequences using neural networks
CN110443351B (zh) * 2014-11-14 2021-05-28 谷歌有限责任公司 生成映像的自然语言描述
US11080587B2 (en) * 2015-02-06 2021-08-03 Deepmind Technologies Limited Recurrent neural networks for data item generation
EP3306534B1 (en) 2015-06-03 2021-07-21 Mitsubishi Electric Corporation Inference device and inference method
US10083157B2 (en) * 2015-08-07 2018-09-25 Google Llc Text classification and transformation based on author
US9965705B2 (en) * 2015-11-03 2018-05-08 Baidu Usa Llc Systems and methods for attention-based configurable convolutional neural networks (ABC-CNN) for visual question answering
US10204299B2 (en) * 2015-11-04 2019-02-12 Nec Corporation Unsupervised matching in fine-grained datasets for single-view object reconstruction
US10043512B2 (en) 2015-11-12 2018-08-07 Google Llc Generating target sequences from input sequences using partial conditioning
WO2017083761A1 (en) * 2015-11-12 2017-05-18 Google Inc. Neural programming
CN114819137B (zh) * 2016-05-20 2025-01-03 渊慧科技有限公司 使用比较集对输入样本进行分类的方法、系统和存储介质
CN106372577A (zh) * 2016-08-23 2017-02-01 北京航空航天大学 一种基于深度学习的交通标志自动识别与标注方法
US10642887B2 (en) * 2016-12-27 2020-05-05 Adobe Inc. Multi-modal image ranking using neural networks
KR102592677B1 (ko) 2017-05-23 2023-10-23 구글 엘엘씨 어텐션-기반의 시퀀스 변환 신경망

Also Published As

Publication number Publication date
JP7214783B2 (ja) 2023-01-30
US10452978B2 (en) 2019-10-22
JP2020506466A (ja) 2020-02-27
AU2020213318B2 (en) 2022-06-02
BR112019014822B1 (pt) 2022-06-07
WO2018217948A1 (en) 2018-11-29
US20220051099A1 (en) 2022-02-17
AU2020213318A1 (en) 2020-08-27
CN110192206A (zh) 2019-08-30
AU2020213317B2 (en) 2022-06-02
EP4156032A1 (en) 2023-03-29
EP4156030A1 (en) 2023-03-29
KR20220133323A (ko) 2022-10-04
US20210019624A1 (en) 2021-01-21
EP3542316A1 (en) 2019-09-25
BR112019014822A2 (pt) 2020-02-27
KR20200129198A (ko) 2020-11-17
KR102448389B1 (ko) 2022-09-28
JP7214784B2 (ja) 2023-01-30
CA3144657A1 (en) 2018-11-29
KR20200129197A (ko) 2020-11-17
PL3542316T3 (pl) 2023-02-20
JP7826531B2 (ja) 2026-03-09
CA3144657C (en) 2023-10-10
EP4156033A1 (en) 2023-03-29
US10719764B2 (en) 2020-07-21
RU2749945C1 (ru) 2021-06-21
JP2021121952A (ja) 2021-08-26
US10956819B2 (en) 2021-03-23
JP2023052483A (ja) 2023-04-11
AU2018271931B2 (en) 2020-05-07
EP4156035A1 (en) 2023-03-29
KR102180002B1 (ko) 2020-11-17
AU2022221389A1 (en) 2022-09-22
AU2020213317A1 (en) 2020-08-27
KR20190089980A (ko) 2019-07-31
US20180341860A1 (en) 2018-11-29
US11113602B2 (en) 2021-09-07
JP7423828B2 (ja) 2024-01-29
US20250217644A1 (en) 2025-07-03
EP4156034A1 (en) 2023-03-29
US12217173B2 (en) 2025-02-04
KR20230151047A (ko) 2023-10-31
US20210019623A1 (en) 2021-01-21
RU2021116658A (ru) 2021-07-05
US20190392319A1 (en) 2019-12-26
KR102486348B1 (ko) 2023-01-09
CA3144674A1 (en) 2018-11-29
AU2024202594A1 (en) 2024-05-09
JP2025084774A (ja) 2025-06-03
US11893483B2 (en) 2024-02-06
JP6884871B2 (ja) 2021-06-09
JP7633438B2 (ja) 2025-02-19
EP3542316B1 (en) 2022-12-07
CA3144674C (en) 2023-10-10
AU2018271931A1 (en) 2019-07-11
JP2021121951A (ja) 2021-08-26
CA3050334C (en) 2023-04-11
CA3050334A1 (en) 2018-11-29
ES2934313T3 (es) 2023-02-21
US20200372357A1 (en) 2020-11-26
US20240144006A1 (en) 2024-05-02
JP2024038420A (ja) 2024-03-19
AU2022221389B2 (en) 2024-01-25
EP4156031A1 (en) 2023-03-29
US20200372358A1 (en) 2020-11-26

Similar Documents

Publication Publication Date Title
KR102592677B1 (ko) 어텐션-기반의 시퀀스 변환 신경망

Legal Events

Date Code Title Description
A107 Divisional application of patent
PA0104 Divisional application for international application

Comment text: Divisional Application for International Patent

Patent event code: PA01041R01D

Patent event date: 20220922

Application number text: 1020207032481

Filing date: 20201110

PA0201 Request for examination
PG1501 Laying open of application
E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

Comment text: Notification of reason for refusal

Patent event date: 20221226

Patent event code: PE09021S01D

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

Patent event code: PE07011S01D

Comment text: Decision to Grant Registration

Patent event date: 20230717

GRNT Written decision to grant
PR0701 Registration of establishment

Comment text: Registration of Establishment

Patent event date: 20231018

Patent event code: PR07011E01D

PR1002 Payment of registration fee

Payment date: 20231019

End annual number: 3

Start annual number: 1

PG1601 Publication of registration