BR112023020614A2 - Processamento de entradas multimodais usando modelos de linguagem - Google Patents
Processamento de entradas multimodais usando modelos de linguagemInfo
- Publication number
- BR112023020614A2 BR112023020614A2 BR112023020614A BR112023020614A BR112023020614A2 BR 112023020614 A2 BR112023020614 A2 BR 112023020614A2 BR 112023020614 A BR112023020614 A BR 112023020614A BR 112023020614 A BR112023020614 A BR 112023020614A BR 112023020614 A2 BR112023020614 A2 BR 112023020614A2
- Authority
- BR
- Brazil
- Prior art keywords
- image
- language models
- processing multimodal
- sequence
- multimodal input
- Prior art date
Links
- 238000013528 artificial neural network Methods 0.000 abstract 1
- 238000004590 computer program Methods 0.000 abstract 1
- 238000000034 method Methods 0.000 abstract 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/096—Transfer learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/284—Lexical analysis, e.g. tokenisation or collocates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/26—Techniques for post-processing, e.g. correcting the recognition result
- G06V30/262—Techniques for post-processing, e.g. correcting the recognition result using context analysis, e.g. lexical, syntactic or semantic context
- G06V30/268—Lexical context
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Editing Of Facsimile Originals (AREA)
- Image Processing (AREA)
- Confectionery (AREA)
- Image Analysis (AREA)
Abstract
processamento de entradas multimodais usando modelos de linguagem. métodos, sistemas e aparelhos, incluindo programas de computador codificados em meios de armazenamento de computador, para processamento de entradas multimodais utilizando modelos de linguagem. em particular, as entradas incluem uma imagem, e a imagem é codificada por uma rede de codificador de imagem para gerar uma sequência de incorporações de imagem que representam a imagem. a sequência de incorporações de imagem é fornecida como pelo menos parte de uma sequência de entrada que é processada por uma rede neural de modelo de linguagem.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GR20210100374 | 2021-06-08 | ||
PCT/EP2022/065502 WO2022258666A1 (en) | 2021-06-08 | 2022-06-08 | Multimodal few-shot learning with frozen language models |
Publications (1)
Publication Number | Publication Date |
---|---|
BR112023020614A2 true BR112023020614A2 (pt) | 2023-12-19 |
Family
ID=82319915
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
BR112023020614A BR112023020614A2 (pt) | 2021-06-08 | 2022-06-08 | Processamento de entradas multimodais usando modelos de linguagem |
Country Status (8)
Country | Link |
---|---|
EP (1) | EP4302239A1 (pt) |
JP (1) | JP2024522328A (pt) |
KR (1) | KR20230152741A (pt) |
CN (1) | CN117121015A (pt) |
BR (1) | BR112023020614A2 (pt) |
CA (1) | CA3214735A1 (pt) |
IL (1) | IL307517A (pt) |
WO (1) | WO2022258666A1 (pt) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116135797B (zh) * | 2023-04-19 | 2023-07-04 | 江苏海峡环保科技发展有限公司 | 污水处理智能控制系统 |
CN116597384B (zh) * | 2023-06-02 | 2024-03-05 | 中国人民解放军国防科技大学 | 基于小样本训练的空间目标识别方法、装置及计算机设备 |
CN116452895B (zh) * | 2023-06-13 | 2023-10-20 | 中国科学技术大学 | 基于多模态对称增强的小样本图像分类方法、装置及介质 |
CN116977461B (zh) * | 2023-06-30 | 2024-03-08 | 北京开普云信息科技有限公司 | 特定场景的人像生成方法、装置、存储介质及设备 |
CN117351331A (zh) * | 2023-10-24 | 2024-01-05 | 北京云上曲率科技有限公司 | 一种为视觉大模型添加适配器的方法及装置 |
-
2022
- 2022-06-08 WO PCT/EP2022/065502 patent/WO2022258666A1/en active Application Filing
- 2022-06-08 JP JP2023561364A patent/JP2024522328A/ja active Pending
- 2022-06-08 CA CA3214735A patent/CA3214735A1/en active Pending
- 2022-06-08 EP EP22735099.8A patent/EP4302239A1/en active Pending
- 2022-06-08 IL IL307517A patent/IL307517A/en unknown
- 2022-06-08 BR BR112023020614A patent/BR112023020614A2/pt unknown
- 2022-06-08 CN CN202280025638.3A patent/CN117121015A/zh active Pending
- 2022-06-08 KR KR1020237033833A patent/KR20230152741A/ko unknown
Also Published As
Publication number | Publication date |
---|---|
EP4302239A1 (en) | 2024-01-10 |
CN117121015A (zh) | 2023-11-24 |
KR20230152741A (ko) | 2023-11-03 |
CA3214735A1 (en) | 2022-12-15 |
IL307517A (en) | 2023-12-01 |
AU2022288746A1 (en) | 2023-09-28 |
JP2024522328A (ja) | 2024-06-18 |
WO2022258666A1 (en) | 2022-12-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
BR112023020614A2 (pt) | Processamento de entradas multimodais usando modelos de linguagem | |
EP4273757A3 (en) | Selecting actions using multi-modal inputs | |
WO2018126325A8 (en) | Learning document embeddings with convolutional neural network architectures | |
BR112019004524A2 (pt) | geração de áudio usando redes nuerais | |
SG10201707702YA (en) | Collaborative Voice Controlled Devices | |
EP3754497A8 (en) | Data processing method and related products | |
BR112017011564A2 (pt) | atualizando modelos classificadores de entendimento de linguagem para um assistente pessoal digital baseado em colaboração coletiva | |
WO2018183650A3 (en) | End-to-end text-to-speech conversion | |
GB2565934A (en) | System, method, and apparatus for operating a unified document surface workspace | |
BR112019004335A2 (pt) | pesquisa de similaridade usando códigos polissêmicos | |
CL2017001807A1 (es) | Métodos para el entendimiento de consulta de lenguaje natural incompleta | |
JP2016520942A5 (pt) | ||
MX2016013014A (es) | Métodos y sistemas para la gestión de los diálogos de un robot. | |
TW201714167A (en) | Language model generation device, language model generation method and program therefor, voice recognition device, and voice recognition method and program therefor | |
BR112017026743A2 (pt) | aparelho e método de decodificação, programa, e, aparelho e método de codificação | |
SG10201808556VA (en) | Machine translation system and method | |
BR112022008851A2 (pt) | Métodos e sistemas para a estimativa de um custo computacional de simulação | |
WO2019140382A3 (en) | Probabilistic modeling system and method | |
BR112018008737A8 (pt) | método para fornecer módulos de filtro, produto de programa de computador, e aparelho para gerenciamento de processo | |
WO2019070467A3 (en) | Disaggregating latent causes for computer system optimization | |
GB2604507A (en) | Updating and implementing document from audio proceeding | |
BR112014012599A8 (pt) | aparelho de processamento de imagens para aplicação de uma máscara, estação de trabalho, aparelho de formação de imagens, método de aplicação de uma máscara a um objeto, e produto de programa de computador | |
JP2018205945A (ja) | 対話応答文書自動作成人工知能装置 | |
GB2585492A (en) | Phonetic patterns for fuzzy matching in natural language processing | |
GB2565014A (en) | Providing analytics in real-time based on unstructured electronic documents |