JP7846697B2 - 合成音声生成 - Google Patents

合成音声生成

Info

Publication number
JP7846697B2
JP7846697B2 JP2023543425A JP2023543425A JP7846697B2 JP 7846697 B2 JP7846697 B2 JP 7846697B2 JP 2023543425 A JP2023543425 A JP 2023543425A JP 2023543425 A JP2023543425 A JP 2023543425A JP 7846697 B2 JP7846697 B2 JP 7846697B2
Authority
JP
Japan
Prior art keywords
encoder
speech
input
control parameters
encoded data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2023543425A
Other languages
English (en)
Japanese (ja)
Other versions
JP2024504316A5 (enExample
JP2024504316A (ja
Inventor
キュンゲン・ビュン
スンクク・ムン
シュフア・ジャン
バヒド・モンタゼリ
レ－フン・キム
エリック・ヴィッサー
Original Assignee
クアルコム，インコーポレイテッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by クアルコム，インコーポレイテッド filed Critical クアルコム，インコーポレイテッド
Publication of JP2024504316A publication Critical patent/JP2024504316A/ja
Publication of JP2024504316A5 publication Critical patent/JP2024504316A5/ja
Application granted granted Critical
Publication of JP7846697B2 publication Critical patent/JP7846697B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems
    • G10L13/02—Methods for producing synthetic speech; Speech synthesisers
    • G10L13/04—Details of speech synthesis systems, e.g. synthesiser structure or memory management
    • G10L13/047—Architecture of speech synthesisers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems
    • G10L13/02—Methods for producing synthetic speech; Speech synthesisers
    • G10L13/033—Voice editing, e.g. manipulating the voice of the synthesiser
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/003—Changing voice quality, e.g. pitch or formants
    • G10L21/007—Changing voice quality, e.g. pitch or formants characterised by the process used
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/003—Changing voice quality, e.g. pitch or formants
    • G10L21/007—Changing voice quality, e.g. pitch or formants characterised by the process used
    • G10L21/013—Adapting to target pitch
    • G10L2021/0135—Voice conversion or morphing

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Acoustics & Sound (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Quality & Reliability (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Child & Adolescent Psychology (AREA)
  • Hospice & Palliative Care (AREA)
  • Psychiatry (AREA)
  • Telephonic Communication Services (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • User Interface Of Digital Computer (AREA)
JP2023543425A 2021-01-21 2021-12-08 合成音声生成 Active JP7846697B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US17/154,372 US11676571B2 (en) 2021-01-21 2021-01-21 Synthesized speech generation
US17/154,372 2021-01-21
PCT/US2021/072800 WO2022159256A1 (en) 2021-01-21 2021-12-08 Synthesized speech generation

Publications (3)

Publication Number Publication Date
JP2024504316A JP2024504316A (ja) 2024-01-31
JP2024504316A5 JP2024504316A5 (enExample) 2024-12-02
JP7846697B2 true JP7846697B2 (ja) 2026-04-15

Family

ID=79092855

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2023543425A Active JP7846697B2 (ja) 2021-01-21 2021-12-08 合成音声生成

Country Status (8)

Country Link
US (1) US11676571B2 (enExample)
EP (1) EP4281964B1 (enExample)
JP (1) JP7846697B2 (enExample)
KR (1) KR20230133295A (enExample)
CN (1) CN116711002A (enExample)
BR (1) BR112023013902A2 (enExample)
TW (1) TWI918793B (enExample)
WO (1) WO2022159256A1 (enExample)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11676571B2 (en) * 2021-01-21 2023-06-13 Qualcomm Incorporated Synthesized speech generation
US12260851B2 (en) * 2021-07-14 2025-03-25 Google Llc Two-level text-to-speech systems using synthetic training data
US12002455B2 (en) * 2021-07-22 2024-06-04 Qualcomm Incorporated Semantically-augmented context representation generation
US20230037541A1 (en) * 2021-07-29 2023-02-09 Xinapse Co., Ltd. Method and system for synthesizing speeches by scoring speeches
CN115359779B (zh) * 2022-07-26 2025-02-11 招联消费金融股份有限公司 语音合成方法、装置、计算机设备和存储介质
CN115394284B (zh) * 2022-08-23 2024-09-24 平安科技（深圳）有限公司 语音合成方法、系统、设备及存储介质
US12682918B2 (en) 2022-09-13 2026-07-14 Qualcomm Incorporated Source speech modification based on an input speech characteristic mapped to multiple reference embeddings within a threshold similarity
US20240304174A1 (en) * 2023-03-08 2024-09-12 Sony Interactive Entertainment Inc. Ambient Noise Capture for Speech Synthesis of In-Game Character Voices
TWI872556B (zh) * 2023-05-25 2025-02-11 國立清華大學 一種跨辨識任務之情緒辨識方法、系統及電腦程式產品

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019116889A1 (ja) 2017-12-12 2019-06-20 ソニー株式会社 信号処理装置および方法、学習装置および方法、並びにプログラム
JP2019109306A (ja) 2017-12-15 2019-07-04 日本電信電話株式会社 音声変換装置、音声変換方法及びプログラム
JP2020013008A (ja) 2018-07-19 2020-01-23 株式会社エーアイ 音声処理装置、音声処理プログラムおよび音声処理方法
JP2020076843A (ja) 2018-11-06 2020-05-21 ヤマハ株式会社 情報処理方法および情報処理装置
US20200342852A1 (en) 2018-01-11 2020-10-29 Neosapience, Inc. Speech translation method and system using multilingual text-to-speech synthesis model

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8155965B2 (en) 2005-03-11 2012-04-10 Qualcomm Incorporated Time warping frames inside the vocoder by modifying the residual
PL2242045T3 (pl) 2009-04-16 2013-02-28 Univ Mons Sposób kodowania i syntezy mowy
CN105390141B (zh) * 2015-10-14 2019-10-18 科大讯飞股份有限公司 声音转换方法和装置
US10332520B2 (en) * 2017-02-13 2019-06-25 Qualcomm Incorporated Enhanced speech generation
KR102072162B1 (ko) * 2018-01-05 2020-01-31 서울대학교산학협력단 인공 지능 기반 외국어 음성 합성 방법 및 장치
KR102473447B1 (ko) * 2018-03-22 2022-12-05 삼성전자주식회사 인공지능 모델을 이용하여 사용자 음성을 변조하기 위한 전자 장치 및 이의 제어 방법
CN109036375B (zh) 2018-07-25 2023-03-24 腾讯科技（深圳）有限公司 语音合成方法、模型训练方法、装置和计算机设备
US11138392B2 (en) 2018-07-26 2021-10-05 Google Llc Machine translation using neural network models
WO2020027619A1 (ko) * 2018-08-02 2020-02-06 네오사피엔스 주식회사 순차적 운율 특징을 기초로 기계학습을 이용한 텍스트-음성 합성 방법, 장치 및 컴퓨터 판독가능한 저장매체
US10741169B1 (en) * 2018-09-25 2020-08-11 Amazon Technologies, Inc. Text-to-speech (TTS) processing
KR20200092505A (ko) * 2019-01-13 2020-08-04 네오데우스 주식회사 심층 합성곱 신경망 인공지능을 기반으로 한 미세조정을 통해 소량의 샘플로 화자 적합된 음성합성 모델을 생성하는 방법
US12032920B2 (en) * 2019-03-29 2024-07-09 Google Llc Direct speech-to-speech translation via machine learning
KR102663669B1 (ko) * 2019-11-01 2024-05-08 엘지전자 주식회사 소음 환경에서의 음성 합성
CN111862953B (zh) * 2019-12-05 2023-08-22 北京嘀嘀无限科技发展有限公司 语音识别模型的训练方法、语音识别方法及装置
US11335324B2 (en) * 2020-08-31 2022-05-17 Google Llc Synthesized data augmentation using voice conversion and speech recognition models
EP3989217B1 (en) * 2020-10-22 2023-09-27 Thomson Licensing Method for detecting an audio adversarial attack with respect to a voice input processed by an automatic speech recognition system, corresponding device, computer program product and computer-readable carrier medium
CN112133282B (zh) * 2020-10-26 2022-07-08 厦门大学 轻量级多说话人语音合成系统及电子设备
CN112562728B (zh) * 2020-11-13 2024-06-18 百果园技术(新加坡)有限公司 生成对抗网络训练方法、音频风格迁移方法及装置
CN112466316A (zh) * 2020-12-10 2021-03-09 青海民族大学 一种基于生成对抗网络的零样本语音转换系统
US11676571B2 (en) * 2021-01-21 2023-06-13 Qualcomm Incorporated Synthesized speech generation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019116889A1 (ja) 2017-12-12 2019-06-20 ソニー株式会社 信号処理装置および方法、学習装置および方法、並びにプログラム
JP2019109306A (ja) 2017-12-15 2019-07-04 日本電信電話株式会社 音声変換装置、音声変換方法及びプログラム
US20200342852A1 (en) 2018-01-11 2020-10-29 Neosapience, Inc. Speech translation method and system using multilingual text-to-speech synthesis model
JP2020013008A (ja) 2018-07-19 2020-01-23 株式会社エーアイ 音声処理装置、音声処理プログラムおよび音声処理方法
JP2020076843A (ja) 2018-11-06 2020-05-21 ヤマハ株式会社 情報処理方法および情報処理装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Songxiang Liu, 外5名，"End-To-End Accent Conversion Without Using Native Utterances"，ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)，2020年05月08日，pp. 6289-6293，[取得日 2025年10月17日], 取得先< https://ieeexplore.ieee.org/document/9053797>

Also Published As

Publication number Publication date
EP4281964B1 (en) 2026-07-22
US11676571B2 (en) 2023-06-13
US20220230623A1 (en) 2022-07-21
KR20230133295A (ko) 2023-09-19
EP4281964A1 (en) 2023-11-29
CN116711002A (zh) 2023-09-05
BR112023013902A2 (pt) 2024-01-23
TWI918793B (zh) 2026-03-21
TW202230328A (zh) 2022-08-01
JP2024504316A (ja) 2024-01-31
WO2022159256A1 (en) 2022-07-28

Similar Documents

Publication Publication Date Title
TWI918793B (zh) 用於合成語音生成的設備、方法、電腦可讀儲存媒體以及裝置
US12380909B2 (en) Context-data based speech enhancement
US11495235B2 (en) System for creating speaker model based on vocal sounds for a speaker recognition system, computer program product, and controller, using two neural networks
WO2024054714A1 (en) Avatar representation and audio generation
CN112183105B (zh) 人机交互方法及装置
CN108922525A (zh) 语音处理方法、装置、存储介质及电子设备
US20240087597A1 (en) Source speech modification based on an input speech characteristic
WO2020079918A1 (ja) 情報処理装置及び情報処理方法
JP2021099454A (ja) 音声合成装置、音声合成プログラム及び音声合成方法
CN119967082B (zh) 语音识别方法、电子设备、存储介质及芯片系统
CN118155602A (zh) 语音合成方法、装置、设备以及计算机可读存储介质
KR102783870B1 (ko) 전자 장치 및 그의 제어 방법
CN119785756B (zh) 语音生成中的数据处理方法、装置及电子设备
CN114267322A (zh) 语音处理方法、装置、计算机可读存储介质及计算机设备
JP2020013008A (ja) 音声処理装置、音声処理プログラムおよび音声処理方法
CN119998869A (zh) 用于媒体生成的媒体片段预测
KR20240073991A (ko) 음성 합성 서비스 제공 방법 및 그 시스템
CN116092466B (zh) 语音模型的处理方法、装置、计算机设备及存储介质
TW202345145A (zh) 使用神經網路和多個子帶網路的音訊樣本重構
WO2025128139A1 (en) Automated attention handling in active noise control systems based on universal sound conversion
WO2026039056A1 (en) Architecture and network topology for universal sound conversion
CN117546233A (zh) 电子设备及其控制方法
TW202605804A (zh) 基於音訊資料分類來選擇一或多個機器學習模型處理分支之系統及方法
TW202536854A (zh) 以機器學習話音合成器使用合成式分析之話音編碼
JP2004004182A (ja) 音声認識装置、音声認識方法及び音声認識プログラム

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20241122

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20241122

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20251017

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20251028

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20260122

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20260310

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20260403

R150 Certificate of patent or registration of utility model

Ref document number: 7846697

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150