JP6299140B2 - Sound processing apparatus and sound processing method - Google Patents

Description

  The present invention relates to a technique for processing an acoustic signal representing sound.

  Conventionally, a sound source separation technique for separating a specific sound component from a mixed sound of a plurality of sound components generated by different sound sources has been proposed. For example, in Non-Patent Document 1 and Non-Patent Document 2, using a source filter model that expresses the frequency characteristics of a singing sound with harmonic characteristics (source) and envelope characteristics (filter), the singing sound and accompaniment of the music A technique for separating a singing sound from an acoustic signal of a mixed sound with sound is disclosed.

Jean-Louis Durrieu, et al., "MAIN INSTRUMENT SEPARATION FROM STEREOPHONIC AUDIO SIGNALS USING A SOURCE / FILTER MODEL", in Proc. EUSIPCO, p.15-18, 2009 Jean-Louis Durrieu, et al., "A musically motivated mid-level representation for pitch estimation and musical audio source separation", IEEE Journal of Selected Topics on Signal Processing 5 (6), p.1180-1191, 2011

  However, under the techniques of Non-Patent Document 1 and Non-Patent Document 2, an acoustic component whose acoustic characteristics are similar to the singing sound (for example, a percussion instrument similar to the acoustic characteristics of consonants) in a section where there is no actual singing sound. There is a possibility that the acoustic component of the performance sound) is erroneously extracted as a singing sound. In view of the above circumstances, an object of the present invention is to separate a singing sound from an acoustic signal with high accuracy.

  In order to solve the above problems, the sound processing apparatus of the present invention includes a voice analysis means for identifying a voiced section and a voiceless section with respect to a reference sound signal representing a singing sound by a user singing a song, and a song singing sound. Voiced separation means for separating the voiced component of the singing sound for the voiced section specified by the voice analysis means in the acoustic signal of the mixed sound of the sound and the accompaniment sound, and the voice of the singing sound for the unvoiced section specified by the voice analysis means in the acoustic signal There are provided unvoiced separation means for separating the unvoiced component, and synthesis processing means for synthesizing the voiced component separated by the voiced separation means and the unvoiced component separated by the unvoiced separation means. In the above configuration, the voiced section and the unvoiced section are identified from the reference acoustic signal of the singing sound of the user singing the song, the voiced component is separated from the voiced section of the acoustic signal, and the unvoiced component is separated from the unvoiced section. The Therefore, there is an advantage that the singing sound can be separated with high accuracy as compared with the configuration in which the voiced component and the unvoiced component are separated from the acoustic signal without using the reference acoustic signal.

  The voiced section means a section where a voiced component in which the harmonic structure is clearly observed is dominant. On the other hand, the silent section is a section in which a silent component in which the harmonic structure is not clearly observed is dominant, and is distinguished from a silent section in which no voice is present.

  In a preferred aspect of the present invention, the voice analysis means includes a section specifying means for specifying the voiced section and the unvoiced section of the reference sound signal, and a consonant corresponding to the consonant of the singing sound in the unvoiced section in the lyrics recognition for the reference sound signal. A voice recognition means for identifying a section, and the voice separation means separates the voiceless component of the singing sound for the consonant section specified by the text recognition means in the acoustic signal. In the above aspect, the consonant section corresponding to the consonant of the singing sound among the unvoiced sections of the reference acoustic signal is specified by the lyrics recognition, and the unvoiced component is separated from the consonant section of the acoustic signal. Therefore, even when the reference acoustic signal includes an unvoiced sound other than the consonant of the singing sound, there is an advantage that the singing sound can be separated from the acoustic signal with high accuracy.

  In a preferred aspect of the present invention, the lyrics recognizing means identifies the consonant of the reference sound signal by the lyrics recognition, and the unvoiced separating means is identified by the lyrics recognizing means among a plurality of base matrices representing frequency characteristics of different consonants. The unvoiced components are separated by supervised non-negative matrix factorization using the base matrix of the consonant as teacher information. In the above aspect, the base matrix corresponding to the consonant specified by the lyrics recognition of the reference acoustic signal is applied to the supervised non-negative matrix factorization for separating the unvoiced component from the acoustic signal. Therefore, there is an advantage that the consonant of the singing sound can be separated as an unvoiced component in the acoustic signal.

  The sound processing apparatus according to a preferred aspect of the present invention comprises learning processing means for generating a base matrix by learning processing using a silent section specified by the section specifying means in the reference sound signal. In the above aspect, since the base matrix applied to the separation of the unvoiced component is generated by the learning process for the reference acoustic signal, there is an advantage that it is not necessary to prepare the base matrix in advance. In addition, the presence or absence of the lyric recognition means is not required for the configuration including the learning processing means.

  The acoustic processing device according to each of the above aspects is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to processing of an acoustic signal, or a general-purpose calculation such as a CPU (Central Processing Unit). This is also realized by cooperation between the processing device and the program. The program of the present invention can be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but a known arbitrary one such as a semiconductor recording medium or a magnetic recording medium This type of recording medium can be included. For example, the program of the present invention can be provided in the form of distribution via a communication network and installed in a computer. The present invention is also specified as an operation method (acoustic processing method) of the acoustic processing device according to each aspect described above.

1 is a configuration diagram of a sound processing apparatus according to a first embodiment of the present invention. It is a specific block diagram of a sound processing apparatus. It is a flowchart of operation | movement of a sound processing apparatus. It is a block diagram of the sound processing apparatus which concerns on 2nd Embodiment.

  FIG. 1 is a configuration diagram of a sound processing apparatus 100 according to the first embodiment of the present invention. As illustrated in FIG. 1, a signal supply device 12, a sound emission device 14, and a sound collection device 16 are connected to the sound processing device 100. The signal supply device 12 supplies the acoustic signal SA to the acoustic processing device 100. The acoustic signal SA is a time-domain signal representing a mixed sound waveform of a plurality of acoustic components (for example, voice and musical sound) having different acoustic characteristics. For example, a reproduction apparatus that acquires the acoustic signal SA from a portable or built-in recording medium (typically a music CD) and supplies the acoustic signal SA to the acoustic processing apparatus 100 may be employed as the signal supply apparatus 12. Note that the signal supply device 12 may be integrated with the sound processing device 100.

  In the first embodiment, a sound signal SA of a mixed sound of a singing sound and an accompaniment sound of a specific music (hereinafter referred to as “target music”) is supplied from the signal supply device 12 to the sound processing device 100. The singing sound can include a voiced component and an unvoiced component. The voiced component is an acoustic component in which a harmonic structure (harmonic structure) in which a fundamental component and a plurality of harmonic components are arranged on the frequency axis at a frequency that is an integral multiple of the fundamental frequency is observed. An unvoiced component is an acoustic component in which no clear harmonic structure is observed. Typically, vowels of singing sounds correspond to voiced components, and consonants (unvoiced consonants) such as friction sounds and burst sounds correspond to unvoiced components. On the other hand, the accompaniment sounds are configured to include different types of musical tones.

  The acoustic processing device 100 according to the first embodiment is a signal processing device (sound source separation device) that generates an acoustic signal SB through acoustic processing on the acoustic signal SA supplied from the signal supply device 12. The acoustic signal SB is a time-domain signal representing the waveform of the sound obtained by separating the singing sound included in the acoustic signal SA (that is, the sound in which the accompaniment sound of the music is suppressed). The sound emitting device 14 (for example, a speaker or a headphone) emits a sound wave corresponding to the acoustic signal SB generated by the acoustic processing device 100. The D / A converter that converts the acoustic signal SB from digital to analog is not shown for convenience.

  The sound collection device 16 collects ambient sound and generates an acoustic signal representing a time waveform of the sound. The sound collection device 16 of the first embodiment supplies an acoustic signal (hereinafter referred to as “reference acoustic signal”) SREF of the singing sound in which the user sang the target song (singing part) to the acoustic processing device 100. The supply of the reference sound signal SREF from the sound collection device 16 to the sound processing device 100 (singing of the target music by the user) and the sound processing of the sound signal SA and the reproduction of the processed sound signal SB in parallel in real time. And executed. The A / D converter for converting the reference acoustic signal SREF from analog to digital is not shown for convenience.

  As illustrated in FIG. 1, the sound processing device 100 is realized by a computer system including an arithmetic processing device 22 and a storage device 24. The storage device 24 stores a program executed by the arithmetic processing device 22 and various data used by the arithmetic processing device 22. A known recording medium such as a semiconductor recording medium or a magnetic recording medium or a combination of a plurality of types of recording media is arbitrarily adopted as the storage device 24. A configuration in which the acoustic signal SA is stored in the storage device 24 (therefore, the signal supply device 12 can be omitted) is also suitable.

  The storage device 24 of the first embodiment stores a plurality of basis matrices M corresponding to different consonants. The base matrix M corresponding to any one type of consonant is an acoustic model (consonant model) that expresses the frequency characteristics of the consonant. The base matrix M of the first embodiment is a non-negative matrix in which a plurality of base vectors m representing a typical consonant frequency characteristic (frequency spectrum) are arranged in the column direction as illustrated in FIG. It is used as teacher information (prior information) in non-negative matrix factorization (NMF) for SA.

  The arithmetic processing unit 22 implements a plurality of functions (speech analysis unit 32, signal processing unit 34) for generating the acoustic signal SB from the acoustic signal SA by executing a program stored in the storage device 24. The voice analysis unit 32 analyzes the acoustic characteristics of the reference acoustic signal SREF supplied from the sound collection device 16. The signal processing unit 34 generates an acoustic signal SB from the acoustic signal SA using the result of the analysis of the reference acoustic signal SREF by the speech analysis unit 32. That is, in the first embodiment, the reference sound signal SREF of the singing sound that the user sang the target song is used as information for assisting sound source separation with respect to the sound signal SA. A configuration in which each function of the arithmetic processing device 22 is distributed over a plurality of integrated circuits, or a configuration in which a dedicated electronic circuit (for example, a DSP) realizes a part of the functions of the arithmetic processing device 22 may be employed. In practice, an element for converting the time domain acoustic signal SA into the frequency domain by, for example, discrete Fourier transform, and an element for converting the frequency domain acoustic signal SB into the time domain by, for example, discrete inverse Fourier transform are installed. Hereinafter, description and illustration are omitted for the sake of convenience.

  FIG. 2 is a specific configuration diagram of the sound processing apparatus 100. The voice analysis unit 32 sequentially specifies a voiced section QV (V: Voiced) and a voiceless section QU (U: Unvoiced) on the time axis from the reference acoustic signal SREF supplied from the sound collection device 16. The voiced section QV is a section in which the voiced component predominates in the voice section of the reference acoustic signal SREF (the section in which voice is present), and the unvoiced section QU is the unvoiced component in the voice section of the reference acoustic signal SREF. It is a section existing in. As illustrated in FIG. 2, the voice analysis unit 32 according to the first embodiment includes a section identification unit 42 and a lyrics recognition unit 44.

  The section specifying unit 42 sequentially specifies the voiced section QV and the unvoiced section QU0 (the section that is the basis of the unvoiced section QU) of the reference acoustic signal SREF. A known technique is arbitrarily adopted to specify the voiced section QV and the unvoiced section QU0. For example, the section specifying unit 42 detects a voice section in which the singing sound is present in the reference sound signal SREF by known voice section detection (VAD: Voice Activity Detection), and a significant pitch (pitch) in the voice section is detected. An observed section (ie, a section in which a clear harmonic structure exists) is specified as a voiced section QV, and a section in which no significant pitch is observed in the voice section (ie, a section in which no clear harmonic structure exists) is silent. It is specified as section QU0.

  The lyrics recognizing unit 44 performs lyrics recognition on the reference sound signal SREF. The lyrics recognizing unit 44 of the first embodiment recognizes the reference acoustic signal SREF in the lyrics, and sequentially selects the sections (consonant sections) corresponding to the consonant of the singing sound among the unvoiced sections QU0 specified by the section specifying unit 42 as the unvoiced sections QU. And consonants (pronunciation contents) C existing in the unvoiced section QU0 of the reference sound signal SREF are sequentially specified. The unvoiced section QU is a section that excludes sections in which an unvoiced sound other than the singing sound (for example, a percussion instrument performance sound) predominates from the initial unvoiced section QU0 specified by the section specifying unit 42. That is, a part of the unvoiced section QU0 is specified as the unvoiced section QU (consonant section). A known speech recognition technique is arbitrarily employed for the lyrics recognition (voice recognition) by the lyrics recognition unit 44.

  The signal processing unit 34 generates the acoustic signal SB from the acoustic signal SA by signal processing (sound source separation) to which the voiced section QV and unvoiced section QU and the consonant C specified by the speech analysis unit 32 are applied. The generation of the acoustic signal SB by the signal processing unit 34 is executed in real time in parallel with the singing by the user (processing of the voice analysis unit 32 with respect to the reference acoustic signal SREF of the singing sound). As illustrated in FIG. 2, the signal processing unit 34 of the first embodiment includes a voiced separation unit 52, an unvoiced separation unit 54, and a synthesis processing unit 56.

  The voiced separation unit 52 separates (emphasizes or extracts) the voiced component V of the singing sound from each voiced section QV specified by the voice analysis section 32 (section specifying section 42) in the acoustic signal SA supplied from the signal supply device 12. To do. For the separation of the voiced component V, a known sound source separation technique is arbitrarily employed. Specifically, sound source separation technology (V-IMM: "Voiced"-) using a source filter model that expresses a voiced component of a singing sound with harmonic characteristics and envelope characteristics. Instantaneous Mixture Model) is preferably used for the separation of the voiced component V by the voiced separation unit 52. That is, a non-negative matrix corresponding to a time series of harmonic characteristics (source) derived from vibration of a sound source such as a vocal cord and a time series of envelope characteristics (filter) derived from modulation in a resonance tube such as a vocal tract The voiced component V is expressed by element-by-element multiplication (Hadamard product) corresponding to the non-negative matrix, and the addition of the voiced component V and the acoustic component other than the voiced component V expresses the frequency characteristic (spectrogram of the acoustic signal SA. The voiced component V is estimated by iterating a predetermined update equation so as to approximate the observation matrix.

  The unvoiced separation unit 54 separates (emphasizes or extracts) the unvoiced component U of the singing sound from each unvoiced section QU specified by the speech analysis unit 32 (lyric recognition unit 44) in the acoustic signal SA supplied from the signal supply device 12. To do. For the separation of the unvoiced component U, a known sound source separation technique can be arbitrarily employed. However, the unvoiced separation unit 54 of the first embodiment performs non-negative matrix factorization using the base matrix M stored in the storage device 24. An unvoiced component U is estimated. Specifically, the unvoiced separation unit 54 searches the basis matrix M corresponding to the consonant C specified by the lyrics recognition unit 44 among the plurality of basis matrices M stored in the storage device 24, and uses the basis matrix M as a teacher. The unvoiced component U is separated by supervised non-negative matrix factorization (Supervised-NMF) used as information (prior information). For the supervised non-negative matrix factorization by the silent separation unit 54, for example, a technique disclosed in Japanese Patent Application Laid-Open No. 2013-33196 is suitably employed. Specifically, an unvoiced component U expressed by a matrix product of a base matrix M of a consonant C and a coefficient matrix that represents a time series of weighted values of each basis vector m, and an acoustic component other than the unvoiced component U are added. The unvoiced component U is estimated by repeating the calculation of a predetermined update formula so as to approximate the frequency characteristic (observation matrix) of the acoustic signal SA. As understood from the above description, in the first embodiment, the voiced component V and the unvoiced component U of the singing sound of the acoustic signal SA are separated by different methods.

  The synthesis processing unit 56 generates the acoustic signal SB by synthesizing the voiced component V separated by the voiced separation unit 52 and the unvoiced component U separated by the unvoiced separation unit 54. Specifically, the synthesis processing unit 56 arranges the voiced component V generated by the voiced separation unit 52 for each voiced section QV and the voiced component U generated by the voiceless separation unit 54 for each voiced section QU on the time axis. Thus, the time domain acoustic signal SB is generated. Therefore, the acoustic signal SB is generated by selectively extracting the singing sound from the acoustic signal SA of the mixed sound of the singing sound and the accompaniment sound of the target music. The acoustic signal SB generated by the synthesis processing unit 56 is supplied to the sound emitting device 14 and is emitted as a sound wave.

  FIG. 3 is a flowchart of the operation executed by the arithmetic processing unit 22. The processing of FIG. 3 is repeatedly executed for each unit section obtained by dividing the reference acoustic signal SREF and the acoustic signal SA on the time axis. When the processing of FIG. 3 is started, the arithmetic processing unit 22 (speech analysis unit 32) executes a speech analysis PA that identifies the voiced section QV and the unvoiced section QU from the unit section of the reference acoustic signal SREF. Specifically, the arithmetic processing unit 22 (section specifying unit 42) specifies the voiced section QV and the unvoiced section QU0 from the unit section of the reference acoustic signal SREF (PA1). Then, the arithmetic processing unit 22 (lyric recognition unit 44) identifies the unvoiced section QU and the consonant C of the unvoiced section QU0 by recognizing the lyrics for the unit section of the reference sound signal SREF (PA2).

  When the voice analysis PA is executed, the arithmetic processing unit 22 (signal processing unit 34) uses the voiced section QV, the unvoiced section QU, and the consonant C specified by the voice analysis PA to generate the acoustic signal SB from the unit section of the acoustic signal SA. The signal processing PB to be generated is executed. Specifically, the arithmetic processing unit 22 separates the voiced component V from the voiced section QV in the unit section (PB1 / voiced separation unit 52), and separates the voiced component U from the voiceless section QU in the unit section. Processing (PB2 / voiceless separation unit 54) is executed. The consonant C is applied to the separation of the unvoiced component U. Then, the arithmetic processing unit 22 (synthesis processing unit 56) generates the acoustic signal SB by synthesizing the voiced component V and the unvoiced component U in the unit section (PB3).

  As described above, in the first embodiment, the voiced section QV and the unvoiced section QU are specified from the reference acoustic signal SREF of the singing sound that the user sang the target song, and the voiced component from the voiced section QV of the acoustic signal SA. V is separated and the unvoiced component U is separated from the unvoiced section QU of the acoustic signal SA. That is, the reference acoustic signal SREF is applied to sound source separation of the voiced component V and the unvoiced component U as auxiliary information. Therefore, compared with the configuration in which the voiced component V and the unvoiced component U are separated from only the acoustic signal SA without using the reference acoustic signal SREF, the singing sound (voiced component V and unvoiced component) is accurately obtained from the acoustic signal SA of the target music. There is the advantage that component U) can be separated.

  In the first embodiment, the unvoiced section (consonant section) QU corresponding to the consonant of the singing sound among the unvoiced section QU0 of the reference acoustic signal SREF is specified by the lyrics recognition, and the unvoiced component U is separated from the unvoiced section QU of the acoustic signal SA. Is done. Therefore, even when an unvoiced sound other than the consonant of the singing sound (for example, a percussion instrument performance sound) is included in the unvoiced section QU0 of the reference sound signal SREF, only the consonant of the singing sound is selectively separated as the unvoiced component U. That is, the effect that the singing sound of the acoustic signal SA can be separated with high accuracy is particularly remarkable.

  In the first embodiment, among the plurality of basis matrices M stored in the storage device 24, the basis matrix M corresponding to the consonant C identified by the lyrics recognition for the unvoiced section QU 0 of the reference acoustic signal SREF is the unvoiced separation unit 54. Applied to separation of unvoiced component U by (supervised non-negative matrix factorization). Therefore, there is an advantage that the consonant of the singing sound in the acoustic signal SA can be separated as the unvoiced component U with high accuracy (and the singing sound of the acoustic signal SA can be separated with high accuracy).

Second Embodiment
A second embodiment of the present invention will be described. In each of the embodiments exemplified below, elements having the same functions and functions as those of the first embodiment will be referred to in the description of the first embodiment, and detailed descriptions thereof will be appropriately omitted.

  FIG. 4 is a configuration diagram of the sound processing apparatus 100 according to the second embodiment. As illustrated in FIG. 4, the arithmetic processing device 22 of the second embodiment functions as a learning processing unit 36 in addition to the same elements (speech analysis unit 32 and signal processing unit 34) as in the first embodiment. The learning processing unit 36 sequentially generates a base matrix M by a learning process using the unvoiced section QU specified by the speech analysis unit 32 (section specifying unit 42) in the reference acoustic signal SREF supplied from the sound collection device 16. . A known machine learning technique is arbitrarily employed for the learning process. The unvoiced separation unit 54 of the signal processing unit 34 is a supervised non-negative matrix factorization using the base matrix M sequentially generated by the learning processing unit 36 as teacher information, and the speech analysis unit 32 (lyric recognition) of the acoustic signal SA. The unvoiced component U is separated from the unvoiced section QU specified by the section 44).

  In the first embodiment, the singing by the user (generation of the reference acoustic signal SREF) and the generation of the acoustic signal SB are executed in parallel in real time. In the second embodiment, after generating the base matrix M using the reference acoustic signal SREF, the acoustic signal SB is generated by applying each base matrix M (that is, the acoustic signal SA after the user sings the target song). The configuration of generating the acoustic signal SB from the above is preferable.

  In the second embodiment, the same effect as in the first embodiment is realized. Further, in the second embodiment, since the base matrix M applied to the separation of the unvoiced component U is generated by the learning process for the reference acoustic signal SREF, there is an advantage that it is not necessary to prepare the base matrix M in advance. In the above example, the unvoiced component U is separated from the unvoiced section QU specified by the lyrics recognizing unit 44 in the acoustic signal SA, but the unvoiced component U from the unvoiced section QU0 specified by the section specifying unit 42 in the acoustic signal SA. Can also be separated. Therefore, the lyrics recognition unit 44 can be omitted in the second embodiment.

<Modification>
Each form illustrated above can be variously modified. Specific modifications are exemplified below. Two or more aspects arbitrarily selected from the following examples can be appropriately combined.

(1) In the first embodiment, the unvoiced component U is separated from the unvoiced section QU specified by the lyrics recognizing unit 44 in the acoustic signal SA of the target music, but the unvoiced section specified by the section specifying unit 42 in the acoustic signal SA. It is also possible for the unvoiced separation unit 54 to separate the unvoiced component U from QU0. That is, the process in which the lyrics recognizing unit 44 specifies the silent section QU corresponding to the consonant of the singing sound can be omitted. As understood from the above description, the unvoiced section identified from the reference acoustic signal SREF by the speech analysis unit 32 of each embodiment described above is the unvoiced section QU0 other than the voiced section QV in the speech section of the reference acoustic signal SREF, and the unvoiced section. It includes both a part of the section QU0 (consonant section corresponding to the consonant of the singing sound in the unvoiced section QU0) QU.

(2) Information extracted from the reference acoustic signal SREF to assist sound source separation by the signal processing unit 34 is not limited to the information (voiced section QV, unvoiced section QU, consonant C) exemplified in the above-described embodiments. For example, the pitch (pitch) extracted from the reference acoustic signal SREF can be used for separation of the voiced component V by the voiced separation unit 52. For example, if an acoustic component within a predetermined range with respect to the pitch extracted from the reference acoustic signal SREF is extracted as a candidate for the voiced component V from the acoustic signal SA of the target music, the pitch of the reference acoustic signal SREF is extracted. It is possible to separate the voiced component V with higher accuracy compared to a configuration that does not use the. Note that a known pitch estimation technique can be arbitrarily employed for estimating the pitch of the reference acoustic signal SREF.

(3) When the user's singing is poor, the voiced section QV and the unvoiced section QU may not match between the reference acoustic signal SREF and the acoustic signal SA. Therefore, a configuration that specifies the voiced section QV and the unvoiced section QU after adjusting the reference acoustic signal SREF is preferable. For example, the voice analysis unit 32 (section specifying unit 42) adjusts the reference sound signal SREF on the time axis so that each time point on the time axis of the reference sound signal SREF matches the time point corresponding to the target music. After (alignment), the voiced section QV and the unvoiced section QU (QU0) are specified. According to the above structure, even when a user's song is not good, there exists an advantage that the song sound of an object music can be isolate | separated with high precision.

(4) In each of the above-described embodiments, the separation of the voiced component V by the voiced separation unit 52 and the separation of the unvoiced component U by the unvoiced separation unit 54 are performed separately, but both the voiced component V and the unvoiced component U are acoustic signals. A configuration that separates from SA in a lump can also be employed. That is, the voiced separation unit 52 and the unvoiced separation unit 54 can be grasped as an integral element.

(5) In each embodiment described above, the singing sound is extracted from the acoustic signal SA of the mixed sound of the singing sound and the accompaniment sound, but the accompaniment sound can be extracted from the acoustic signal SA. For example, it is possible to generate an acoustic signal obtained by separating (emphasizing or extracting) the accompaniment sound of the target music by subtracting the acoustic signal SB generated in each form described above from the acoustic signal SA.

(6) The sound processing apparatus 100 can be realized by a server device that communicates with a terminal device such as a mobile phone. For example, the acoustic processing device 100 generates the acoustic signal SB from the acoustic signal SA using the reference acoustic signal SREF received from the terminal device via the communication network, and transmits the acoustic signal SB to the terminal device. The acoustic signal SA to be processed is a signal supplied from the signal supply device 12 connected to the sound processing device 100 or a signal received by the sound processing device 100 from the terminal device via the communication network.

DESCRIPTION OF SYMBOLS 100 ... Sound processing device, 12 ... Signal supply device, 14 ... Sound emission device, 16 ... Sound collection device, 22 ... Arithmetic processing device, 24 ... Memory | storage device, 32 ... Voice analysis part, 34 ... A signal processing unit, 36 a learning processing unit, 42 a section specifying unit, 44 a lyrics recognition unit, 52 a voiced separation unit, 54 a voiceless separation unit, and 56 a synthesis processing unit.

Claims (6)

Voice analysis means for identifying voiced and unvoiced sections for a reference acoustic signal representing a singing sound of a user singing a song;
Voiced separation means for separating the voiced component of the singing sound from the acoustic signal of the mixed sound of the singing sound of the music and the accompaniment sound for the voiced section specified by the voice analysis means;
Unvoiced separation means for separating the unvoiced component of the singing sound for the unvoiced section identified by the voice analysis means of the acoustic signal;
An acoustic processing apparatus comprising: a synthesis processing unit that synthesizes the voiced component separated by the voiced separation unit and the voiced component separated by the voiceless separation unit. The voice analysis means includes
Section specifying means for specifying a voiced section and an unvoiced section of the reference acoustic signal;
Lyrics recognition means for identifying a consonant section corresponding to a consonant of a singing sound among the unvoiced sections in the lyrics recognition for the reference acoustic signal,
The sound processing apparatus according to claim 1, wherein the unvoiced separating unit separates unvoiced components of a singing sound for a consonant section specified by the lyrics recognizing unit in the acoustic signal. The lyric recognition means identifies a consonant of the reference acoustic signal by the lyric recognition,
The unvoiced separation means separates unvoiced components by supervised non-negative matrix factorization using the basis matrix of the consonant specified by the lyrics recognition means as teacher information among a plurality of basis matrices representing frequency characteristics of different consonants. The sound processing apparatus according to claim 2. The acoustic processing device according to claim 3, further comprising learning processing means for generating the base matrix by learning processing using a silent section specified by the section specifying means in the reference acoustic signal. Comprising learning processing means for generating the basis matrix in learning processing using a silent section specified by the section specifying means in the reference acoustic signal;
The unvoiced separating means separates unvoiced components by supervised non-negative matrix factorization using the basis matrix generated by the learning processing means as teacher information.
The sound processing apparatus according to claim 3 .   Computer
  Identify voiced and unvoiced intervals for the reference acoustic signal representing the singing sound of the user singing the song,
  Separating the voiced component of the singing sound for the voiced section of the mixed sound of the singing sound and the accompaniment sound of the music;
  Separating the unvoiced component of the singing sound for the unvoiced section of the acoustic signal;
  Combining the voiced component and the unvoiced component
  Sound processing method.

Images