JP2023020577A - masking device - Google Patents

Abstract

To provide a masking device that generates and reproduces a masking sound that responds to utterance of a person in real time.SOLUTION: A masking device 1 includes: a detection unit 111 that detects a sound signal indicating a sound from an output signal output from a microphone; an analysis unit 112 that analyzes the sound signal to generate feature data indicating features of the sound; and a generation unit 114 that based on the feature data, generates masking data indicating a music that masks the sound.SELECTED DRAWING: Figure 4

Description

The present invention relates to masking devices.

Conventionally, in order to prevent third parties from grasping the content of conversational voices between people in automobiles, counters of shops and hospitals, etc., technology has been used to output a masking sound that drowns out the conversational voices. .

For example, Patent Literature 1 discloses an anonymizing device for anonymizing dialogue voice. The anonymizing device includes a storage device in which voice data representing voices of general conversation and music data representing music are stored in advance. The anonymization device includes an anonymization data generation device that generates anonymization data by synthesizing audio data and music data read from a storage device. Further, the anonymizing device includes a music reproducing device that reproduces the anonymized data. By reproducing this anonymized data, for example, at a bank window, conversation between a bank employee and a user can be anonymized so that it cannot be heard by a third party.

JP 2012-141524 A

However, the anonymization device according to Patent Document 1 generates anonymization data as a masking sound by synthesizing audio data and music data as sample data stored in advance. That is, the technique according to Patent Document 1 does not generate masking sounds in response to human speech in real time.

In view of the above circumstances, one aspect of the present disclosure is to generate masking data corresponding to human speech in real time, and play music that masks human speech based on the generated masking data. It is an object of the present invention to provide a masking device that

In order to solve the above problems, a masking device according to one aspect of the present disclosure includes a detection unit that detects an audio signal indicating audio from an output signal output from a microphone, and by analyzing the audio signal, The apparatus includes an analysis unit that generates feature data indicating features of the voice, and a generation unit that generates masking data indicating music for masking the voice based on the feature data.

1 is a block diagram illustrating the configuration of a masking device 1 according to a first embodiment; FIG. It is an example of the top view of the vehicle C which mounts the masking apparatus 1 which concerns on 1st Embodiment. It is an example of the side view of the vehicle C which mounts the masking apparatus 1 which concerns on 1st Embodiment. 3 is a block diagram illustrating the functional configuration of the control device 11; FIG. FIG. 3 is a diagram showing an example of frequency bands of a plurality of musical instruments to which a plurality of parts included in music data respectively correspond; 3 is a diagram showing the level of each part included in music data and masking data output by a generation unit 114. FIG. 4 is a flow chart showing the operation of the masking device 1 according to the first embodiment; 3 is a block diagram illustrating the functional configuration of the control device 11; FIG. FIG. 4 is a diagram showing the level of each part included in the music data and masking data output by the generator 114A;

[1. First Embodiment]
[1-1. Configuration of the First Embodiment]
FIG. 1 is a block diagram illustrating the configuration of a masking device 1 according to the first embodiment of the present disclosure. The masking device 1 generates masking data Dm indicating music for masking the collected human voice according to the characteristics of the collected human voice, and reproduces the music for masking the voice based on the generated masking data Dm. It is a device that Specifically, the masking device 1 includes a control device 11 , a storage device 12 , an operation device 13 , a sound pickup device 14 and a reproduction device 15 .

The control device 11 of FIG. 1 is, for example, one or more processors that control each element of the masking device 1 . For example, the control device 11 includes one or more types of CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit). It consists of a processor.

The storage device 12 is, for example, one or a plurality of memories composed of known recording media such as magnetic recording media or semiconductor recording media. The storage device 12 stores a control program PR1 executed by the control device 11 and various data used by the control device 11, especially music data Dx. Note that the storage device 12 may be configured by a combination of multiple types of recording media. Also, the storage device 12 may be a removable recording medium detachable from the masking device 1, or an external recording medium (for example, online storage) with which the masking device 1 can communicate via a communication network.

The operation device 13 is an input device that receives instructions from the user. The operation device 13 is, for example, a plurality of operators that can be operated by the user, or a touch panel that detects contact from the user. Among other things, the operation device 13 functions as a switch for instructing the start and end of the operation of the masking device 1 . The operation device 13 is also used to store music data Dx supplied from the outside in the storage device 12 .

The sound pickup device 14 is a microphone that includes a sound pickup unit that picks up ambient sound and converts the picked sound into an electric signal. The sound pickup part may have any structure as long as it is configured to pick up sound, and for example, a windbreak structure is applicable. Ambient sounds may also include human speech. The sound collecting device 14 of this embodiment generates an analog sound signal based on the collected sound. The sound collection device 14 also includes an AD converter that converts sound signals into sound data Ds. Sound data Ds is output from the sound collection device 14 .

The playback device 15 plays back music under the control of the control device 11 based on the masking data Dm generated by the control device 11 . Masking data Dm indicates music. The playback device 15 includes a DA converter, an amplifier, and a speaker. Masking data Dm, which is a digital signal, is input to the DA converter. The input masking data Dm is converted into a masking signal that is an analog signal. The masking signal is amplified by the amplifier so that it has an amplitude suitable for sound emission by the speaker in the subsequent stage. Music indicated by the masking signal whose amplitude is amplified is emitted from a speaker as a sound emitting device. As an example, the masking device 1 according to the present embodiment is preferably used in a vehicle C shown in FIG. be done.

FIG. 2 is an example of a plan view of a vehicle C equipped with the masking device 1 according to this embodiment, and FIG. 3 is an example of a side view of the vehicle C. As shown in FIG.

In the example shown in FIGS. 2 and 3, in addition to the masking device 1, the passenger compartment R of the vehicle C includes four seats 51 to 54 arranged in a rectangular shape, a ceiling 6, a front light door 71, A front left door 72, a rear right door 73, and a rear left door 74 are arranged. Seat 51 is the driver's seat, seat 52 is the passenger's seat, seat 53 is the rear right seat, and seat 54 is the rear left seat. Each of the seats 51 to 54 is made of cloth or leather and has sound absorption properties. The seats 51-54 face in a common direction. Each of the seats 51-54 has a headrest 51-1-54-1.

The masking device 1 includes a microphone as the sound collecting device 14, and a first speaker 15-1, a second speaker 15-2, a third speaker 15-3, and a fourth speaker 15-4, which are elements of the reproducing device 15. is composed of The sound collecting device 14 is arranged on the ceiling 6 of the passenger compartment R. The sound collection device 14 preferably includes a first sound collection device 14-1 and a second sound collection device 14-2. In this case, the first sound collecting device 14-1 is installed on the ceiling 6 of the passenger compartment R near the seats 51 and 52, which are the front seats. Also, the first sound pickup device 14-1 preferably has directivity so as to easily pick up the voices of the persons seated on the seats 51 and 52. FIG. Similarly, the second sound pickup device 14-2 is installed on the ceiling 6 of the passenger compartment R near the seats 53 and 54, which are the rear seats. Also, the second sound pickup device 14-2 preferably has directivity so as to easily pick up the voices of the persons seated on the seats 53 and 54. FIG. However, the configuration of the sound pickup device 14 is not limited to this. It is preferable that the sound pickup device 14 can pick up the voices of the persons sitting on the front seats 51 and 52 and the voices of the persons sitting on the rear seats 53 and 54 separately. , can be of any configuration.

The first speaker 15-1 is installed on the headrest 51-1. The second speaker 15-2 is installed on the headrest 52-1. The third speaker 15-3 is installed on the headrest 53-1. The fourth speaker 15-4 is installed on the headrest 54-1. It should be noted that these installation locations are merely examples, and the present invention is not limited to these. For example, each of the first speaker 15-1 to the fourth speaker 15-4 may be installed under the front right door 71, under the front left door 72, under the rear right door 73, and under the rear left door 74. good.

When the first sound pickup device 14-1 picks up the voices of persons seated on the front seats 51 and 52, the third speaker installed in the headrest 53-1 of the rear seat 53 15-3 and the fourth speaker 15-4 installed in the headrest 54-1 of the seat 54 emit music indicated by the masking signal. This is because if the music indicated by the masking signal is emitted from the first speaker 15-1 and the second speaker 15-2, which are the speakers in the front seats, it may interfere with conversation between the front seats. be.

This makes it possible, for example, to prevent the driver's voice from being heard by a person sitting in the back seat. As a result, the driver can feel secure that his/her conversation is not being overheard by the rear seats, and can concentrate on driving.

On the other hand, when the second sound pickup device 14-2 picks up the voices of persons seated on the rear seats 53 and 54, it is installed on the headrest 51-1 of the front seat 51. Music indicated by the masking signal is emitted from the first speaker 15-1 and the second speaker 15-2 installed on the headrest 52-1 of the seat 52. FIG. This is because if the music indicated by the masking signal is emitted from the third speaker 15-3 and the fourth speaker 15-4, which are the rear seat speakers, there is a risk of interfering with the conversation between the rear seats. be.

This makes it possible to prevent the driver from hearing the conversation of the person sitting in the back seat. Furthermore, using music as the masking sound allows the driver to concentrate on driving.

Note that when both the first sound collection device 14-1 and the second sound collection device 14-2 collect voices of persons seated on the seats 51 to 54, the first speaker 15-1 to the first speaker 15-1 Music indicated by the masking signal is not emitted from any of the four speakers 15-4. This is so as not to disturb the conversation between the front and rear seats.

FIG. 4 is a block diagram illustrating the functional configuration of the control device 11. As shown in FIG. The control device 11 functions as a detection unit 111, an analysis unit 112, an acquisition unit 113, a generation unit 114, and a selection unit 115 by reading the control program PR1 and executing the read control program PR1.

The detection unit 111 detects voice data Dv representing human voice from the sound data Ds output from the sound collection device 14 . The audio data Dv has an unvoiced period with no audio and a voiced period with audio. The detection unit 111 is configured by, for example, a bandpass filter whose passband is the voice band. Sounds indicated by the sound data Ds may include running sounds, musical sounds, etc. in addition to voices. The sound data Dv is extracted from the sound data Ds by the detection unit 111 .

Also, the detection unit 111 outputs a control signal S to the selection unit 115 . When the detection unit 111 detects a voice section from the voice data Dv, the control signal S takes a value indicating "ON". On the other hand, when the detection unit 111 detects a silent period, the control signal S has a value indicating "OFF".

The analysis unit 112 analyzes the voice data Dv detected by the detection unit 111 to generate feature data Df indicating voice features. More specifically, the analysis unit 112 generates feature data Df indicating voice features by analyzing the voice data Dv in the voice section. Here, the "speech features" include at least one of speech pitch, speech level, and speech formants. "Speech pitch" refers to the fundamental frequency of speech. "Audio level" is the volume of audio. A "speech formant" is a frequency band in the frequency spectrum of speech that is more intense than its surroundings. The frequency bands are called "first formant", "second formant", "third formant", and so on in ascending order. The quality of speech is determined by the height of each frequency of a plurality of formants.

In particular, when the feature data Df generated by the analysis unit 112 includes the pitch of the speech or the formant of the speech, the analysis unit 112 analyzes the pitch or the formant of the speech to find out who uttered the speech. , it is possible to determine whether a person is male or female. Specifically, when the pitch of the voice is equal to or greater than a predetermined value, the analysis unit 112 determines that the speaker of the voice is female. On the other hand, when the pitch of the voice is less than the predetermined value, the analysis unit 112 determines that the speaker of the voice is male. Further, when the first formant and the second formant of the vowels included in the speech are equal to or greater than a predetermined value, the analysis unit 112 determines that the speaker of the speech is female. On the other hand, when the first formant and the second formant of the vowels included in the speech are less than the predetermined values, the analysis unit 112 determines that the speaker of the speech is male.

Acquisition unit 113 acquires music data Dx from storage device 12 . As will be described later, the music indicated by the masking data Dm generated by the masking device 1 includes a plurality of parts in one-to-one correspondence with a plurality of tones. The music data Dx includes a plurality of part data Dp1, Dp2, . . . Dpn in one-to-one correspondence with the plurality of parts. n is an integer of 2 or more. In addition, when there is no need to distinguish each part, it is simply referred to as part data Dp.

FIG. 5 is a diagram showing an example of a frequency band of human voice and frequency bands of a plurality of timbres corresponding to a plurality of parts included in music indicated by masking data Dm. In FIG. 5, the top row shows frequencies. The second row shows the code. The example shown in FIG. 5 shows an example in which the same C code is used and the frequency increases by one octave from C0 to C8. The third to ninth rows indicate the frequency bands of human speech. The 10th to 14th rows indicate the frequency band of the performance sound of the musical instrument.

As shown in FIG. 5, human speech has a frequency band from approximately 73 Hz to approximately 1047 Hz.

Bass, especially male voices, has a frequency range of approximately D2 to F4, ie approximately 73 Hz to approximately 350 Hz. The male voice baritone has a vocal range of approximately G2 to G4, or a frequency range of approximately 98 Hz to approximately 392 Hz. The male voice, the tenor, has a vocal range of approximately C3 to C5, or a frequency range of approximately 131 Hz to approximately 523 Hz. Male voice generally has a frequency band from approximately 73 Hz to approximately 523 Hz.

Alto, the female voice, has a vocal range of approximately F3 to E5, or a frequency range of approximately 175 Hz to approximately 659 Hz. The female voice, mezzo-soprano, has a vocal range of approximately A3 to A5, ie, a frequency band of approximately 220 Hz to approximately 880 Hz. The female voice, the soprano, has a vocal range of approximately C4 to C6, ie, a frequency range of approximately 262 Hz to approximately 1047 Hz. Female voice generally has a frequency band from approximately 175 Hz to approximately 1047 Hz.

On the other hand, as shown in FIG. 5, musical instrument performance sounds have a frequency band of approximately 25 Hz to approximately 4400 Hz. As an example, the contrabass corresponding to part data Dp1 has a sound range from E1 to G3, that is, a frequency band from approximately 41 Hz to approximately 196 Hz. The cello corresponding to the part data Dp2 has a frequency range from approximately C2 to C5, that is, approximately 65 Hz to approximately 523 Hz. The viola corresponding to the part data Dp3 has a sound range of approximately C3 to C6, that is, a frequency band of approximately 131 Hz to approximately 1047 Hz. The violin corresponding to part data Dp4 has a sound range from G3 to E7, that is, a frequency band from approximately 196 Hz to approximately 2637 Hz.

Comparing the frequency band of human voice with the frequency band of sound played by a musical instrument, it can be said that the frequency band of male voice is generally included in the frequency band of sound played by a cello. On the other hand, it can be said that the frequency band of female voices is generally included in the frequency band of viola performance sounds.

Each of the parts included in the music indicated by the masking data Dm is associated with the pitch or formant of human speech. As an example, a cello part may be associated with a voice pitch indicating male voice. Alternatively, the cello part may be associated with a formant indicating male voice among voice formants. Similarly, a viola part may be associated with a pitch indicating that it is a female voice, among voice pitches. Alternatively, a viola part may be associated with a formant indicating a female voice among voice formants.

The music data Dx may be MIDI (Musical Instrument Digital Interface) data. When the music data Dx is MIDI data, the music data Dx of a predetermined piece of music includes a plurality of part data Dp each corresponding to each timbre. Here, the timbre corresponding to each part data Dp includes not only instrumental sounds, but also timbres of non-instrumental sounds such as human voices and synthesized sounds. Alternatively, the music data Dx may be PCM data obtained by sampling a music signal. Further, when the music data Dx is PCM data, the music data Dx may be composed of a plurality of PCM data corresponding to a plurality of timbres on a one-to-one basis. When the music data Dx is PCM data in which a plurality of timbres are mixed, the music data Dx is decomposed into PCM data of a plurality of timbres by a well-known sound source separation technique, and a predetermined timbre (cello, viola, etc.) to be used for masking. A plurality of PCM data correspond to part data Dp1-Dpn.

Returning to FIG. 4, the generation unit 114 generates masking data Dm representing music for masking the voice based on the feature data Df generated by the analysis unit 112 . In particular, in the present embodiment, the generation unit 114 selects one part data Dp from among the plurality of part data Dp1 to Dpn included in the music data Dx acquired by the acquisition unit 113 based on the characteristic data Df. . Next, the generation unit 114 generates the masking data Dm by correcting at least one of the pitch of the sound indicated by the selected part data Dp and the level of the sound. When the selected part data Dp is Dps, the masking data Dm is one part data Dps' in which the part data Dps is corrected, and among the plurality of part data Dp1 to Dpn, which are subject to the correction. and part data Dp excluding one part data Dps.

More specifically, when the feature of speech includes formants, the generation unit 114 selects the part data Dps in the range overlapping the formants of the speech. Alternatively, when the pitch is included in the feature of the voice, the generation unit 114 selects the part data Dps of the range including the same frequency as the pitch of the voice. As an example, if the vocal formant or pitch corresponds to a male voice, the generator 114 selects the cello part. On the other hand, if the vocal formant or pitch corresponds to a female voice, the generator 114 selects the viola part.

If there is no appropriate part, the generation unit 114 selects the part data Dps in the range closest to the formant of the voice among the voice features from the existing part data Dp1 to Dpn. . Alternatively, the generation unit 114 selects the part data Dps of the range having the frequency closest to the pitch of the voice among the voice features from the existing part data Dp1 to Dpn.

Then, the generation unit 114 corrects the selected part data Dps so that the sound level indicated by the selected part data Dps is changed according to the voice level, and generates part data Dps'. More specifically, the generator 114 corrects the part data Dps so that the sound indicated by the audio data Dv can be masked by the music to be emitted when the music based on the part data Dps' is emitted from the speaker. Furthermore, the generation unit 114 generates masking data Dm from the corrected part data Dps' and the part data Dp excluding the part data Dps to be corrected among the plurality of part data Dp. In particular, when the level of the voice detected by the detection unit 111 is high, the generation unit 114 increases the level of the sound indicated by the selected part data Dp according to the volume of the voice. correct.

Further, in the present embodiment, when the detecting unit 111 detects the voice section of the voice data Dv while the acquiring unit 113 is reading the music data Dx from the storage device 12, the generating unit 114 performs the above-described Masking data Dm is generated by executing the correction. Furthermore, the generation unit 114 outputs the generated masking data Dm to the selection unit 115 .

In parallel with outputting the masking data Dm, the generation unit 114 outputs the music data Dx acquired from the acquisition unit 113 to the selection unit 115 .

The selection unit 115 selects one of the masking data Dm and the music data Dx based on the control signal S input from the detection unit 111 and outputs it to the reproduction device 15 . More specifically, when the control signal S has a value indicating “ON”, the selection unit 115 selects the masking data Dm and outputs the selected masking data Dm to the reproduction device 15 . On the other hand, when the control signal S has a value indicating “OFF”, the selection unit 115 selects the music data Dx and outputs the selected music data Dx to the reproduction device 15 .

The playback device 15 has a function of converting the format of MIDI data or PCM data into the format of music data. As a result, the reproduction device 15 always reproduces the music indicated by the music data Dx, and in the middle of the reproduction, switches the operation so as to reproduce the music indicated by the masking data Dm. At this time, the generation unit 114 corrects the part data Dps indicating one part of the music originally reproduced. Therefore, a person listening to the music played back by the playback device 15 does not feel uncomfortable.

FIG. 6 is a diagram showing the level of each part data Dp included in the music data Dx and the masking data Dm output by the generator 114. As shown in FIG. Note that the example shown in FIG. 6 shows a case where the human voice indicated by the voice data Dv is male voice. At time t1, the generation unit 114 outputs cello part data Dp2 and other part data Dp1, Dp3, and Dp4 in parallel to the selection unit 115 as music data Dx. During this time, the selection unit 115 outputs the music data Dx to the playback device 15 . At time t2, when the detection unit 111 detects human speech, the analysis unit 112 generates feature data Df indicating speech features including at least one of the level of the speech, the pitch of the speech, and the formants. to generate The generation unit 114 selects cello part data Dp2 as part data Dp whose range includes the same frequency as the pitch of the voice, or part data Dp whose range overlaps with the formants of the voice. Furthermore, the generation unit 114 corrects the part data Dp2 so as to increase the sound level indicated by the cello part data Dp2 according to the level of the voice, and generates part data Dp2'. The generation unit 114 outputs the masking data Dm including the cello part data Dp2 with the raised sound level to the playback device 15 . The sound levels of the other part data Dp1, Dp3 and Dp4 included in the masking data Dm are not changed. Selector 115 selects masking data Dm from music data Dx and masking data Dm based on control signal S, and outputs selected masking data Dm to playback device 15 . At time t3, when the detection unit 111 no longer detects human voice, the generation unit 114 returns the level of the cello part data Dp2 to the original level. After that, the generation unit 114 continues to output the music data Dx including the cello part data Dp2 and the other part data Dp1, Dp3 and Dp4 to the playback device 15 .

Instead of correcting the sound level, or in addition to correcting the sound level, the generator 114 converts the selected part data Dp to the pitch of the sound indicated by the sound data Dv detected by the detector 111. Part data Dp' may be generated by correcting the selected part data Dp so that the pitch of the indicated sound is brought closer. The difference between the pitch of the voice and the pitch of the sound after correction is smaller than the difference between the pitch of the voice and the pitch of the sound before correction. Therefore, the pitch of the voice and the corrected pitch of the sound may not match.

More specifically, the generation unit 114 corrects the selected part data Dp so as to move the key of the sound indicated by the selected part data Dp up or down in octave units according to the pitch of human speech, and corrects the selected part data Dp. Dp' may be generated. As a result, the generation unit 114 can correct only the selected part data Dp in a state in which the music represented by the music data Dx is established as a piece of music without changing the tone of the music indicated by the music data Dx.

Alternatively, the generation unit 114 corrects the selected part data Dp so that the code of the sound indicated by the selected part data Dp is raised or lowered in semitone units according to the pitch of human speech, and generates part data Dp′. may As a result, although the tone of the music indicated by the music data Dx changes, the generator 114 can finely adjust the pitch of the sound indicated by the selected part data Dp. By correcting the pitch of the sound in this way, the pitch of the sound approaches the pitch of the voice, so that the masking effect is improved.

[1-2. Operation of the First Embodiment]
FIG. 7 is a flow chart showing the operation of the masking device 1 according to the first embodiment. The operation of the masking device 1 according to the first embodiment will be described below with reference to FIG.

In step S<b>1 , the acquisition unit 113 acquires music data Dx from the storage device 12 .

In step S<b>2 , the generating section 114 outputs the music data Dx acquired from the acquiring section 113 to the selecting section 115 . The selection unit 115 outputs the music data Dx to the reproduction device 15 .

In step S3, when human voice is detected by the detection unit 111 (S3: YES), the masking device 1 executes the process of step S4. If the detection unit 111 does not detect human voice (S3: NO), the masking device 1 executes the process of step S2.

In step S4, the analysis unit 112 analyzes the audio signal detected by the detection unit 111 to generate feature data Df indicating audio features.

In step S<b>5 , the generation unit 114 generates masking data Dm representing music for masking the voice based on the feature data Df generated by the analysis unit 112 . More specifically, in step S5, the generation unit 114 selects one part data Dps from among the plurality of part data Dp included in the music data Dx acquired by the acquisition unit 113, based on the characteristic data Df. . Next, the generation unit 114 corrects the selected part data Dps so that the sound level indicated by the selected part data Dps is changed according to the feature data Df, and generates part data Dps'. Furthermore, the generation unit 114 generates masking data Dm from the part data Dps' and the part data Dp excluding the part data Dps to be corrected among the plurality of part data Dp. Note that the generation unit 114 may change the pitch of the sound according to the feature data Df instead of or in addition to the sound level indicated by the selected part data Dps. In particular, the generation unit 114 performs the above correction when the detection unit 111 detects a voice section while the acquisition unit 113 is reading the music data Dx from the storage device 12 .

In step S<b>6 , the generation unit 114 outputs the generated masking data Dm to the selection unit 115 . Selector 115 outputs masking data Dm to playback device 15 .

[2. Second Embodiment]
A masking device 1 according to a second embodiment of the present disclosure will be described below. Among the components provided in the masking device 1 according to the second embodiment, the same reference numerals are used for the same components as the components provided in the masking device 1 according to the first embodiment, and descriptions of their functions are omitted. do.

[2-1. Configuration of Second Embodiment]
FIG. 8 is a block diagram illustrating the functional configuration of the control device 11 included in the masking device 1 according to the second embodiment. The masking device 1 according to the second embodiment includes a generator 114A instead of the generator 114 included in the masking device 1 according to the first embodiment.

The generation unit 114A outputs predetermined part data Dp among the plurality of part data Dp to the selection unit 115 as the music data Dx. On the other hand, the generator 114A performs the same correction as the generator 114 does. Then, the generation unit 114A outputs the masking data Dm including the predetermined part data Dp and the corrected one part data Dps′ to the selection unit 115 .

FIG. 9 is a diagram showing the level of each part data Dp included in the masking data Dm generated by the generator 114A. Note that the example shown in FIG. 9 shows a case where the human voice is male voice. At time t1, the generation unit 114 outputs the part data Dp other than the cello to the selection unit 115 in advance as the music data Dx. "Other part data" is, for example, violin part data Dp4. During this time, the selection unit 115 outputs the music data Dx to the playback device 15 . When the detection unit 111 detects human speech at time t2, the analysis unit 112 generates speech feature data Df including at least one of the pitch, level, and formants of the speech. The generation unit 114 selects cello part data Dp2 as part data Dp whose range includes the same frequency as the pitch of the voice, or part data Dp whose range overlaps with the formants of the voice. Further, the generation unit 114 corrects the cello part data Dp2 so that the sound level indicated by the cello part data Dp2 is changed according to the voice level, and generates part data Dp2'. The generating unit 114 outputs masking data Dm including the cello part data Dp2′ whose sound level is corrected and the violin part data Dp4 whose sound level is not corrected to the selecting unit 115 . Selector 115 selects masking data Dm from music data Dx and masking data Dm based on control signal S, and outputs selected masking data Dm to playback device 15 . At time t3, when the detection unit 111 no longer detects human voice, the generation unit 114 stops outputting the corrected part data Dp2' for the cello. After that, the generation unit 114 continues to output the violin part data Dp4, which is the other part data Dp, to the selection unit 115 as the music data Dx. The selection unit 115 outputs the music data Dx to the reproduction device 15 .

[2-2. Operation of Second Embodiment]
Since the operation of the masking device 1 according to the second embodiment is basically the same as the operation of the masking device 1 according to the first embodiment, illustration thereof is omitted.

In step S<b>2 , the generation unit 114</b>A outputs to the selection unit 115 predetermined part data Dp among the plurality of part data Dp included in the music data Dx. The selection unit 115 outputs predetermined part data Dp to the reproduction device 15 as music data Dx.

In step S5, the generation unit 114A performs the same correction as the generation unit 114, and generates masking data Dm including the predetermined part data Dp in step S2 and the corrected one part data Dps'.

[3. Modification]
The above embodiment can be variously modified. Specific modification modes are exemplified below. Two or more aspects arbitrarily selected from the following examples may be combined as appropriate unless contradictory.

[3-1. Modification 1]
In the first and second embodiments described above, the generating units 114 and 114A generate the masking data Dm by correcting the music data Dx acquired from the storage device 12 by the acquiring unit 113 . However, the method of generating masking data Dm in the embodiment of the present invention is not limited to this. For example, the generators 114 and 114A may generate new music and generate masking data Dm corresponding to the generated music. For example, generators 114 and 114A may generate new songs by applying conventional techniques for automatically composing or accompaniment based on designated keys and chords. In this case, the generators 114 and 114A may determine the key based on the pitch of the human voice detected by the detector 111, and automatically generate a new song based on the pre-selected chords. .

[3-2. Modification 2]
In the above-described first and second embodiments, the reproducing device 15 reproduces music as masking sound based on the masking data Dm output from the generator 114 . In this modified example, the playback device 15 may further play back music as a masking sound, especially when human voice is detected by the detection unit 111 .

[4. Note]
For example, the following aspects can be grasped from the above-described embodiments and the like.

The masking device 1 according to the aspect (first aspect) of the present disclosure includes a detection unit 111 that detects an audio signal representing audio from an output signal output from the sound pickup device 14 . The masking device 1 also includes an analysis unit 112 that analyzes the audio signal to generate feature data Df that indicates the feature of the audio. Further, the masking device 1 includes a generation unit 114 that generates masking data Dm representing music for masking voice based on the feature data Df.

With this configuration, the detection unit 111 detects human voice in real time, the analysis unit 112 extracts features of the voice, and the generation unit 114 generates music data Dx corresponding to the features of the voice. It becomes possible. Therefore, the masking device 1 can generate masking data Dm corresponding to human speech in real time, and reproduce music for masking human speech based on the generated masking data Dm. In addition, since the sound used for masking is music, there is an advantage that listening for a long time does not cause fatigue.

In addition, in the example of the first aspect (second aspect), the speech features include at least one of speech pitch, speech level, and speech formants.

With this configuration, as a specific feature, it is possible to generate masking data Dm representing music as a masking sound according to at least one of the pitch, level, and formants of human speech. . For example, it is possible to determine whether the utterance is male or female according to the pitch and formants of human speech, and to generate a masking sound according to the determination result.

Further, the example of the first mode (third mode) further includes an acquisition unit 113 that acquires music data Dx representing music. The generator 114 generates masking data Dm by correcting the music data Dx based on the characteristic data Df.

With this configuration, the masking sound can be easily generated by correcting the pre-stored music data Dx to generate the masking data Dm representing the masking sound.

Further, in the example of the first aspect (fourth aspect), the music includes a plurality of parts corresponding to a plurality of timbres on a one-to-one basis. Further, the music data Dx includes a plurality of part data Dp in one-to-one correspondence with a plurality of parts. Moreover, the generation unit 114 selects one part data Dps from the plurality of part data Dp based on the feature data Df. Furthermore, the generation unit 114 corrects at least one of the pitch of the sound indicated by the one part data Dps and the level of the sound indicated by the one part data Dps based on the feature data Df, thereby generating the masking data Dm. to generate

With this configuration, masking sound is produced by correcting at least one of the pitch and sound level of the sound emitted in the music indicated by the music data Dx in accordance with the characteristics of human speech. Masking data Dm can be generated.

In addition, in the example of the first aspect (fifth aspect), the speech features include speech formants and speech levels. The generation unit 114 selects one part data Dps whose range overlaps the formant of the voice indicated by the feature data Df from among the plurality of part data Dp, and selects one part data Dps according to the level of the voice indicated by the feature data Df. The selected one part data Dps is changed so as to change the sound level indicated by the part data Dps.

With this configuration, for example, it is possible to select part data Dps according to whether the human voice is a male voice or a female voice, and match the level of the selected part data Dps to the level of the human voice. It becomes possible.

Also, in the example of the first aspect (sixth aspect), the audio features include the pitch of the audio and the level of the audio. The generation unit 114 selects one part data Dps whose range includes the same frequency as the pitch of the sound indicated by the feature data Df from among the plurality of part data Dp, and selects according to the level of the sound indicated by the feature data Df. The selected one part data Dps is corrected so as to change the sound level indicated by the one part data Dps.

With this configuration, for example, it is possible to select part data Dps according to whether the human voice is a male voice or a female voice, and match the level of the selected part data Dps to the level of the human voice. It becomes possible.

In addition, in the example of the first aspect (seventh aspect), the speech features include speech formants and speech levels. The generation unit 114 selects one part data Dps whose range overlaps the formants of the speech indicated by the feature data Df from among the plurality of part data Dp, and selects one part data Dps according to the pitch of the speech indicated by the feature data Df. The selected one part data Dps is corrected so as to change the pitch of the part data Dps.

With this configuration, for example, it is possible to select part data Dp according to whether the human voice is a male voice or a female voice, and match the pitch of the selected part data Dp to the pitch of the human voice. It becomes possible.

In addition, in the example of the first mode (eighth mode), the speech feature includes the pitch of the speech. The generation unit 114 selects one part data Dps whose range includes the same frequency as the pitch of the voice indicated by the feature data Df from among the plurality of part data Dp, and selects according to the pitch of the voice indicated by the feature data Df. The selected one part data Dps is corrected so as to change the pitch of the selected one part data Dps.

With this configuration, it is possible, for example, to select a part according to whether the human voice is male or female, and match the pitch of the selected part to the pitch of the human voice.

In addition, in the example of the first mode (the ninth mode), the generation unit 114 moves the key of the selected piece of part data Dp up and down in octave units.

With this configuration, the generation unit 114 can correct only the selected part data Dps in a state of forming a piece of music without changing the tone of the music indicated by the music data Dx.

In addition, in the example of the first aspect (tenth aspect), the generation unit 114 moves the chord of the selected piece of part data Dps up and down in semitone units.

With this configuration, the generator 114 can finely adjust the pitch of the selected part data Dps.

Further, in the example of the first aspect (eleventh aspect), the masking data Dm includes the corrected one part data Dps' and the one part data Dps out of the plurality of part data Dp. and part data Dp.

With this configuration, the part data Dps representing the sound played by one musical instrument is corrected, and the corrected part data Dps' and the part representing the sound played by a musical instrument different from the musical instrument for which the part data Dp was corrected are provided. The masking data Dm can be generated from the data Dp.

The example of the first mode (twelfth mode) further includes a storage device 12 that stores the music data Dx. Acquisition unit 113 reads music data Dx from storage device 12 . If the detection unit 111 detects an audio signal while the acquisition unit 113 is reading the music data Dx, the generation unit 114 performs the above correction.

By having this configuration, the masking device 1 plays a piece of music including performance sounds of a plurality of musical instruments in advance, and only after sensing human voice, can it perform, for example, some of the musical instruments according to the characteristics of the voice. It is possible to make the playing sound louder. As a result, when a masking sound is suddenly output at the same time when a person speaks, it is possible to suppress the sense of discomfort felt by the person speaking.

Also, the example of the first aspect (the thirteenth aspect) further includes a storage device 12 that stores the music data Dx. Acquisition unit 113 reads music data Dx from storage device 12 . When the detection unit 111 does not detect an audio signal, the generation unit 114A outputs a predetermined part out of the plurality of part data Dp as the music data Dx. Further, when an audio signal is detected by the detection unit 111, the generation unit 114A performs the above correction, and outputs the masking data Dm including the predetermined part data Dp and the corrected one part data Dps'. .

With this configuration, the masking device 1 plays music indicated by a certain part data Dp in advance, and only after detecting human voice, according to the characteristics of the voice, the masking device 1 performs the music indicated by the other part data Dps. It is possible to insert music. As a result, when a masking sound is suddenly output at the same time when a person speaks, it is possible to suppress the sense of discomfort felt by the person speaking.

Further, in the example of the first mode (14th mode), the music data Dx may be MIDI data.

With this configuration, it is possible to generate masking data Dm representing a masking sound by correcting MIDI data as music data Dx.

Alternatively, in the example of the first aspect (fifteenth aspect), the music data Dx may be a sound signal.

With this configuration, it is possible to generate the masking data Dm representing the masking sound by correcting the sound signal as the music data Dx.

In addition, in the example of the first aspect (sixteenth aspect), the generation unit 114 generates a new piece of music as music, and generates masking data Dm corresponding to the generated piece of music.

With this configuration, it is possible to automatically generate the melody of the masking sound.

Further, the example of the first mode (17th mode) further includes a reproducing device 15 that reproduces music based on the masking data Dm.

With this configuration, it is possible to reproduce music as a masking sound.

In addition, in the example of the first mode (18th mode), the reproducing device 15 reproduces music when the detection unit 111 detects sound.

With this configuration, it is possible to reproduce music as a masking sound in synchronization with the timing of human speech.

DESCRIPTION OF SYMBOLS 11... Control device, 12... Storage device, 13... Operation device, 14... Sound collection device, 14-1... First sound collection device, 14-2... Second sound collection device, 15... Reproducing device, 15- DESCRIPTION OF SYMBOLS 1... 1st speaker, 15-2... 2nd speaker, 15-3... 3rd speaker, 15-4... 4th speaker, 51-54... Seat, 71... Front right door, 72... Front left door, 73... Rear light door 74 Rear left door 111 Detecting unit 112 Analyzing unit 113 Acquiring unit 114, 114A Generating unit

Claims (18)

a detection unit that detects an audio signal representing audio from an output signal output from a microphone;
an analysis unit that generates feature data indicating features of the voice by analyzing the voice signal;
a generation unit that generates masking data indicating music for masking the voice based on the feature data;
A masking device comprising a 2. The masking device of claim 1, wherein the voice features include at least one of the pitch of the voice, the level of the voice, and the formants of the voice. further comprising an acquisition unit for acquiring music data representing music,
3. The masking apparatus according to claim 1, wherein said generator generates said masking data by correcting said music data based on said feature data. The music includes a plurality of parts that correspond one-to-one with a plurality of tones,
the music data includes a plurality of part data corresponding to the plurality of parts on a one-to-one basis;
The generating unit
selecting one part data from among the plurality of part data based on the feature data;
generating the masking data by correcting at least one of the pitch of the sound indicated by the one part data and the level of the sound indicated by the one part data based on the feature data;
4. A masking device according to claim 3. the features of the speech include formants of the speech and levels of the speech;
The generating unit
Selecting, from among the plurality of part data, one part data whose range overlaps with the formant of the speech indicated by the feature data;
5. Masking according to claim 4, wherein the selected one part data is corrected so as to change the sound level indicated by the selected one part data according to the sound level indicated by the feature data. Device. the audio features include the pitch of the audio and the level of the audio;
The generating unit
Selecting one part data whose range includes the same frequency as the pitch of the voice indicated by the feature data from among the plurality of part data,
5. Masking according to claim 4, wherein the selected one part data is corrected so as to change the sound level indicated by the selected one part data according to the sound level indicated by the feature data. Device. the features of the speech include formants of the speech and pitch of the speech;
The generating unit
Selecting, from among the plurality of part data, one part data whose range overlaps with the formant of the speech indicated by the feature data;
Claims 4 to 6, wherein the selected one part data is corrected so as to change the pitch of the sound indicated by the selected one part data according to the pitch of the voice indicated by the feature data. The masking device according to any one of Claims 1 to 3. the features of the speech include the pitch of the speech;
The generating unit
Selecting one part data whose range includes the same frequency as the pitch of the voice indicated by the feature data from among the plurality of part data,
Claims 4 to 6, wherein the selected one part data is corrected so as to change the pitch of the sound indicated by the selected one part data according to the pitch of the voice indicated by the feature data. The masking device according to any one of Claims 1 to 3. 9. The masking device according to claim 7, wherein said generation unit raises or lowers the key of said selected one part data in octave units. 10. The masking device according to any one of claims 7 to 9, wherein the generation unit raises and lowers the chord of the selected one part data in units of semitones. 11. The masking data according to any one of claims 4 to 10, including the corrected one part data and part data other than the one part data among the plurality of part data. masking device. further comprising a storage unit for storing the music data,
The acquisition unit reads the music data from the storage device,
12. The generation unit according to any one of claims 4 to 11, wherein the generation unit performs the correction when the detection unit detects the audio signal while the acquisition unit is reading the music data. A masking device as described in Clause. further comprising a storage unit for storing the music data,
The acquisition unit reads the music data from the storage unit,
The generating unit
outputting predetermined part data among the plurality of part data as the music data when the audio signal is not detected by the detection unit;
Claims 4 to 11, wherein when the detection unit detects the audio signal, the correction is performed, and the masking data including the predetermined part data and the corrected one part data is output. The masking device according to any one of Claims 1 to 3. 14. The masking device according to any one of claims 3 to 13, wherein said music data is MIDI data. 14. The masking device according to any one of claims 3 to 13, wherein said music data is a sound signal. 3. The masking device according to claim 1, wherein said generating unit generates a new piece of music as said music, and generates said masking data corresponding to said generated piece of music. 17. The masking device according to any one of claims 1 to 16, further comprising a reproducing unit that reproduces the music based on the masking data. 18. The masking device according to claim 17, wherein said reproducing unit reproduces said music when said sound is detected by said detecting unit.

Images