JP2018081285A - Sound processing apparatus, sound processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound processing apparatus, a sound processing method, and program, capable of allowing a user to enjoy as if he/she plays a musical instrument in a hall and the like without giving him/her a feeling of strangeness.SOLUTION: An indirect sound component-generating unit 113 generates an indirect sound component which corresponds to a performance sound of a user. An output control unit 115 outputs a sound formed by mixing the performance sound with the indirect sound component, to an output unit 14. In the output control unit 115, the performance sound thus inputted is provided without passing through the indirect sound component-generating unit 113. The output control unit 115 outputs the sound formed by mixing the performance sound provided without passing through the indirect sound component-generating unit 113 with the indirect sound component, to the output unit 14.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a sound processing device, a sound processing method, and a program.

  A device for enjoying content such as music and movies reproduces the sound field of a hall by adding indirect sound components (such as reverberation components) to sound signals of the content and emitting sound from a speaker. Some have a function (Patent Document 1).

Japanese Patent Application Laid-Open No. 2015-50493

  For example, if a pseudo indirect sound component can be added to the sound of an electronic musical instrument (or electric musical instrument) being played by the user and emitted from a speaker, the user can play the instrument in a hall or the like. You will be able to enjoy the feeling that you are. By the way, when the musical instrument sound is emitted from the speaker, it is desirable that the delay from the performance by the user until the performance sound is emitted from the speaker is small. This is because when this delay is large, the user feels uncomfortable. In this regard, in the additional processing for adding a pseudo indirect sound component to the input sound, the processing amount increases in order to reproduce a more natural sound field. When a pseudo indirect sound component is added to an instrument sound of an electronic musical instrument being played by the user and the sound is emitted from the speaker, the performance sound is played from the speaker after being played by the user due to the delay due to the addition process. There is a possibility that the delay until sound is emitted becomes large, and as a result, the user feels uncomfortable.

  The present invention has been made in view of the above problems, and an object of the present invention is to enable the user to enjoy the feeling of playing a musical instrument in a hall or the like, while preventing the user from feeling uncomfortable. An object is to provide a sound processing device, a sound processing method, and a program.

  In order to solve the above problems, a sound processing apparatus according to the present invention includes an input unit that receives an input of a user's performance sound, a generation unit that generates an indirect sound component corresponding to the performance sound, the performance sound, and the Output control means for outputting to the output means a sound obtained by mixing the indirect sound component, and the output control means is supplied with the inputted performance sound without being supplied through the generation means, and the output The control means outputs to the output means a sound obtained by mixing the performance sound supplied without going through the generating means and the indirect sound component.

  Further, the sound processing method according to the present invention includes an input step for receiving an input of a user's performance sound, and a step of generating the indirect sound component via a generation means for generating an indirect sound component corresponding to the performance sound; An output control step for outputting to the output means a sound obtained by mixing the performance sound and the indirect sound component, and in the output control step, the performance sound supplied without the generation means; A sound obtained by mixing the indirect sound is output to the output means.

  Further, the program according to the present invention includes a generation unit that generates an indirect sound component corresponding to a user's performance sound, and an output control that outputs a sound obtained by mixing the performance sound and the indirect sound component to an output unit. And a computer functioning as the means, wherein the performance sound input is supplied to the output control means without going through the generation means, and the output control means is supplied without going through the generation means. A sound obtained by mixing the sound and the indirect sound component is output to the output means. An information storage medium according to the present invention is a computer-readable information storage medium recording the above program.

  In the present invention, “performance” refers to an action of making a sound, and “performance” includes not only the action of playing a musical instrument but also the action of singing a song. That is, the “performance sound” includes not only the performance sound of the musical instrument but also the singing sound.

  According to the present invention, it is possible to make it possible for the user to enjoy the feeling of playing a musical instrument in a hall or the like, while avoiding the user from feeling uncomfortable.

It is a figure which shows the structure of the system provided with the sound processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of a user's performance environment. It is a functional block diagram of the sound processing apparatus which concerns on 1st Embodiment. It is a flowchart which shows the process performed with the sound processing apparatus which concerns on 1st Embodiment. It is a figure for demonstrating an example of the production | generation method of an indirect sound component. It is a functional block diagram of the sound processing apparatus which concerns on 2nd Embodiment. It is a flowchart which shows the process performed with the sound processing apparatus which concerns on 2nd Embodiment. It is a figure for demonstrating the sound emitted from a speaker. It is a functional block diagram of the sound processing apparatus which concerns on 3rd Embodiment. It is a flowchart which shows the process performed with the sound processing apparatus which concerns on 3rd Embodiment. It is a figure for demonstrating the sound emitted from a speaker. It is a functional block diagram of the sound processing apparatus which concerns on 4th Embodiment. It is a flowchart which shows the process performed with the sound processing apparatus which concerns on 4th Embodiment. It is a functional block diagram of the sound processing apparatus which concerns on 5th Embodiment. It is a flowchart which shows the process performed with the sound processing apparatus which concerns on 5th Embodiment. It is a functional block diagram of the sound processing apparatus which concerns on 6th Embodiment. It is a flowchart which shows the process performed with the sound processing apparatus which concerns on 6th Embodiment. It is a functional block diagram of the sound processing apparatus which concerns on 7th Embodiment. It is a flowchart which shows the process performed with the sound processing apparatus which concerns on 7th Embodiment. It is a functional block diagram of the sound processing apparatus which concerns on 8th Embodiment. It is a flowchart which shows the process performed with the sound processing apparatus which concerns on 8th Embodiment. It is a functional block diagram of the sound processing apparatus which concerns on 9th Embodiment. It is a flowchart which shows the process performed with the sound processing apparatus which concerns on 9th Embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

  [First Embodiment] First, the first embodiment will be described. FIG. 1 shows the configuration of a system including a sound processing apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the system includes a sound processing device 1, a content reproduction device 2, a microphone 3, an electronic musical instrument 4, an electric musical instrument 5, a speaker 6 (an example of sound emission means), and a display device 7. Note that the content playback apparatus 2 may play back content (such as music or video) stored in an optical storage medium, or play back content distributed via a network, for example. Also good.

  The sound processing device 1 is an AV receiver, for example. The sound processing device 1 includes a CPU 11, a memory 12, an input unit 13, an output unit 14, an acoustic signal processing unit 15, and a video signal processing unit 16.

  The CPU 11 controls the input unit 13, the output unit 14, the audio signal processing unit 15, and the video signal processing unit 16 based on a program stored in the memory 12 and executes information processing. Although omitted in FIG. 1, the sound processing apparatus 1 is provided with a network interface for performing data communication via a network, and the program is downloaded via the network and stored in the memory 12. Alternatively, the sound processing apparatus 1 includes a component for reading a program from an information storage medium such as a memory card, and the program is read from the information storage medium and stored in the memory 12.

  The input unit 13 can accept input of an audio signal and a video signal based on content data from the content reproduction device 2, supply the audio signal to the audio signal processing unit 15, and supply the video signal to the video signal processing unit 16. Supply.

  The input unit 13 can also accept an input of a user's performance sound. “Performance” indicates an act of producing a sound, and “performance” includes not only an act of playing an instrument but also an act of singing a song. For this reason, the “performance sound” includes not only the performance sound of the musical instrument but also the singing sound. In the following, the performance sound of an instrument is referred to as “instrument sound” for convenience.

  For example, the input unit 13 is connected to the microphone 3 and can receive an input of an acoustic signal output from the microphone 3, and supplies the acoustic signal to the acoustic signal processing unit 15. The microphone 3 collects sound and outputs the collected sound as an acoustic signal. The microphone 3 is used to input an instrumental sound of an acoustic instrument played by a user and a user's singing sound to the sound processing device 1.

  Further, for example, the input unit 13 is connected to the electronic musical instrument 4 or the electric musical instrument 5 played by the user, and can receive an input of an acoustic signal output from the electronic musical instrument 4 or the electric musical instrument 5. Is supplied to the acoustic signal processing unit 15.

  Note that the input unit 13 may include a wireless network interface, and an acoustic signal may be input to the input unit 13 via wireless communication. In other words, content sounds and performance sounds may be input to the sound processing device 1 via wireless communication.

  The acoustic signal processing unit 15 is a DSP (Digital Signal Processor), for example, and executes processing related to the acoustic signal according to control from the CPU 11. The acoustic signal output from the acoustic signal processing unit 15 is emitted from the speaker 6 via the output unit 14.

  The video signal processing unit 16 is, for example, a DSP (Digital Signal Processor), and executes processing related to the video signal in accordance with control from the CPU 11. The video signal output from the video signal processing unit 16 is displayed on the display device 7 via the output unit 14.

  In the sound processing apparatus 1 according to the first embodiment, a user who plays an acoustic instrument or sings a song at home or the like can enjoy the feeling of playing in a hall or the like. Hereinafter, a configuration for realizing such a function will be described. As shown in FIG. 1, the sound processing device 1 has a function of receiving input of musical instrument sounds of the electronic musical instrument 4 or the electric musical instrument 5 and content reproduced by the content reproduction device 2 by the speaker 6 or the display device 7. Although the output function is provided, these functions are not essential in the first embodiment.

  FIG. 2 shows an example of the performance environment of the user. In the example shown in FIG. 2, the microphone 3 is installed in front of the user U. The microphone 3 is used to collect the performance sound of the user. For example, when the user is playing an acoustic instrument, the instrument sound is collected by the microphone 3 and input to the input unit 13. For example, when the user sings a song, the singing sound is collected by the microphone 3 and input to the input unit 13.

  In the example shown in FIG. 2, a plurality of speakers 6A, 6B, 6C, 6D, and 6E are installed. Specifically, a speaker 6A is installed in front of the user U. Speakers 6B and 6C are installed on the left front and right front as viewed from the user U, and speakers 6D and 6E are installed on the left rear and right rear as viewed from the user U, respectively. In the example shown in FIG. 2, five speakers 6A to 6E are installed, but four or less speakers 6 may be installed, or six or more speakers 6 may be installed. For example, only the speakers 6B and 6C may be installed.

  FIG. 3 is a functional block diagram showing functions realized by the sound processing apparatus 1 according to the first embodiment. As shown in FIG. 3, the sound processing apparatus 1 according to the first embodiment includes a performance sound adjustment unit 101, a preprocessing unit 102 (an example of a first processing unit), an indirect sound component generation unit 103, and a post processing unit 104. (An example of second processing means) and an output control unit 105. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, when the CPU 11 controls the acoustic signal processing unit 15 according to a program, the above functional block is realized.

  FIG. 4 is a flowchart showing processing executed by the sound processing apparatus 1 according to the first embodiment. Hereinafter, the function of each functional block will be described with reference to FIG.

  First, the performance sound adjusting unit 101 adjusts the performance sound by performing a predetermined process on the performance sound input from the microphone 3 (S10). For example, the performance sound adjustment unit 101 performs howling reduction processing for reducing howling in the microphone 3 on the performance sound. Further, for example, the performance sound adjustment unit 101 performs effect processing (for example, processing for deleting unnecessary frequency bands or adjusting the sound pressure level before generating the indirect sound) on the performance sound. Also good. The performance sound processed by the performance sound adjustment unit 101 is supplied to the preprocessing unit 102.

  The preprocessing unit 102 performs preprocessing on the supplied sound (here, the performance sound) (S11). For example, the preprocessing unit 102 performs sound adjustment processing using an equalizer on the supplied sound. The performance sound that has been processed by the preprocessing unit 102 is supplied to the indirect sound component generation unit 103. In FIG. 3, the performance sound adjustment unit 101 and the preprocessing unit 102 are shown as separate functional blocks, but they may be configured integrally.

  The indirect sound component generation unit 103 generates a pseudo indirect sound component corresponding to the performance sound (S12). That is, the indirect sound component generation unit 103 assumes a case where a performance sound is emitted in an acoustic space such as a hall, and generates an indirect sound component (such as a reverberation component) generated in the acoustic space in that case. Various known methods can be employed as a method of generating the pseudo indirect sound component. For example, the indirect sound component generation unit 103 may generate the position of the indirect sound (reverberation sound) in the assumed acoustic space, the delay time of the indirect sound relative to the direct sound, the ratio of the level of the indirect sound to the sound pressure level of the direct sound, etc. Based on the information, a pseudo indirect sound component corresponding to the performance sound is generated.

  For example, the indirect sound component generation unit 103 includes an indirect sound component addition unit that adds an indirect sound component corresponding to the supplied sound to the supplied sound, and the indirect sound component generation unit 103 performs the performance sound. Is supplied to the indirect sound component adding unit. And the indirect sound component production | generation part 103 acquires only an indirect sound component by removing the original performance sound from the sound (performance sound to which the indirect sound component was added) output from the indirect sound component addition part.

  FIG. 5 is a diagram for explaining an example of a method for generating an indirect sound component. FIG. 5A shows an example of a performance sound. This performance sound corresponds to a direct sound component. For example, the performance sound (direct sound component) shown in FIG. 5A is stored in each of the first buffer and the second buffer. The indirect sound component adding unit adds an indirect sound component corresponding to the performance sound to the performance sound (direct sound component) stored in the first buffer. Here, various known methods can be adopted as a method of adding the indirect sound component. In this case, as shown in FIG. 5B, for example, the direct sound component and the indirect sound component of the performance sound are stored in the first buffer. Thereafter, the indirect sound component generation unit 103 uses the direct sound component of the performance sound stored in the second buffer (see FIG. 5B) from the direct sound component and the indirect sound component of the performance sound stored in the first buffer (FIG. 5B). By subtracting FIG. 5 (A)), only the indirect sound component as shown in FIG. 5 (C) is acquired.

  The method for generating the indirect sound component is not limited to the above example. For example, the performance sound (direct sound component) shown in FIG. 5A may be stored in the first buffer, and the indirect sound component corresponding to the performance sound (direct sound component) may be generated in the second buffer. .

  The indirect sound component generated by the indirect sound component generation unit 103 is supplied to the post processing unit 104. The post-processing unit 104 performs post-processing on the supplied sound (indirect sound component here) (S13). For example, the post processing unit 104 performs processing for adjusting the supplied sound in accordance with the characteristics of the speaker 6. The indirect sound component that has been processed by the post-processing unit 104 is supplied to the output control unit 105.

  The output control unit 105 outputs the supplied indirect sound component to the output unit 14 (an example of output means) (S14). That is, the output control unit 105 outputs the indirect sound component to the output unit 14 while restricting the performance sound (instrumental instrument sound or singing sound of the acoustic musical instrument) input from the microphone 3 from being output to the output unit 14. The indirect sound component output to the output unit 14 is emitted by the speaker 6.

  Here, “restricting the performance sound from being output to the output unit 14” means, for example, preventing the performance sound from being output to the output unit 14. That is, the output control unit 105 outputs only the indirect sound component to the output unit 14 without outputting the performance sound (direct sound component) input from the microphone 3 to the output unit 14. In other words, the output control unit 105 prevents the performance sound (direct sound component) input from the microphone 3 from being emitted from the speaker 6 and causes only the indirect sound component to be emitted from the speaker 6.

  “Restricting the output of the performance sound to the output unit 14” means, for example, outputting the performance sound to the output unit 14 so that the performance sound is emitted at a considerably lower volume than the indirect sound component. Good. That is, the output control unit 105 emits a performance sound (direct sound component) input from the microphone 3 at a considerably lower volume (a volume that is difficult to hear by the user's ears) than the normal volume. The indirect sound component may be output to the output unit 14 at a normal volume while being output to the output unit 14. In other words, the output control unit 105 causes the performance sound (direct sound component) input from the microphone 3 to be emitted from the speaker 6 at a considerably lower volume than the normal volume, and the indirect sound component is the normal volume. The sound is emitted from the speaker 6.

  When the speaker 6 is built in the sound processing device 1, the output control unit 105 outputs the supplied indirect sound component to the speaker 6 (another example of output means).

  According to the sound processing device 1 according to the first embodiment described above, a pseudo indirect sound component (such as a reverberation component) corresponding to a user's performance sound (instrument sound or singing sound of an acoustic instrument) is transmitted from the speaker 6. Since the sound is emitted, the user can enjoy the feeling of playing an acoustic instrument or singing a song in a hall or church. In addition, according to the sound processing device 1 according to the first embodiment, since the user's performance sound (instrumental sound or singing sound of an acoustic instrument) is restricted from being emitted from the speaker 6, It is possible to prevent the user from feeling uncomfortable due to the ability to hear performance sounds emitted from different positions.

  [Second Embodiment] Next, a second embodiment will be described. The hardware configuration of the sound processing apparatus 1 according to the second embodiment is the same as that of the first embodiment. The user's performance environment is basically the same as in the first embodiment. However, in the second embodiment, since the electronic musical instrument 4 or the electric musical instrument 5 connected to the input unit 13 of the sound processing device 1 is played by the user, the microphone 3 is unnecessary.

  In the sound processing apparatus 1 according to the second embodiment, a user who is playing the electronic musical instrument 4 or the electric musical instrument 5 at home or the like can enjoy the feeling of playing in a hall or the like. Hereinafter, a configuration for realizing such a function will be described. As shown in FIG. 1, the sound processing device 1 has a function of outputting the content reproduced by the content reproduction device 2 through the speaker 6 and the display device 7, but these functions are the second embodiment. Then it is not essential.

  FIG. 6 is a functional block diagram showing functions realized by the sound processing apparatus 1 according to the second embodiment. As shown in FIG. 6, the sound processing apparatus 1 according to the second embodiment includes a performance sound adjustment unit 111, a preprocessing unit 112, an indirect sound component generation unit 113, a post processing unit 114, and an output control unit 115. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, when the CPU 11 controls the acoustic signal processing unit 15 according to a program, the above functional block is realized.

  FIG. 7 is a flowchart showing processing executed by the sound processing apparatus 1 according to the second embodiment. Hereinafter, the function of each functional block will be described with reference to FIG.

  First, the performance sound adjustment unit 111 adjusts the performance sound by performing a predetermined process on the performance sound input from the electronic musical instrument 4 or the electric musical instrument 5 (S20). For example, the performance sound adjustment unit 111 performs effect processing (for example, distortion processing on the guitar sound) on the performance sound. Note that the performance sound adjusting unit 111 does not execute a process that generates a large delay, and only performs a process with a small delay. The performance sound that has been processed by the performance sound adjustment unit 111 is supplied to the preprocessing unit 112.

  The preprocessing unit 112 performs preprocessing on the supplied sound (performance sound here) (S21). Further, the indirect sound component generation unit 113 generates a pseudo indirect sound component corresponding to the performance sound (S22). Then, the post processing unit 114 performs post processing on the supplied sound (indirect sound component here) (S23). Steps S21 to S23 are basically the same as steps S11 to S13 of the first embodiment, and the preprocessing unit 112, the indirect sound component generation unit 113, and the postprocessing unit 114 are the preprocessing unit 102 of the first embodiment, Since it is basically the same as the indirect sound component generation unit 103 and the post processing unit 104, description thereof is omitted here.

  The performance sound that has been processed by the performance sound adjustment unit 111 is also supplied to the output control unit 115 via the path 119. A path 119 is a path that reaches the output control unit 115 without going through the preprocessing unit 112, the indirect sound component generation unit 113, and the postprocessing unit 114. In other words, the path 119 is a path with less delay compared to the path reaching the output control unit 115 via the preprocessing unit 112, the indirect sound component generation unit 113, and the post processing unit 114. For example, the preprocessing unit 112, the indirect sound component generation unit 113, and the post processing unit 114 execute processing based on the performance sound stored in the buffer. However, in the path 119, the performance sound is stored in the buffer. Without being supplied to the output control unit 115.

  The output control unit 115 mixes the performance sound (direct sound component) supplied via the path 119 and the indirect sound component generated by the indirect sound component generation unit 113 and outputs the mixed sound to the output unit 14. (S24). The mixed sound output to the output unit 14 is emitted by the speaker 6.

  FIG. 8 is a diagram for explaining the sound emitted from the speaker 6. Here, as shown in FIG. 8A, it is assumed that the performance sound B is input after the performance sound A is input. These performance sounds A and B correspond to direct sound components. In this case, the indirect sound component generation unit 113 generates an indirect sound component A corresponding to the performance sound A and supplies the indirect sound component A to the output control unit 115 as shown in FIG. The indirect sound component generation unit 113 generates the indirect sound component B corresponding to the performance sound B after the indirect sound component A is generated, but is omitted here.

  Since the amount of processing in the preprocessing unit 112, the indirect sound component generation unit 113, and the post processing unit 114 is large, and the processing in these functional blocks requires time, the indirect sound component A is generated in these functional blocks. The sound is emitted from the speaker 6 with a delay corresponding to the time required for processing. On the other hand, the performance sounds A and B (direct sound components) are emitted from the speaker 6 via a path 119 with little delay (a path without substantial delay). For this reason, as shown in FIG. 8 (C), the indirect sound component A corresponding to the performance sound A is delayed with respect to the actual sound, mixed with the performance sound B after the performance sound A, and the mixed sound. Is emitted from the speaker 6.

  According to the sound processing apparatus 1 according to the second embodiment described above, pseudo indirect sound components (such as reverberation components) are added to the user's performance sound (instrument sound of the electronic musical instrument 4 or the electric musical instrument 5). Since the sound is emitted from the speaker 6, the user can enjoy the feeling of playing a musical instrument in a hall or the like. Further, according to the sound processing apparatus 1 according to the second embodiment, the performance sound of the user is emitted from the speaker 6 via the path 119 with a small delay, and thus the performance sound is played by the user after being played by the user. The delay until sound is emitted from can be reduced. As a result, it is possible to prevent the user from feeling uncomfortable due to a large delay from when the performance sound is played by the user until the performance sound is emitted from the speaker 6.

  In the sound processing device 1 according to the second embodiment, the indirect sound component corresponding to the performance sound of the user is generated later than the indirect sound component generated when the performance sound is emitted in the actual acoustic space. However, even if some delay occurs in the indirect sound component, there is no possibility that the user will feel uncomfortable, so there is no particular problem.

  [Third Embodiment] Next, a third embodiment will be described. The hardware configuration of the sound processing apparatus 1 according to the third embodiment is the same as that of the first embodiment. The user's performance environment is the same as in the second embodiment.

  In the sound processing apparatus 1 according to the third embodiment, a user who is playing the electronic musical instrument 4 or the electric musical instrument 5 at home or the like enjoys the feeling of playing in a hall or the like as a member of a music content player. Is possible. Hereinafter, a configuration for realizing such a function will be described.

  FIG. 9 is a functional block diagram showing functions realized by the sound processing apparatus 1 according to the third embodiment. As shown in FIG. 9, the sound processing apparatus 1 according to the third embodiment includes a performance sound adjustment unit 121, a preprocessing unit 122, an indirect sound component generation unit 123, a post processing unit 124, an output control unit 125, and a content decoding unit. Part 126 is included. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, when the CPU 11 controls the acoustic signal processing unit 15 according to a program, the above functional block is realized.

  FIG. 10 is a flowchart showing processing executed by the sound processing apparatus 1 according to the third embodiment. Hereinafter, the function of each functional block will be described with reference to FIG.

  First, the content decoding unit 126 converts the multi-channel content sound input from the content reproduction apparatus 2 into a PCM signal by format decoding (S30).

  Further, the performance sound adjustment unit 121 adjusts the performance sound by performing a predetermined process on the performance sound input from the electronic musical instrument 4 or the electric musical instrument 5 (S31). Since step S31 is the same as step S20 of the second embodiment, and the performance sound adjustment unit 121 is the same as the performance sound adjustment unit 111 of the second embodiment, the description thereof is omitted here.

  In FIG. 10, for the sake of convenience, steps S30 and S31 are shown to be executed in order, but steps S30 and S31 are executed in parallel.

  The content sound converted into the PCM signal is mixed with the performance sound converted into the PCM signal by the AD conversion circuit (S32), and the mixed sound is supplied to the preprocessing unit 122. Note that the performance sound is also supplied to the output control unit 125 via the path 129. The route 129 is the same as the route 119 of the second embodiment.

  The preprocessing unit 122 performs preprocessing on the mixed sound (S33). For example, the preprocessing unit 122 performs an audio adjustment process using an equalizer on the mixed sound. The mixed sound that has been processed by the preprocessing unit 122 is supplied to the indirect sound component generation unit 123.

  The indirect sound component generation unit 123 generates a pseudo indirect sound component corresponding to the mixed sound (S34). That is, the indirect sound component generation unit 123 generates an indirect sound component corresponding to the performance sound (direct sound component) and the content sound. The indirect sound component generation unit 123 assumes a case where the mixed sound is emitted in an acoustic space such as a hall, and generates an indirect sound component (such as a reverberation component) generated in the acoustic space in that case. Various known methods can be employed as a method of generating the pseudo indirect sound component.

  For example, the indirect sound component generation unit 123 performs a process of adding an indirect sound to the mixed sound stored in the first buffer, and then stores the mixed sound in the second buffer from the sound stored in the first buffer. The indirect sound component corresponding to the mixed sound is acquired by subtracting the original mixed sound. The indirect sound component generation unit 123 generates an indirect sound corresponding to the mixed sound in the second buffer by executing a process of generating the indirect sound based on the mixed sound stored in the first buffer. By doing so, you may make it acquire the indirect sound component corresponding to the said mixed sound.

  The indirect sound component generated by the indirect sound component generation unit 123 is supplied to the output control unit 125 through the post-processing unit 124 together with the content sound.

  The post processing unit 124 executes post processing (S35). Step S35 is basically the same as step S13 of the first embodiment, and the post-processing unit 124 is basically the same as the post-processing unit 104 of the first embodiment, and thus description thereof is omitted here.

  The output control unit 125 mixes the performance sound (direct sound component) supplied via the path 129 and the content sound and indirect sound component supplied from the post-processing unit 124, and sends the mixed sound to the output unit 14. Output (S36). The mixed sound output to the output unit 14 is emitted by the speaker 6.

  FIG. 11 is a diagram for explaining the sound emitted from the speaker 6. Here, as shown in FIG. 11A, it is assumed that the performance sound B and the content sound B are input after the performance sound A and the content sound A are input. The performance sounds A and B correspond to direct sound components. In FIG. 11, for convenience, the performance sound A and the content sound A are shown as being slightly shifted in time, but the input time points of the performance sound A and the content sound A are the same. The same applies to the performance sound B and the content sound B.

  In the case of the example shown in FIG. 11A, the indirect sound component generation unit 123 generates an indirect sound component A corresponding to the mixed sound of the performance sound A and the content sound A as shown in FIG. The indirect sound component A is supplied to the output control unit 125 together with the content sound A. The indirect sound component generation unit 113 generates the indirect sound component B corresponding to the mixed sound of the performance sound B and the content sound B after the indirect sound component A is generated, but is omitted here. doing.

  The amount of processing in the preprocessing unit 122, the indirect sound component generation unit 123, and the post processing unit 124 is large, and it takes time for processing in these functional blocks. The sound is emitted from the speaker 6 with a delay of a delay time corresponding to the time required for processing in the functional block. On the other hand, the performance sounds A and B (direct sound components) are emitted from the speaker 6 via a path 129 with little delay (a path without substantial delay). For this reason, as shown in FIG. 11 (C), the indirect sound component A is delayed from the actual time and mixed with the performance sound B after the performance sound A, and the mixed sound is emitted from the speaker 6. Is done.

  According to the sound processing apparatus 1 according to the third embodiment described above, a pseudo indirect sound component with respect to the user's performance sound (instrument sound of the electronic musical instrument 4 or the electric musical instrument 5) and the multi-channel content sound. Since (severance component etc.) is added and sound is emitted from the speaker 6, the user can enjoy the feeling of being performed in a hall or the like as a member of a music content player. Further, according to the sound processing apparatus 1 according to the third embodiment, the performance sound of the user is emitted from the speaker 6 via the path 129 with a small delay, and therefore the performance sound is played by the user after being played by the user. The delay until sound is emitted from can be reduced. As a result, it is possible to prevent the user from feeling uncomfortable due to a large delay from when the performance sound is played by the user until the performance sound is emitted from the speaker 6.

  [Fourth Embodiment] Next, a fourth embodiment will be described. The hardware configuration of the sound processing apparatus 1 according to the fourth embodiment is the same as that of the first embodiment. The user's performance environment is the same as in the first embodiment.

  In the sound processing apparatus 1 according to the fourth embodiment, the user can enjoy the feeling of being a member of a music content player and singing a song in a hall or playing an acoustic instrument. . Hereinafter, a configuration for realizing such a function will be described.

  FIG. 12 is a functional block diagram showing functions realized by the sound processing apparatus 1 according to the fourth embodiment. As shown in FIG. 12, the sound processing apparatus 1 according to the fourth embodiment includes a performance sound adjustment unit 131, a preprocessing unit 132, an indirect sound component generation unit 133, a post processing unit 134, an output control unit 135, and a content decoding unit. Part 136. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, when the CPU 11 controls the acoustic signal processing unit 15 according to a program, the above functional block is realized.

  FIG. 13 is a flowchart showing processing executed by the sound processing apparatus 1 according to the fourth embodiment. Hereinafter, the function of each functional block will be described with reference to FIG.

  First, the content decoding unit 136 converts the multi-channel content sound input from the content reproduction apparatus 2 into a PCM signal by format decoding (S40). Step S40 is basically the same as step S30 of the third embodiment, and the content decoding unit 136 is basically the same as the content decoding unit 126 of the third embodiment.

  However, the content decoding unit 136 of the fourth embodiment includes a specific component removing unit 136A. In step S40, the specific component removing unit 136A removes the specific component included in the content sound. Specifically, the specific component removing unit 136A removes the specific component corresponding to the performance sound input from the microphone 3 from the content sound. For example, when the user's singing sound is input from the microphone 3, the specific component removing unit 136A removes the vocal component from the content sound. Since the vocal component is often included in the center channel in the multi-channel content sound, the specific component removing unit 136A removes the vocal component from the content sound by removing the center channel. The method of removing the vocal component from the content sound is not limited to this method, and various known methods can be employed. For example, when the instrument sound of an acoustic instrument is input from the microphone 3, the specific component removal unit 136A may remove the instrument sound component of the acoustic instrument from the content sound. Note that the type of performance sound input from the microphone 3 (for example, singing sound, guitar sound, piano sound, or the like) may be automatically determined by analyzing the performance sound. The user may specify via the input device.

  The performance sound adjustment unit 131 adjusts the performance sound by performing predetermined processing on the performance sound input from the microphone 3 (S41). Step S41 is the same as step S10 of the first embodiment, and the performance sound adjustment unit 131 is the same as the performance sound adjustment unit 101 of the first embodiment, and thus description thereof is omitted here.

  In FIG. 13, for convenience, steps S40 and S41 are shown to be executed in order, but steps S40 and S41 are executed in parallel.

  The content sound converted into the PCM signal is mixed with the performance sound converted into the PCM signal by the AD conversion circuit (S42), and the mixed sound is supplied to the preprocessing unit 132. Based on the mixed sound, processing by the preprocessing unit 132, the indirect sound component generation unit 133, and the post processing unit 134 is executed (S43, S44, S45). Steps S43 to S45 are basically the same as steps S33 to S35 of the third embodiment, and the preprocessing unit 132, the indirect sound component generation unit 133, and the post processing unit 134 are the preprocessing unit 122 of the third embodiment, Since it is basically the same as the indirect sound component generation unit 123 and the post processing unit 124, description thereof is omitted here.

  Note that the performance sound is supplied to the output control unit 135 via the path 139. The route 139 is the same as the route 119 of the second embodiment.

  Similar to the output control unit 125 of the third embodiment, the output control unit 135 includes the performance sound (direct sound component) supplied via the path 139 and the content sound and indirect sound component supplied from the post-processing unit 134. And the mixed sound is output to the output unit 14 (S46). The mixed sound output to the output unit 14 is emitted by the speaker 6.

  According to the sound processing device 1 according to the fourth embodiment described above, a pseudo indirect sound component (reverberation) with respect to a user's performance sound (singing sound or instrumental sound of an acoustic instrument) and multi-channel content sound. Component) and the like, and the sound is emitted from the speaker 6, the user can enjoy the feeling of being a member of the music content player and singing a song in a hall or playing an acoustic instrument. . Further, according to the sound processing apparatus 1 according to the fourth embodiment, the performance sound of the user is emitted from the speaker 6 through the path 139 with a small delay, so that the performance sound is played after being played by the user. The delay until sound is emitted from can be reduced. As a result, it is possible to prevent the user from feeling uncomfortable due to a large delay from when the performance sound is played by the user until the performance sound is emitted from the speaker 6.

  Furthermore, according to the sound processing apparatus 1 according to the fourth embodiment, for example, when the user is singing a song, the vocal component included in the content sound is removed, so the user becomes a vocal of the music content. You can enjoy the feeling of singing in a hall.

  In the above description, the vocal component of the content sound is removed before the performance sound and the content sound are mixed. However, the removal of the vocal component of the content sound is performed by mixing the performance sound and the content sound. It may be done later.

  [Fifth Embodiment] Next, a fifth embodiment will be described. The hardware configuration of the sound processing apparatus 1 according to the fifth embodiment is the same as that of the first embodiment. The user's performance environment is the same as in the first embodiment or the second embodiment.

  Also in the sound processing apparatus 1 according to the fifth embodiment, it is possible to enjoy the feeling that a user is playing in a hall or the like as a member of a music content performer. In particular, in the sound processing apparatus 1 according to the fifth embodiment, it is possible to feel a sense of unity between the user's performance sound and the content sound. Hereinafter, a configuration for realizing such a function will be described.

  FIG. 14 is a functional block diagram showing functions implemented by the sound processing apparatus 1 according to the fifth embodiment. As shown in FIG. 14, the sound processing apparatus 1 according to the fifth embodiment includes a performance sound adjustment unit 141, a preprocessing unit 142, an indirect sound component generation unit 143, a post processing unit 144, an output control unit 145, and a content decoding unit. 146 and a content sound adjustment unit 147. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, when the CPU 11 controls the acoustic signal processing unit 15 according to a program, the above functional block is realized.

  FIG. 15 is a flowchart showing processing executed by the sound processing apparatus 1 according to the fifth embodiment. Hereinafter, the function of each functional block will be described with reference to FIG.

  First, the content decoding unit 146 converts the multi-channel content sound input from the content reproduction device 2 into a PCM signal by format decoding (S50). Step S50 is basically the same as step S30 in the third embodiment, and the content decoding unit 146 is basically the same as the content decoding unit 126 in the third embodiment, and thus description thereof is omitted here.

  The content sound adjustment unit 147 adjusts the content sound in order to match the characteristics of the performance sound and the content sound (S51). The content sound adjustment unit 147 includes an indirect sound component removal unit 147A. The indirect sound component removing unit 147A removes the indirect sound component included in the content sound in order to match the amount of the indirect sound component between the performance sound and the content sound. Various known methods can be adopted as a method for removing the indirect sound component included in the content sound.

  The performance sound input from the electronic musical instrument 4 or the electric musical instrument 5 includes only the direct sound component and does not include the indirect sound component, whereas the content sound includes the direct sound component and the indirect sound component. There is. For this reason, for example, the indirect sound component removal unit 147A removes the indirect sound component included in the content sound and outputs only the direct sound component of the content sound. For example, the indirect sound component removing unit 147A identifies an indirect sound component included in the content sound, and removes the indirect sound component by lowering the sound pressure level of the identified indirect sound component. That is, the indirect sound component removing unit 147A removes the indirect sound component almost completely by lowering the sound pressure level of the indirect sound component to zero (almost zero).

  Note that the indirect sound component removing unit 147A may remove (reduce) the indirect sound component to some extent by lowering the sound pressure level of the indirect sound component to some extent. That is, “removing the indirect sound component” includes not only removing the indirect sound component almost completely but also removing (reducing) the indirect sound component to some extent. Here, “a certain degree” is such a level that the user does not feel uncomfortable even if the indirect sound component remains.

  The performance sound adjustment unit 131 adjusts the performance sound (S52). Step S52 is the same as steps S10 and 20 of the first embodiment or the second embodiment, and the performance sound adjustment unit 141 is the same as the performance sound adjustment units 110 and 111 of the first embodiment or the second embodiment. The description is omitted here.

  In FIG. 15, for the sake of convenience, steps S50, S51 and step S52 are shown to be executed in order, but steps S50, S51 and step S52 are executed in parallel.

  The content sound from which the indirect sound component has been removed (direct sound component) is mixed with the performance sound (S53), and the mixed sound is supplied to the indirect sound component generation unit 143 via the preprocessing unit 142. The preprocessing unit 142 performs preprocessing on the mixed sound (S54). Since step S54 is the same as step S11 of the first embodiment, and the preprocessing unit 142 is the same as the preprocessing unit 112 of the first embodiment, description thereof is omitted here.

  The indirect sound component generation unit 143 generates a pseudo indirect sound component corresponding to the mixed sound (the direct sound component of the content sound and the direct sound component of the performance sound) (S55). Step S55 is basically the same as step S34 of the third embodiment, and the indirect sound component generation unit 143 is basically the same as the indirect sound component generation unit 123 of the third embodiment, and thus description thereof is omitted here. To do.

  The indirect sound component generated by the indirect sound component generation unit 143 is supplied to the output control unit 145 via the post processing unit 144. The post processing unit 144 executes post processing (S56). Step S56 is the same as step S13 of the first embodiment, and the post-processing unit 144 is the same as the post-processing unit 114 of the first embodiment, and thus description thereof is omitted here. As shown in FIG. 14, the content sound from which the indirect sound component is removed (that is, the direct sound component of the content sound) is also supplied to the output control unit 145. The performance sound (direct sound component) is supplied to the output control unit 145 via the path 149. The route 149 is the same as the route 129 of the second embodiment.

  The output control unit 145 mixes the performance sound (direct sound component) supplied via the path 149, the content sound (direct sound component), and the indirect sound component generated by the indirect sound component generation unit 143, The mixed sound is output to the output unit 14 (S57). The mixed sound output to the output unit 14 is emitted by the speaker 6.

  According to the sound processing apparatus 1 according to the fifth embodiment described above, the user can enjoy the feeling of playing in a hall or the like as a member of a music content player. In addition, according to the sound processing device 1 according to the fifth embodiment, it is possible to match the amounts of indirect sound components of the user performance sound and the content sound, and as a result, the user performance sound and the content sound The user can feel a sense of unity.

  As shown in FIG. 14, the instrument sound or singing sound of an acoustic instrument may be input from the microphone 3 also in the fifth embodiment. However, in this case, since the performance sound may include an indirect sound component, the performance sound adjustment unit 141 may remove the indirect sound component included in the performance sound.

  [Sixth Embodiment] Next, a sixth embodiment will be described. The sixth embodiment is a modification of the fifth embodiment. In the sound processing apparatus 1 according to the sixth embodiment, the indirect sound component of the content sound is removed only when a performance sound is input.

  FIG. 16 is a functional block diagram showing functions realized by the sound processing apparatus 1 according to the sixth embodiment, and FIG. 17 is a flowchart showing processing executed by the sound processing apparatus 1 according to the sixth embodiment. It is. As shown in FIG. 16, the sound processing apparatus 1 according to the sixth embodiment includes a performance sound adjustment unit 141, a preprocessing unit 142, an indirect sound component generation unit 143, a post processing unit 144, an output control unit 145, and a content decoding unit. 146, a content sound adjustment unit 147, and an input detection unit 148. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, when the CPU 11 controls the acoustic signal processing unit 15 according to a program, the above functional block is realized.

  The sound processing apparatus 1 according to the sixth embodiment includes an input detection unit 148 and is different from the fifth embodiment in that steps S51A and 51B are included instead of step S51. Hereinafter, differences from the fifth embodiment will be mainly described.

  In the sound processing apparatus 1 according to the sixth embodiment, the input detection unit 148 detects that a performance sound is input from the electronic musical instrument 4, the electric musical instrument 5, or the microphone 3. The indirect sound component removal unit 147A removes the indirect sound component included in the content sound according to the detection result of the input detection unit 148. Specifically, when it is detected that a performance sound is input (S51A: Yes), the indirect sound component removing unit 147A removes the indirect sound component from the content sound (S51B). On the other hand, when it is not detected that the performance sound is input (S51A: No), the indirect sound component removing unit 147A does not remove the indirect sound component from the content sound. Although omitted in FIG. 17, in this case, steps S52 and S53 are not executed, only the content sound is supplied to the preprocessing unit 142, and the content sound is output to the output unit 14 in step S57.

  In the sound processing apparatus 1 according to the sixth embodiment described above, the indirect sound component is removed from the content sound only when a performance sound is input. When the performance sound is not input, it is not necessary to match the amount of the indirect sound component between the performance sound and the content sound, and it is not necessary to remove the indirect sound component from the content sound. In this regard, according to the sound processing device 1 according to the sixth embodiment, when it is not necessary to remove the indirect sound component from the content sound, the process of removing the indirect sound component from the content sound is not executed. It becomes possible to reduce the processing load of the processing apparatus 1.

  [Seventh Embodiment] Next, a seventh embodiment will be described. The hardware configuration of the sound processing apparatus 1 according to the seventh embodiment is the same as that of the first embodiment. The user's performance environment is the same as in the first embodiment or the second embodiment.

  In the fifth and sixth embodiments, in order to match the amount of the indirect sound component of the content sound with the performance sound, the indirect sound component included in the content sound is removed, whereas the sound processing according to the seventh embodiment is performed. In the device 1, an indirect sound component is added to the performance sound in accordance with the amount of the indirect sound component of the content sound.

  FIG. 18 is a functional block diagram showing functions realized by the sound processing apparatus 1 according to the seventh embodiment. As shown in FIG. 18, the sound processing apparatus 1 according to the seventh embodiment includes a performance sound adjustment unit 151, a preprocessing unit 152, an indirect sound component generation unit 153, a post processing unit 154, an output control unit 155, and a content decoding unit. 156 and an indirect sound component amount analysis unit 157. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, when the CPU 11 controls the acoustic signal processing unit 15 according to a program, the above functional block is realized.

  FIG. 19 is a flowchart showing processing executed by the sound processing apparatus 1 according to the seventh embodiment. Hereinafter, the function of each functional block will be described with reference to FIG.

  First, the content decoding unit 156 converts the multi-channel content sound input from the content reproduction apparatus 2 into a PCM signal by format decoding (S60). Step S60 is basically the same as step S30 of the third embodiment, and the content decoding unit 156 is basically the same as the content decoding unit 126 of the third embodiment, and thus description thereof is omitted here.

  The indirect sound component amount analysis unit 157 analyzes the amount of the indirect sound component included in the content sound (S61). For example, the indirect sound component amount analysis unit 157 analyzes the number and magnitude (sound pressure level) of indirect sound components included in the content sound. Various known methods can be adopted as a method of analyzing the amount of indirect sound component included in the content sound.

  The performance sound adjustment unit 151 adjusts the performance sound (S62). Step S62 is basically the same as steps S10 and S20 of the first embodiment or the second embodiment, and the performance sound adjustment unit 151 is basically the same as the performance sound adjustment units 101 and 111 of the first embodiment or the second embodiment. The same.

  However, the performance sound adjustment unit 151 of the seventh embodiment also plays a role of adjusting the performance sound in order to match the characteristics of the performance sound and the content sound. That is, the performance sound adjustment unit 151 includes an indirect sound component addition unit 151A. In step S62, the indirect sound component addition unit 151A adds an indirect sound component corresponding to the performance sound to the performance sound. In particular, the indirect sound component adding unit 151A sets the amount of the indirect sound component added to the performance sound based on the analysis result of the indirect sound component amount analyzing unit 157. That is, the indirect sound component adding unit 151A sets the number and size of indirect sound components added to the performance sound in accordance with the number and size of indirect sound components included in the content sound. That is, the indirect sound component adding unit 151A sets the number and magnitude of the indirect sound components added to the performance sound to the same extent as the number and magnitude of the indirect sound components included in the content sound.

  In the sound processing device 1 according to the seventh embodiment, the content sound and the performance sound to which the indirect sound component is added by the indirect sound component adding unit 151A are mixed (S63), and the mixed sound is supplied to the preprocessing unit 152. Is done. Based on the mixed sound, processing by the preprocessing unit 152, the indirect sound component generation unit 153, and the post processing unit 154 is executed (S64, S65, S66). Steps S64 to S66 are basically the same as steps S33 to S35 of the third embodiment, and the preprocessing unit 152, the indirect sound component generation unit 153, and the post processing unit 154 are the preprocessing unit 122 of the third embodiment, Since it is basically the same as the indirect sound component generation unit 123 and the post processing unit 124, description thereof is omitted here.

  The performance sound to which the indirect sound component is added by the indirect sound component adding unit 151A is supplied to the output control unit 155 via the path 159. The route 159 is the same as the route 119 of the second embodiment.

  The output control unit 155 mixes the content sound, the performance sound to which the indirect sound component is added by the indirect sound component adding unit 151A, and the indirect sound component generated by the indirect sound component generation unit 153, and outputs the mixed sound. The data is output to the output unit 14 (S67). The mixed sound output to the output unit 14 is emitted by the speaker 6.

  According to the sound processing apparatus 1 according to the seventh embodiment described above, the user can enjoy the feeling of playing in a hall or the like as a member of a music content player. In addition, according to the sound processing device 1 according to the seventh embodiment, it is possible to match the amounts of indirect sound components of the user performance sound and the content sound, and as a result, the user performance sound and the content sound The user can feel a sense of unity.

  As shown in FIG. 18, the instrument sound or singing sound of an acoustic instrument may also be input from the microphone 3 in the seventh embodiment. However, in this case, since the indirect sound component may be included in advance in the performance sound included from the microphone 3, the performance sound adjusting unit 151 once removes the indirect sound component included in the performance sound, and then the indirect sound component. The indirect sound component may be added to the performance sound by the adding unit 151A.

  [Eighth Embodiment] Next, an eighth embodiment will be described. The eighth embodiment is a modification of the seventh embodiment. In the sound processing apparatus 1 according to the eighth embodiment, the method of adding the indirect sound to the performance sound is changed according to the type of the performance sound.

  FIG. 20 is a functional block diagram illustrating functions realized by the sound processing device 1 according to the eighth embodiment, and FIG. 21 is a flowchart illustrating processing executed by the sound processing device 1 according to the eighth embodiment. It is. As shown in FIG. 20, the sound processing apparatus 1 according to the eighth embodiment includes a performance sound adjustment unit 151, a preprocessing unit 152, an indirect sound component generation unit 153, a post processing unit 154, an output control unit 155, and a content decoding unit. 156, an indirect sound component amount analysis unit 157, and a performance sound type identification unit 158. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, when the CPU 11 controls the acoustic signal processing unit 15 according to a program, the above functional block is realized.

  The sound processing apparatus 1 according to the eighth embodiment is different from the seventh embodiment in that it includes a performance sound type specifying unit 158 and includes steps S62A and S62B instead of step S62. Hereinafter, differences from the seventh embodiment will be mainly described.

  In the sound processing apparatus 1 according to the eighth embodiment, the performance sound type specifying unit 158 specifies the type of the input performance sound (step S62A). For example, the performance sound type identification unit 158 determines whether or not the input performance sound is an instrument sound. When the input performance sound is an instrument sound, the performance sound type specifying unit 158 specifies the type of the instrument sound. In other words, the performance sound type identification unit 158 identifies which of the multiple types of musical instruments (for example, guitar, violin, piano, etc.) the input performance sound is. Further, for example, the performance sound type identification unit 158 determines whether or not the input performance sound is a singing sound. Various known methods can be adopted as a method for specifying the type of performance sound.

  In addition, the indirect sound component adding unit 151A of the eighth embodiment is based on the indirect sound corresponding to the performance sound based not only on the analysis result of the indirect sound component amount analysis unit 157 but also on the specification result of the performance sound type specifying unit 158. The component is added to the performance sound (S62B). That is, the indirect sound component adding unit 151A sets the indirect sound component to be added to the performance sound based on the result of the specification of the performance sound type specifying unit 158 as well as the result of the analysis of the indirect sound component amount analysis unit 157. To do.

  Since the radiation characteristic of the performance sound in the actual acoustic space is different for each type of performance sound, the indirect sound component adding unit 151A adds the indirect sound to the performance sound based on the radiation characteristic that is different for each type of performance sound. Set ingredients.

  For example, since the instrumental sound of a guitar tends to be emitted in the front direction compared to other directions, the indirect sound component adding unit 151A sets the channel corresponding to the frontal direction when the performance sound is the instrumental sound of the guitar. Indirect sound components (reverberation components, etc.) are added to it. Alternatively, the indirect sound component adding unit 151A makes the amount of the indirect sound component added to the channel corresponding to the front direction larger than the amount of the indirect sound component added to the other channels.

  Further, for example, since the violin instrument sound tends to be radiated upward as compared to other directions, the indirect sound component adding unit 151 </ b> A has a channel corresponding to the upward direction when the performance sound is a violin instrument sound. Indirect sound components (reverberation components, etc.) are added to. Alternatively, the indirect sound component adding unit 151A makes the amount of the indirect sound component added to the channel corresponding to the upward direction larger than the amount of the indirect sound component added to the other channel.

  Further, for example, since the singing sound tends to be emitted in the front direction compared to other directions, when the performance sound is a singing sound, the indirect sound component adding unit 151A is indirectly connected to the channel corresponding to the front direction. Adds sound components (such as reverberation components). Alternatively, the indirect sound component adding unit 151A increases the amount of the indirect sound component added to the channel corresponding to the front direction than the amount of the indirect sound component added to the other channels.

  According to the sound processing device 1 according to the eighth embodiment described above, it is possible to add an indirect sound component to the performance sound based on the radiation characteristics of the user's performance sound, and a more natural indirection. Sound components can be added to the performance sound.

  [Ninth Embodiment] Next, a ninth embodiment will be described. The hardware configuration of the sound processing apparatus 1 according to the ninth embodiment is the same as that of the first embodiment. The user's performance environment is the same as in the first embodiment or the second embodiment.

  In the sixth embodiment to the eighth embodiment, the amounts of indirect sound components are matched between the performance sound and the content sound, whereas in the sound processing device 1 according to the ninth embodiment, the timbre is changed between the performance sound and the content sound. It is designed to match.

  FIG. 22 is a functional block diagram showing functions realized by the sound processing apparatus 1 according to the ninth embodiment. As shown in FIG. 22, the sound processing apparatus 1 according to the ninth embodiment includes a performance sound adjustment unit 161, a preprocessing unit 162, an indirect sound component generation unit 163, a post processing unit 164, an output control unit 165, and a content decoding unit. 166, a first timbre analysis unit 167, and a second timbre analysis unit 168. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, when the CPU 11 controls the acoustic signal processing unit 15 according to a program, the above functional block is realized.

  FIG. 23 is a flowchart showing processing executed by the sound processing apparatus 1 according to the ninth embodiment. Hereinafter, the function of each functional block will be described with reference to FIG.

  First, the content decoding unit 166 converts the multi-channel content sound input from the content reproduction apparatus 2 into a PCM signal by format decoding (S70). Step S70 is basically the same as step S30 of the third embodiment, and the content decoding unit 166 is basically the same as the content decoding unit 126 of the third embodiment, and thus description thereof is omitted here.

  The second timbre analysis unit 168 analyzes the timbre of the content sound (S71). For example, when a plurality of types of instrument sounds are included in the content sound, the second timbre analysis unit 168 identifies a type of instrument sound included in the performance sound among the plurality of types of instrument sounds, and the instrument sound Analyzing the tone of For example, when the performance sound includes a singing sound, the second timbre analysis unit 168 analyzes the timbre of the singing sound included in the content sound. Various known methods can be adopted as a method for specifying an instrument sound or a singing sound included in the content sound and a method for analyzing a tone color of an instrument sound or a singing sound.

  The first timbre analysis unit 167 analyzes the timbre of the performance sound (S72). The first timbre analysis unit 167 identifies an instrument sound or singing sound included in the performance sound, and analyzes the timbre of the instrument sound or singing sound. Various known methods can be employed as a method for analyzing the tone color of the performance sound.

  In FIG. 23, for convenience, steps S70 and 71 and step S72 are shown to be executed in order, but steps S70, S71 and step S72 are executed in parallel.

  The performance sound adjustment unit 161 adjusts the performance sound (S73). Step S62 is basically the same as steps S10 and S20 of the first embodiment or the second embodiment, and the performance sound adjustment unit 161 is basically the same as the performance sound adjustment units 101 and 111 of the first embodiment or the second embodiment. The same.

  However, the performance sound adjustment unit 161 of the ninth embodiment also plays a role of adjusting the performance sound to match the characteristics of the performance sound and the content sound. That is, the performance sound adjustment unit 161 includes a timbre adjustment unit 161A. In step S73, the timbre adjustment unit 161A is based on a comparison between the analysis result of the first timbre analysis unit 167 and the analysis result of the second timbre analysis unit 168. Adjust the tone of the performance sound.

  For example, the first timbre analysis unit 167 obtains an analysis result that there are many high-frequency components of a violin sound (violin instrument sound) included in the performance sound, and the high-frequency component of the violin sound included in the content sound is When the second timbre analysis unit 168 obtains an analysis result indicating that there is little, the timbre adjustment unit 161A reduces the high-frequency component of the violin sound included in the performance sound. In short, if the amount of the specific band component of the instrument sound included in the performance sound and the amount of the specific band component of the same type of instrument sound included in the content sound are different, the specific band component and content of the instrument sound included in the performance sound The tone color adjustment unit 161A adjusts the specific band component of the instrument sound included in the performance sound so as to be set to the same level as the high frequency component of the same type of instrument sound included in the sound.

  In the sound processing apparatus 1 according to the ninth embodiment, the content sound and the performance sound whose tone color has been adjusted by the tone color adjusting unit 161A are mixed (S74), and the mixed sound is supplied to the preprocessing unit 162. Based on the mixed sound, processing by the preprocessing unit 162, the indirect sound component generation unit 163, and the post processing unit 164 is executed (S75, S76, S77). Steps S75 to S77 are basically the same as steps S33 to S35 of the third embodiment, and the preprocessing unit 162, the indirect sound component generation unit 163, and the postprocessing unit 164 are the preprocessing unit 122 of the third embodiment, Since it is basically the same as the indirect sound component generation unit 123 and the post processing unit 124, description thereof is omitted here.

  Note that the performance sound whose tone color has been adjusted by the tone color adjusting unit 161A is supplied to the output control unit 165 via the path 169. The route 169 is the same as the route 119 of the second embodiment.

  Similar to the output control unit 125 of the third embodiment, the output control unit 165 outputs the performance sound supplied through the path 169 (the performance sound whose tone color has been adjusted by the tone color adjustment unit 161A) and the post-processing unit 164. The supplied content sound and indirect sound component are mixed and the mixed sound is output to the output unit 14 (S78). The mixed sound output to the output unit 14 is emitted by the speaker 6.

  According to the sound processing apparatus 1 according to the ninth embodiment described above, the user can enjoy the feeling of playing in a hall or the like as a member of a music content player. In addition, according to the sound processing device 1 according to the ninth embodiment, it is possible to match the tone of the user performance sound and the content sound, and as a result, the user's sense of unity of the performance sound and the content sound can be obtained. Will be able to feel enough.

  Instead of the timbre adjustment unit 161A that adjusts the timbre of the performance sound, the timbre that adjusts the timbre of the content sound based on the comparison between the analysis result of the first timbre analysis unit 167 and the analysis result of the second timbre analysis unit 168. An adjustment unit may be provided.

  The timbre adjustment unit obtains, for example, an analysis result that the high frequency component of the violin sound included in the performance sound is obtained by the first timbre analysis unit 167, and the high frequency component of the violin sound included in the content sound is obtained. When the second timbre analysis unit 168 obtains an analysis result indicating that there is little, the high frequency component of the violin sound included in the content sound may be increased.

  In addition, for example, the timbre adjustment unit obtains an analysis result by the first timbre analysis unit 167 that the high frequency component of the violin sound (violin instrument sound) is high in the performance sound, and the violin sound in the content sound. When the second timbre analysis unit 168 obtains an analysis result indicating that the high-frequency component of the guitar sound, which is a different instrument sound, is obtained, the high-frequency component of the guitar sound included in the content sound is reduced. Also good.

  Note that both a tone color adjustment unit 161A that adjusts the tone color of the performance sound and a tone color adjustment unit that adjusts the tone color of the content sound may be provided. For example, the first timbre analysis unit 167 obtains an analysis result that the high frequency component of the violin sound included in the performance sound is obtained, and the second timbre analysis unit 168 determines the high frequency of the violin sound included in the content sound. If the analysis result indicates that there are few band components, the high frequency component of the violin sound included in the performance sound and the high frequency component of the violin sound included in the content sound should be set to the same level. The high frequency component of the violin sound included and the high frequency component of the violin sound included in the performance sound may be adjusted respectively.

  [Modification] The present invention is not limited to the first to ninth embodiments described above.

  For example, a plurality of the first to ninth embodiments may be combined.

  Further, for example, the above description is based on the assumption that the sound processing apparatus 1 is an AV receiver, and the functions described above can be realized by using an AV receiver. A device other than the receiver may be used, and the functions described above may be realized by a device other than the AV receiver. For example, the sound processing device 1 may be built in a speaker. Further, for example, the sound processing device 1 may be realized by a desktop computer, a laptop computer, a tablet computer, a smartphone, or the like.

  [Appendix] As can be understood from the description of the embodiment described above, various technical ideas including the invention described below are disclosed in this specification.

  The sound processing apparatus according to the present invention mixes an input means for receiving an input of a performance sound of a user, a generation means for generating an indirect sound component corresponding to the performance sound, and the performance sound and the indirect sound component. Output control means for outputting the sound to the output means, the performance sound input is supplied to the output control means without going through the generation means, and the output control means includes the generation means. A sound obtained by mixing the performance sound supplied without intervention and the indirect sound component is output to the output means.

  Further, the sound processing method according to the present invention includes an input step for receiving an input of a user's performance sound, and a step of generating the indirect sound component via a generation means for generating an indirect sound component corresponding to the performance sound; An output control step for outputting to the output means a sound obtained by mixing the performance sound and the indirect sound component, and in the output control step, the performance sound supplied without the generation means; A sound obtained by mixing the indirect sound is output to the output means.

  Further, the program according to the present invention includes a generation unit that generates an indirect sound component corresponding to a user's performance sound, and an output control that outputs a sound obtained by mixing the performance sound and the indirect sound component to an output unit. And a computer functioning as the means, wherein the performance sound input is supplied to the output control means without going through the generation means, and the output control means is supplied without going through the generation means. A sound obtained by mixing the sound and the indirect sound component is output to the output means. An information storage medium according to the present invention is a computer-readable information storage medium recording the above program.

  In the above invention, the output control means outputs to the output means a sound obtained by mixing the performance sound and an indirect sound component corresponding to the performance sound input before the performance sound. It may be.

  In the above invention, the generation means generates an indirect sound component corresponding to a sound obtained by mixing the performance sound and a content sound that is obtained based on content data, and the output control means includes the output sound A sound obtained by mixing the performance sound, the content sound, and the indirect sound component supplied without using the generation means may be output to the output means.

  In the said invention, when the said user's song sound is input as said performance sound, you may make it include the means to remove the song sound contained in the said content sound.

  In the above invention, when the content sound includes the specifying means for specifying the type of instrument sound included in the performance sound of the user, and the type of instrument sound specified by the specifying means, the instrument sound is used as the content sound. And means for removing the sound from the sound.

  In the above invention, it includes an adding means for adding an indirect sound component corresponding to the supplied sound to the supplied sound, and the generating means outputs a sound obtained by mixing the performance sound and the content sound. The indirect sound component may be generated by supplying to the adding means and removing the original performance sound and the content sound from the sound output from the adding means.

  In the above invention, the first processing means for adjusting the supplied sound and supplying it to the adding means, and the second processing means for adjusting the sound output from the adding means in accordance with the characteristics of the sound emitting means, The performance sound may be supplied to the first processing means.

  In the above invention, the input means receives an input of the performance sound through a microphone, and the sound processing device performs a howling reduction process for reducing howling in the microphone with respect to the input performance sound. The generating means may generate an indirect sound component corresponding to the performance sound subjected to the howling reduction processing.

  1 sound processing device, 2 content playback device, 3 microphone, 4 electronic device, 5 electrical device, 6, 6A, 6B, 6C, 6D, 6E speaker, 11 CPU, 12 memory, 13 input unit, 14 output unit, 15 sound Signal processing unit, 16 video signal processing unit, 101, 111, 121, 131, 141, 151, 161 Performance sound adjustment unit, 102, 112, 122, 132, 142, 152, 162 Preprocessing unit, 103, 113, 123 , 133, 143, 153, 163 Indirect sound component generation unit, 104, 114, 124, 134, 144, 154, 164 Post processing unit, 105, 115, 125, 135, 145, 155, 165 Output control unit, 119, 129, 139, 149, 159, 169 route, 126, 136, 146, 156 66 content decoding unit, 136A specific component removing unit, 147 content sound adjusting unit, 147A indirect sound component removing unit, 148 input detecting unit, 151A indirect sound component adding unit, 157 indirect sound component analyzing unit, 158 performance sound type specifying unit, 161A tone color adjustment unit, 167 first tone color analysis unit, 168 second tone color analysis unit, U user.

Claims (10)

Input means for receiving input of the user's performance sound;
Generating means for generating an indirect sound component corresponding to the performance sound;
Output control means for outputting to the output means a sound obtained by mixing the performance sound and the indirect sound component;
Including
The output control means is supplied with the input performance sound without going through the generation means, and the output control means is supplied with the performance sound without going through the generation means, and the indirect sound component. Output the sound that is mixed to the output means,
A sound processing apparatus. The sound processing apparatus according to claim 1,
The output control means outputs to the output means a sound obtained by mixing the performance sound and an indirect sound component corresponding to the performance sound input before the performance sound.
A sound processing apparatus. The sound processing device according to claim 1 or 2,
The generating means generates an indirect sound component corresponding to a sound obtained by mixing the performance sound and a content sound that is a sound obtained based on content data,
The output control means outputs to the output means a sound obtained by mixing the performance sound supplied without going through the generating means, the content sound, and the indirect sound component;
A sound processing apparatus. The sound processing apparatus according to claim 3,
Means for removing the singing sound included in the content sound when the user singing sound is input as the performance sound;
A sound processing apparatus. The sound processing apparatus according to claim 3 or 4,
Specifying means for specifying the type of instrument sound included in the user's performance sound;
Means for removing the instrument sound from the content sound when the content sound includes the type of instrument sound specified by the specifying means,
A sound processing apparatus. The sound processing apparatus according to any one of claims 2 to 5,
Adding means for adding an indirect sound component corresponding to the supplied sound to the supplied sound;
The generating means supplies a sound obtained by mixing the performance sound and the content sound to the adding means, and removes the original performance sound and the content sound from the sound output from the adding means. Generating the indirect sound component;
A sound processing apparatus. The sound processing device according to claim 6,
First processing means for adjusting the supplied sound and supplying it to the additional means;
Second processing means for adjusting the sound output from the adding means in accordance with the characteristics of the sound emitting means,
The performance sound is supplied to the first processing means.
A sound processing apparatus. The sound processing device according to any one of claims 1 to 7,
The input means accepts an input of the performance sound via a microphone,
The sound processing apparatus includes means for performing howling reduction processing for reducing howling in the microphone on the input performance sound,
The generating means generates an indirect sound component corresponding to the performance sound that has been subjected to the howling reduction processing.
A sound processing apparatus. An input step for receiving an input of a user's performance sound;
Generating the indirect sound component via generating means for generating an indirect sound component corresponding to the performance sound;
An output control step of outputting to the output means a sound obtained by mixing the performance sound and the indirect sound component;
Including
In the output control step, a sound obtained by mixing the performance sound supplied without going through the generation unit and the indirect sound is output to the output unit.
And a sound processing method. Generating means for generating an indirect sound component corresponding to the user's performance sound; and
Output control means for outputting to the output means a sound obtained by mixing the performance sound and the indirect sound component;
Function as a computer
The output control means is supplied with the input performance sound without going through the generation means, and the output control means is supplied with the performance sound without going through the generation means, and the indirect sound component. Output the sound that is mixed to the output means,
A program characterized by that.

Images