JP6819236B2 - Sound processing equipment, sound processing methods, and programs - Google Patents

Description

The present invention relates to a sound processing device, a sound processing method, and a program.

For devices for enjoying content such as music and movies, the sound field of the hall is reproduced by adding a pseudo indirect sound component (reverberation component, etc.) to the acoustic signal of the content and emitting sound from the speaker. Some have a function (Patent Document 1).

Japanese Unexamined Patent Publication No. 2015-50493

For example, if a pseudo indirect sound component can be added to the musical instrument sound of the musical instrument played by the user or the singing sound of the user and the sound is emitted from the speaker together with the music content, the user is a performer of the music content. You will be able to enjoy the feeling of playing an instrument or singing a song as a member. However, in this case, since the characteristics of the user's musical instrument sound (or singing sound) and the content sound are different, the user cannot sufficiently feel the sense of unity between the user's musical instrument sound (or singing sound) and the content sound. There is a risk. For example, by making the amount of the indirect sound component of the user's instrument sound (or singing sound) different from the amount of the indirect sound component of the content sound, the user's instrument sound (or singing sound) and the content sound can be united. The user may not be able to fully feel it.

The present invention has been made in view of the above problems, and an object of the present invention is to enable a user to perform according to a content while feeling a sense of unity between the user's musical instrument sound or singing sound and the content sound. The purpose of the present invention is to provide a sound processing device, a sound processing method, and a program.

In order to solve the above problems, the sound processing device according to the present invention has characteristics of an input means for receiving input of a user's performance sound, the performance sound, and a content sound which is a sound obtained based on the content data. An adjustment means for adjusting at least one of the performance sound and the content sound, a generation means for generating an indirect sound component corresponding to a sound obtained by mixing the performance sound and the content sound, and the above-mentioned It includes an output control means for outputting a sound formed by mixing the performance sound, at least the direct sound component of the content sound, and the indirect sound component to the output means.

Further, the sound processing method according to the present invention is an adjustment step of adjusting at least one of the performance sound and the content sound in order to match the characteristics of the user's performance sound and the content sound obtained based on the content data. A generation step of generating an indirect sound component corresponding to a sound obtained by mixing the performance sound and the content sound, the performance sound, at least a direct sound component of the content sound, and the indirect sound component. It includes an output control step for outputting the mixed sound to the output means.

Further, the program according to the present invention is an adjusting means for adjusting at least one of the performance sound and the content sound in order to match the characteristics of the user's performance sound and the content sound obtained based on the content data. A generation means for generating an indirect sound component corresponding to a sound obtained by mixing a performance sound and the content sound, and the performance sound, at least a direct sound component of the content sound, and the indirect sound component are mixed. This is a program for operating a computer as an output control means for outputting a sound to an output means. The information storage medium according to the present invention is a computer-readable information storage medium on which the above program is recorded.

In the present invention, "performance" means an act of producing a sound, and "performance" includes not only an act of playing a musical instrument but also an act of singing a song. That is, the "performance sound" includes not only the performance sound of the musical instrument but also the singing sound.

According to the present invention, the user can perform a performance according to the content while feeling a sense of unity between the user's musical instrument sound or singing sound and the content sound.

It is a figure which shows the structure of the system provided with the sound processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the performance environment of a user. It is a functional block diagram of the sound processing apparatus which concerns on 1st Embodiment. It is a flow chart which shows the process executed by the sound processing apparatus which concerns on 1st Embodiment. It is a figure for demonstrating an example of the method of generating an indirect sound component. It is a functional block diagram of the sound processing apparatus which concerns on 2nd Embodiment. It is a flow figure which shows the process executed by the sound processing apparatus which concerns on 2nd Embodiment. It is a figure for demonstrating the sound emitted from a speaker. It is a functional block diagram of the sound processing apparatus which concerns on 3rd Embodiment. It is a flow chart which shows the process executed by the sound processing apparatus which concerns on 3rd Embodiment. It is a figure for demonstrating the sound emitted from a speaker. It is a functional block diagram of the sound processing apparatus which concerns on 4th Embodiment. It is a flow diagram which shows the process executed by the sound processing apparatus which concerns on 4th Embodiment. It is a functional block diagram of the sound processing apparatus which concerns on 5th Embodiment. It is a flow chart which shows the process executed by the sound processing apparatus which concerns on 5th Embodiment. It is a functional block diagram of the sound processing apparatus which concerns on 6th Embodiment. It is a flow chart which shows the process executed by the sound processing apparatus which concerns on 6th Embodiment. It is a functional block diagram of the sound processing apparatus which concerns on 7th Embodiment. It is a flow chart which shows the process executed by the sound processing apparatus which concerns on 7th Embodiment. It is a functional block diagram of the sound processing apparatus which concerns on 8th Embodiment. It is a flow chart which shows the process executed by the sound processing apparatus which concerns on 8th Embodiment. It is a functional block diagram of the sound processing apparatus which concerns on 9th Embodiment. It is a flow chart which shows the process executed by the sound processing apparatus which concerns on 9th Embodiment.

Hereinafter, examples of embodiments of the present invention will be described with reference to the drawings.

[First Embodiment] First, the first embodiment will be described. FIG. 1 shows a configuration of a system including a sound processing device according to a first embodiment of the present invention. As shown in FIG. 1, this system includes a sound processing device 1, a content reproduction device 2, a microphone 3, an electronic musical instrument 4, an electric musical instrument 5, a speaker 6 (an example of sound emitting means), and a display device 7. The content reproduction device 2 may, for example, reproduce the content (music, moving image, etc.) stored in the optical storage medium, or reproduce the content distributed via the network. May be good.

The sound processing device 1 is, for example, an AV receiver or the like. The sound processing device 1 includes a CPU 11, a memory 12, an input unit 13, an output unit 14, an audio signal processing unit 15, and a video signal processing unit 16.

The CPU 11 controls the input unit 13, the output unit 14, the audio signal processing unit 15, and the video signal processing unit 16 and executes information processing based on the program stored in the memory 12. Although omitted in FIG. 1, the sound processing device 1 is provided with a network interface for performing data communication via the network, and the program is downloaded via the network and stored in the memory 12. Alternatively, the sound processing device 1 is provided with a component for reading a program from an information storage medium such as a memory card, and the program is read from the information storage medium and stored in the memory 12.

The input unit 13 can receive the input of the audio signal and the video signal based on the content data from the content reproduction device 2, supplies the audio signal to the audio signal processing unit 15, and supplies the video signal to the video signal processing unit 16. Supply.

The input unit 13 can also accept the input of the user's performance sound. The "performance" indicates the act of producing a sound, and the "performance" includes not only the act of playing a musical instrument but also the act of singing a song. Therefore, the "performance sound" includes not only the performance sound of the musical instrument but also the singing sound. In the following, the performance sound of the musical instrument will be referred to as "musical instrument sound" for convenience.

For example, the input unit 13 is connected to the microphone 3 and can receive the input of the acoustic signal output from the microphone 3, and supplies the acoustic signal to the acoustic signal processing unit 15. The microphone 3 collects sound and outputs the collected sound as an acoustic signal. The microphone 3 is used to input the musical instrument sound of an acoustic musical instrument played by the user and the singing sound of the user to the sound processing device 1.

Further, for example, the input unit 13 can be connected to the electronic musical instrument 4 or the electric musical instrument 5 played by the user to receive the input of the acoustic signal output from the electronic musical instrument 4 or the electric musical instrument 5, and the acoustic signal can be received. Is supplied to the acoustic signal processing unit 15.

The input unit 13 may include a wireless network interface so that an acoustic signal is input to the input unit 13 via wireless communication. That is, the content sound and the performance sound may be input to the sound processing device 1 via wireless communication.

The acoustic signal processing unit 15 is, for example, a DSP (Digital Signal Processor), and executes processing related to the acoustic signal according to the control from the CPU 11. The acoustic signal output from the acoustic signal processing unit 15 is emitted from the speaker 6 via the output unit 14.

The video signal processing unit 16 is, for example, a DSP (Digital Signal Processor), and executes processing related to the video signal according to the control from the CPU 11. The video signal output from the video signal processing unit 16 is displayed on the display device 7 via the output unit 14.

In the sound processing device 1 according to the first embodiment, it is possible for a user who plays an acoustic musical instrument or sings a song at home or the like to enjoy the feeling of playing in a hall or the like. Hereinafter, a configuration for realizing such a function will be described. As shown in FIG. 1, the sound processing device 1 has a function of receiving the input of the musical instrument sound of the electronic musical instrument 4 or the electric musical instrument 5, and the content reproduced by the content reproduction device 2 is transmitted by the speaker 6 or the display device 7. Although it has a function to output, these functions are not essential in the first embodiment.

FIG. 2 shows an example of the user's playing environment. In the example shown in FIG. 2, the microphone 3 is installed in front of the user U. The microphone 3 is used to collect the performance sound of the user. For example, when the user is playing an acoustic musical instrument, the musical instrument sound is picked up by the microphone 3 and input to the input unit 13. Further, for example, when the user is singing a song, the singing sound is picked up by the microphone 3 and input to the input unit 13.

Further, in the example shown in FIG. 2, a plurality of speakers 6A, 6B, 6C, 6D, and 6E are installed. Specifically, the speaker 6A is installed in front of the user U. Further, the speakers 6B and 6C are installed on the left front and the right front when viewed from the user U, and the speakers 6D and 6E are installed on the left rear and the right rear when viewed from the user U, respectively. In the example shown in FIG. 2, five speakers 6A to 6E are installed, but four or less speakers 6 may be installed, or six or more speakers 6 may be installed. For example, only the speakers 6B and 6C may be installed.

FIG. 3 is a functional block diagram showing a function realized by the sound processing device 1 according to the first embodiment. As shown in FIG. 3, the sound processing device 1 according to the first embodiment includes a performance sound adjusting unit 101, a pre-processing unit 102 (an example of the first processing means), an indirect sound component generation unit 103, and a post-processing unit 104. (An example of the second processing means), and the output control unit 105. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, the above functional block is realized by the CPU 11 controlling the acoustic signal processing unit 15 according to a program.

FIG. 4 is a flow chart showing a process executed by the sound processing device 1 according to the first embodiment. Hereinafter, the functions of each functional block will be described with reference to FIG.

First, the performance sound adjustment unit 101 adjusts the performance sound by performing a predetermined process on the performance sound input from the microphone 3 (S10). For example, the performance sound adjustment unit 101 applies a howling reduction process to the performance sound to reduce howling in the microphone 3. Further, for example, the performance sound adjustment unit 101 performs effect processing (for example, processing for deleting an unnecessary frequency band or adjusting the sound pressure level before generating an indirect sound) on the performance sound. May be good. The performance sound processed by the performance sound adjustment unit 101 is supplied to the preprocessing unit 102.

The preprocessing unit 102 executes preprocessing on the supplied sound (here, the playing sound) (S11). For example, the preprocessing unit 102 performs voice adjustment processing or the like by an equalizer on the supplied sound. The performance sound processed by the preprocessing unit 102 is supplied to the indirect sound component generation unit 103. Although the performance sound adjusting unit 101 and the preprocessing unit 102 are shown as separate functional blocks in FIG. 3, they may be integrally configured.

The indirect sound component generation unit 103 generates a pseudo indirect sound component corresponding to the performance sound (S12). That is, the indirect sound component generation unit 103 assumes a case where a performance sound is emitted in an acoustic space such as a hall, and generates an indirect sound component (reverberation component or the like) generated in the acoustic space in that case. Various known methods can be adopted as a method for generating a pseudo indirect sound component. For example, the indirect sound component generation unit 103 determines the position where the indirect sound (reverberation sound) is generated in the assumed acoustic space, the delay time of the indirect sound with respect to the direct sound, the ratio of the level of the indirect sound to the sound pressure level of the direct sound, and the like. Based on the information, a pseudo indirect sound component corresponding to the playing sound is generated.

For example, the indirect sound component generation unit 103 includes an indirect sound component addition unit that adds an indirect sound component corresponding to the supplied sound to the supplied sound, and the indirect sound component generation unit 103 includes a performance sound. Is supplied to the indirect sound component addition part. Then, the indirect sound component generation unit 103 acquires only the indirect sound component by removing the original performance sound from the sound output from the indirect sound component addition unit (the performance sound to which the indirect sound component is added).

FIG. 5 is a diagram for explaining an example of a method of generating an indirect sound component. FIG. 5A shows an example of a performance sound. This playing sound corresponds to a direct sound component. For example, the performance sound (direct sound component) shown in FIG. 5A is stored in each of the first buffer and the second buffer. The indirect sound component addition unit adds an indirect sound component corresponding to the performance sound to the performance sound (direct sound component) stored in the first buffer. Here, various known methods can be adopted as a method for adding the indirect sound component. In this case, as shown in FIG. 5B, for example, the first buffer stores the direct sound component and the indirect sound component of the performance sound. After that, the indirect sound component generation unit 103 changes the direct sound component and the indirect sound component of the performance sound stored in the first buffer (FIG. 5B) to the direct sound component of the performance sound stored in the second buffer (FIG. 5B). By subtracting FIG. 5 (A), only the indirect sound component as shown in FIG. 5 (C) is acquired.

The method of generating the indirect sound component is not limited to the above example. For example, the performance sound (direct sound component) shown in FIG. 5A may be stored in the first buffer, and the indirect sound component corresponding to the performance sound (direct sound component) may be generated in the second buffer. ..

The indirect sound component generated by the indirect sound component generation unit 103 is supplied to the post-processing unit 104. The post-processing unit 104 executes post-processing on the supplied sound (here, an indirect sound component) (S13). For example, the post-processing unit 104 performs a process for adjusting the supplied sound according to the characteristics of the speaker 6. The indirect sound component processed by the post-processing unit 104 is supplied to the output control unit 105.

The output control unit 105 outputs the supplied indirect sound component to the output unit 14 (an example of the output means) (S14). That is, the output control unit 105 outputs the indirect sound component to the output unit 14 while limiting the output of the performance sound (musical instrument sound or singing sound of the acoustic instrument) input from the microphone 3 to the output unit 14. The indirect sound component output to the output unit 14 is emitted by the speaker 6.

Here, "restricting the output of the performance sound to the output unit 14" means, for example, not to output the performance sound to the output unit 14. That is, the output control unit 105 does not output the performance sound (direct sound component) input from the microphone 3 to the output unit 14, but outputs only the indirect sound component to the output unit 14. In other words, the output control unit 105 prevents the performance sound (direct sound component) input from the microphone 3 from being emitted from the speaker 6, and causes only the indirect sound component to be emitted from the speaker 6.

"Restricting the output of the performance sound to the output unit 14" means, for example, even if the performance sound is output to the output unit 14 so as to be emitted at a volume considerably lower than that of the indirect sound component. Good. That is, the output control unit 105 emits the performance sound (direct sound component) input from the microphone 3 at a volume considerably lower than the normal volume (a volume low enough to be hard to hear by the user). While outputting to the output unit 14, the indirect sound component may be output to the output unit 14 at a normal volume. In other words, the output control unit 105 causes the performance sound (direct sound component) input from the microphone 3 to be emitted from the speaker 6 at a volume considerably lower than the normal volume, and the indirect sound component is the normal volume. Is made to emit sound from the speaker 6.

When the speaker 6 is built in the sound processing device 1, the output control unit 105 outputs the supplied indirect sound component to the speaker 6 (another example of the output means).

According to the sound processing device 1 according to the first embodiment described above, a pseudo indirect sound component (reverberation component, etc.) corresponding to a user's playing sound (musical instrument sound or singing sound of an acoustic instrument) is transmitted from the speaker 6. Since the sound is emitted, the user can enjoy the feeling of playing an acoustic musical instrument or singing a song in a hall or a church. Further, according to the sound processing device 1 according to the first embodiment, the user's performance sound (musical instrument sound or singing sound of an acoustic instrument) is restricted from being emitted from the speaker 6, so that the original sounding position is used. Can be designed so as not to give the user a sense of discomfort due to hearing performance sounds emitted from different positions.

[Second Embodiment] Next, the second embodiment will be described. The hardware configuration of the sound processing device 1 according to the second embodiment is the same as that of the first embodiment. Further, the playing environment of the user is basically the same as that of the first embodiment. However, in the second embodiment, since the electronic musical instrument 4 or the electric musical instrument 5 connected to the input unit 13 of the sound processing device 1 is played by the user, the microphone 3 is unnecessary.

In the sound processing device 1 according to the second embodiment, a user who is playing the electronic musical instrument 4 or the electric musical instrument 5 at home or the like can enjoy the feeling of playing in the hall or the like. Hereinafter, a configuration for realizing such a function will be described. As shown in FIG. 1, the sound processing device 1 has a function of outputting the content reproduced by the content reproduction device 2 to the speaker 6 and the display device 7, but these functions are the second embodiment. Is not essential.

FIG. 6 is a functional block diagram showing a function realized by the sound processing device 1 according to the second embodiment. As shown in FIG. 6, the sound processing device 1 according to the second embodiment includes a performance sound adjustment unit 111, a pre-processing unit 112, an indirect sound component generation unit 113, a post-processing unit 114, and an output control unit 115. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, the above functional block is realized by the CPU 11 controlling the acoustic signal processing unit 15 according to a program.

FIG. 7 is a flow chart showing a process executed by the sound processing device 1 according to the second embodiment. Hereinafter, the functions of each functional block will be described with reference to FIG. 7.

First, the performance sound adjustment unit 111 adjusts the performance sound by performing a predetermined process on the performance sound input from the electronic musical instrument 4 or the electric musical instrument 5 (S20). For example, the performance sound adjustment unit 111 performs effect processing (for example, distortion processing on the guitar sound) on the performance sound. Note that the performance sound adjustment unit 111 does not execute a process that causes a large delay, and only a process with a small delay is executed. The performance sound processed by the performance sound adjustment unit 111 is supplied to the preprocessing unit 112.

The preprocessing unit 112 executes preprocessing on the supplied sound (here, the playing sound) (S21). Further, the indirect sound component generation unit 113 generates a pseudo indirect sound component corresponding to the performance sound (S22). Then, the post-processing unit 114 executes post-processing on the supplied sound (here, an indirect sound component) (S23). Steps S21 to S23 are basically the same as steps S11 to S13 of the first embodiment, and the pre-processing unit 112, the indirect sound component generation unit 113, and the post-processing unit 114 are the pre-processing unit 102 of the first embodiment. Since it is basically the same as the indirect sound component generation unit 103 and the post-processing unit 104, the description thereof will be omitted here.

The performance sound processed by the performance sound adjustment unit 111 is also supplied to the output control unit 115 via the path 119. The path 119 is a path leading to the output control unit 115 without going through the preprocessing unit 112, the indirect sound component generation unit 113, and the post processing unit 114. In other words, the path 119 is a path having less delay than the path leading to the output control unit 115 via the preprocessing unit 112, the indirect sound component generation unit 113, and the post processing unit 114. For example, in the preprocessing unit 112, the indirect sound component generation unit 113, and the post processing unit 114, processing is executed based on the performance sound stored in the buffer, but in the path 119, the performance sound is stored in the buffer. It is supplied to the output control unit 115 without any problem.

The output control unit 115 mixes the performance sound (direct sound component) supplied via the path 119 and the indirect sound component generated by the indirect sound component generation unit 113, and outputs the mixed sound to the output unit 14. (S24). The mixed sound output to the output unit 14 is emitted by the speaker 6.

FIG. 8 is a diagram for explaining the sound emitted from the speaker 6. Here, as shown in FIG. 8A, it is assumed that the performance sound B is input after the performance sound A is input. These performance sounds A and B correspond to direct sound components. In this case, as shown in FIG. 8B, the indirect sound component generation unit 113 generates the indirect sound component A corresponding to the performance sound A, and the indirect sound component A is supplied to the output control unit 115. The indirect sound component generation unit 113 also generates the indirect sound component B corresponding to the performance sound B after the indirect sound component A is generated, but this is omitted here.

Since the amount of processing in the preprocessing unit 112, the indirect sound component generation unit 113, and the post-processing unit 114 is large and processing in these functional blocks takes time, the indirect sound component A is the indirect sound component A in these functional blocks. Sound is emitted from the speaker 6 with a delay of a delay time corresponding to the time required for processing. On the other hand, the performance sounds A and B (direct sound components) are emitted from the speaker 6 via a path 119 with a small delay (a path in which a substantial delay does not occur). Therefore, as shown in FIG. 8C, the indirect sound component A corresponding to the performance sound A is delayed from the actual state and is mixed with the performance sound B after the performance sound A, and the mixed sound is mixed. Is emitted from the speaker 6.

According to the sound processing device 1 according to the second embodiment described above, a pseudo indirect sound component (reverberation component, etc.) is added to the user's performance sound (musical instrument sound of the electronic musical instrument 4 or the electric musical instrument 5). Since the sound is emitted from the speaker 6, the user can enjoy the feeling of playing an musical instrument in a hall or the like. Further, according to the sound processing device 1 according to the second embodiment, since the user's performance sound is emitted from the speaker 6 via the path 119 having a small delay, the performance sound is emitted from the speaker 6 after being played by the user. It is possible to suppress the delay until the sound is emitted from the sound. As a result, it is possible to prevent the user from feeling uncomfortable due to a large delay from the performance by the user until the performance sound is emitted from the speaker 6.

In the sound processing device 1 according to the second embodiment, the indirect sound component corresponding to the user's performance sound is generated later than the indirect sound component generated when the performance sound is emitted in the actual acoustic space. However, even if there is a slight delay in the indirect sound component, it is unlikely that the user will feel uncomfortable, so that no particular problem will occur.

[Third Embodiment] Next, the third embodiment will be described. The hardware configuration of the sound processing device 1 according to the third embodiment is the same as that of the first embodiment. The playing environment of the user is the same as that of the second embodiment.

In the sound processing device 1 according to the third embodiment, a user playing an electronic musical instrument 4 or an electric musical instrument 5 at home or the like can enjoy the feeling of playing in a hall or the like as a member of a music content player. Is possible. Hereinafter, a configuration for realizing such a function will be described.

FIG. 9 is a functional block diagram showing a function realized by the sound processing device 1 according to the third embodiment. As shown in FIG. 9, the sound processing device 1 according to the third embodiment includes a performance sound adjustment unit 121, a pre-processing unit 122, an indirect sound component generation unit 123, a post-processing unit 124, an output control unit 125, and a content decoding unit. Includes part 126. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, the above functional block is realized by the CPU 11 controlling the acoustic signal processing unit 15 according to a program.

FIG. 10 is a flow chart showing a process executed by the sound processing device 1 according to the third embodiment. Hereinafter, the functions of each functional block will be described with reference to FIG.

First, the content decoding unit 126 converts the multi-channel content sound input from the content playback device 2 into a PCM signal by format decoding (S30).

Further, the performance sound adjusting unit 121 adjusts the performance sound by performing a predetermined process on the performance sound input from the electronic musical instrument 4 or the electric musical instrument 5 (S31). Since step S31 is the same as step S20 of the second embodiment and the performance sound adjustment unit 121 is the same as the performance sound adjustment unit 111 of the second embodiment, the description thereof will be omitted here.

In FIG. 10, for convenience, steps S30 and S31 are shown to be executed in order, but steps S30 and S31 are executed in parallel.

The content sound converted into the PCM signal is mixed with the performance sound converted into the PCM signal by the AD conversion circuit (S32), and the mixed sound is supplied to the preprocessing unit 122. The performance sound is also supplied to the output control unit 125 via the path 129. Route 129 is similar to Route 119 of the second embodiment.

The preprocessing unit 122 executes preprocessing on the mixed sound (S33). For example, the preprocessing unit 122 performs voice adjustment processing or the like by an equalizer on the mixed sound. The mixed sound processed by the preprocessing unit 122 is supplied to the indirect sound component generation unit 123.

The indirect sound component generation unit 123 generates a pseudo indirect sound component corresponding to the mixed sound (S34). That is, the indirect sound component generation unit 123 generates an indirect sound component corresponding to the performance sound (direct sound component) and the content sound. The indirect sound component generation unit 123 assumes the case where the mixed sound is emitted in an acoustic space such as a hall, and generates an indirect sound component (reverberation component or the like) generated in the acoustic space in that case. Various known methods can be adopted as a method for generating a pseudo indirect sound component.

For example, the indirect sound component generation unit 123 performs a process of adding an indirect sound to the mixed sound stored in the first buffer, and then stores the sound stored in the first buffer in the second buffer. By subtracting the original mixed sound, the indirect sound component corresponding to the mixed sound is obtained. The indirect sound component generation unit 123 generates an indirect sound corresponding to the mixed sound in the second buffer by executing a process of generating the indirect sound based on the mixed sound stored in the first buffer. By doing so, the indirect sound component corresponding to the mixed sound may be acquired.

The indirect sound component generated by the indirect sound component generation unit 123 is supplied to the output control unit 125 together with the content sound via the post-processing unit 124.

The post-processing unit 124 executes post-processing (S35). Since step S35 is basically the same as step S13 of the first embodiment and the post-processing unit 124 is basically the same as the post-processing unit 104 of the first embodiment, description thereof will be omitted here.

The output control unit 125 mixes the performance sound (direct sound component) supplied via the path 129 with the content sound and the indirect sound component supplied from the post-processing unit 124, and transfers the mixed sound to the output unit 14. Output (S36). The mixed sound output to the output unit 14 is emitted by the speaker 6.

FIG. 11 is a diagram for explaining the sound emitted from the speaker 6. Here, as shown in FIG. 11A, it is assumed that the performance sound B and the content sound B are input after the performance sound A and the content sound A are input. The performance sounds A and B correspond to direct sound components. Further, in FIG. 11, for convenience, the performance sound A and the content sound A are shown with a slight time lag, but it is assumed that the input time points of the performance sound A and the content sound A are the same. The same applies to the performance sound B and the content sound B.

In the case of the example shown in FIG. 11A, the indirect sound component generation unit 123 generates the indirect sound component A corresponding to the mixed sound of the performance sound A and the content sound A as shown in FIG. 11B. , The indirect sound component A is supplied to the output control unit 125 together with the content sound A. In the indirect sound component generation unit 113, after the indirect sound component A is generated, the indirect sound component B corresponding to the mixed sound of the performance sound B and the content sound B is also generated, but is omitted here. doing.

Since the processing amount of the pre-processing unit 122, the indirect sound component generation unit 123, and the post-processing unit 124 is large and the processing by these functional blocks takes time, the indirect sound component A and the content sound A are described. Sound is emitted from the speaker 6 with a delay time corresponding to the time required for processing in the functional block. On the other hand, the performance sounds A and B (direct sound components) are emitted from the speaker 6 via the path 129 with less delay (the path where substantially no delay occurs). Therefore, as shown in FIG. 11C, the indirect sound component A is delayed from the actual state and is mixed with the performance sound B after the performance sound A, and the mixed sound is emitted from the speaker 6. Will be done.

According to the sound processing device 1 according to the third embodiment described above, a pseudo indirect sound component is provided for the user's performance sound (musical instrument sound of electronic musical instrument 4 or electric musical instrument 5) and multi-channel content sound. Since (reverberation component, etc.) is added and sound is emitted from the speaker 6, the user can enjoy the feeling of playing in a hall or the like as a member of a performer of music content. Further, according to the sound processing device 1 according to the third embodiment, since the user's performance sound is emitted from the speaker 6 via the path 129 having a small delay, the performance sound is emitted from the speaker 6 after being played by the user. It is possible to suppress the delay until the sound is emitted from the sound. As a result, it is possible to prevent the user from feeling uncomfortable due to a large delay from the performance by the user until the performance sound is emitted from the speaker 6.

[Fourth Embodiment] Next, the fourth embodiment will be described. The hardware configuration of the sound processing device 1 according to the fourth embodiment is the same as that of the first embodiment. The playing environment of the user is the same as that of the first embodiment.

In the sound processing device 1 according to the fourth embodiment, it is possible for the user to enjoy the feeling of singing a song or playing an acoustic musical instrument in a hall or the like as a member of a music content player. .. Hereinafter, a configuration for realizing such a function will be described.

FIG. 12 is a functional block diagram showing a function realized by the sound processing device 1 according to the fourth embodiment. As shown in FIG. 12, the sound processing device 1 according to the fourth embodiment includes a performance sound adjustment unit 131, a pre-processing unit 132, an indirect sound component generation unit 133, a post-processing unit 134, an output control unit 135, and a content decoding unit. Includes part 136. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, the above functional block is realized by the CPU 11 controlling the acoustic signal processing unit 15 according to a program.

FIG. 13 is a flow chart showing a process executed by the sound processing device 1 according to the fourth embodiment. Hereinafter, the functions of each functional block will be described with reference to FIG.

First, the content decoding unit 136 converts the multi-channel content sound input from the content playback device 2 into a PCM signal by format decoding (S40). Step S40 is basically the same as step S30 of the third embodiment, and the content decoding unit 136 is basically the same as the content decoding unit 126 of the third embodiment.

However, the content decoding unit 136 of the fourth embodiment includes the specific component removing unit 136A, and in step S40, the specific component removing unit 136A removes the specific component included in the content sound. Specifically, the specific component removing unit 136A removes the specific component corresponding to the performance sound input from the microphone 3 from the content sound. For example, when the user's singing sound is input from the microphone 3, the specific component removing unit 136A removes the vocal component from the content sound. Since the vocal component is often included in the center channel of the multi-channel content sound, the specific component removing unit 136A removes the vocal component from the content sound by removing the center channel. The method of removing the vocal component from the content sound is not limited to this method, and various known methods can be adopted. Further, for example, when the musical instrument sound of an acoustic instrument is input from the microphone 3, the specific component removing unit 136A may remove the musical instrument sound component of the acoustic instrument from the content sound. The type of performance sound input from the microphone 3 (for example, singing sound, guitar sound, piano sound, etc.) may be automatically determined by analyzing the performance sound. , The user may specify via an input device.

The performance sound adjustment unit 131 adjusts the performance sound by performing a predetermined process on the performance sound input from the microphone 3 (S41). Since step S41 is the same as step S10 of the first embodiment and the performance sound adjustment unit 131 is the same as the performance sound adjustment unit 101 of the first embodiment, the description thereof will be omitted here.

Note that, in FIG. 13, for convenience, steps S40 and S41 are shown to be executed in order, but steps S40 and S41 are executed in parallel.

The content sound converted into the PCM signal is mixed with the performance sound converted into the PCM signal by the AD conversion circuit (S42), and the mixed sound is supplied to the preprocessing unit 132. Then, based on the mixed sound, processing by the pre-processing unit 132, the indirect sound component generation unit 133, and the post-processing unit 134 is executed (S43, S44, S45). Steps S43 to S45 are basically the same as steps S33 to S35 of the third embodiment, and the pre-processing unit 132, the indirect sound component generation unit 133, and the post-processing unit 134 are the pre-processing unit 122 of the third embodiment. Since it is basically the same as the indirect sound component generation unit 123 and the post-processing unit 124, the description thereof will be omitted here.

The performance sound is supplied to the output control unit 135 via the path 139. Route 139 is similar to Route 119 of the second embodiment.

Similar to the output control unit 125 of the third embodiment, the output control unit 135 includes a performance sound (direct sound component) supplied via the path 139, and a content sound and an indirect sound component supplied from the post-processing unit 134. And are mixed, and the mixed sound is output to the output unit 14 (S46). The mixed sound output to the output unit 14 is emitted by the speaker 6.

According to the sound processing device 1 according to the fourth embodiment described above, a pseudo indirect sound component (reverberation) with respect to the user's performance sound (singing sound or musical instrument sound of an acoustic instrument) and the multi-channel content sound. Since the sound is emitted from the speaker 6 with the addition of components), the user can enjoy the feeling of singing a song or playing an acoustic instrument in a hall or the like as a member of a music content player. .. Further, according to the sound processing device 1 according to the fourth embodiment, since the user's performance sound is emitted from the speaker 6 via the path 139 having a small delay, the performance sound is emitted from the speaker 6 after being played by the user. It is possible to suppress the delay until the sound is emitted from the sound. As a result, it is possible to prevent the user from feeling uncomfortable due to a large delay from the performance by the user until the performance sound is emitted from the speaker 6.

Further, according to the sound processing device 1 according to the fourth embodiment, for example, when the user is singing a song, the vocal component contained in the content sound is removed, so that the user becomes a vocal of the music content. You can enjoy the feeling of singing a song in a hall or the like.

In the above, it has been described that the vocal component of the content sound is removed before mixing the performance sound and the content sound. However, in the removal of the vocal component of the content sound, the performance sound and the content sound are mixed. It may be done later.

[Fifth Embodiment] Next, the fifth embodiment will be described. The hardware configuration of the sound processing device 1 according to the fifth embodiment is the same as that of the first embodiment. The playing environment of the user is the same as that of the first embodiment or the second embodiment.

Also in the sound processing device 1 according to the fifth embodiment, it is possible for the user to enjoy the feeling of playing in a hall or the like as a member of the performer of the music content. In particular, in the sound processing device 1 according to the fifth embodiment, it is possible to feel a sense of unity between the user's performance sound and the content sound. Hereinafter, a configuration for realizing such a function will be described.

FIG. 14 is a functional block diagram showing a function realized by the sound processing device 1 according to the fifth embodiment. As shown in FIG. 14, the sound processing device 1 according to the fifth embodiment includes a performance sound adjustment unit 141, a pre-processing unit 142, an indirect sound component generation unit 143, a post-processing unit 144, an output control unit 145, and a content decoding unit. 146 and content sound adjustment unit 147 are included. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, the above functional block is realized by the CPU 11 controlling the acoustic signal processing unit 15 according to a program.

FIG. 15 is a flow chart showing a process executed by the sound processing device 1 according to the fifth embodiment. Hereinafter, the functions of each functional block will be described with reference to FIG.

First, the content decoding unit 146 converts the multi-channel content sound input from the content playback device 2 into a PCM signal by format decoding (S50). Since step S50 is basically the same as step S30 of the third embodiment and the content decoding unit 146 is basically the same as the content decoding unit 126 of the third embodiment, the description thereof will be omitted here.

The content sound adjustment unit 147 adjusts the content sound in order to match the characteristics of the performance sound and the content sound (S51). The content sound adjusting unit 147 includes an indirect sound component removing unit 147A. The indirect sound component removing unit 147A removes the indirect sound component included in the content sound in order to match the amount of the indirect sound component between the performance sound and the content sound. Various known methods can be adopted as a method for removing the indirect sound component contained in the content sound.

When the performance sound input from the electronic musical instrument 4 or the electric musical instrument 5 contains only the direct sound component and does not contain the indirect sound component, whereas the content sound contains the direct sound component and the indirect sound component. There is. Therefore, for example, the indirect sound component removing unit 147A removes the indirect sound component included in the content sound and outputs only the direct sound component of the content sound. For example, the indirect sound component removing unit 147A removes the indirect sound component by specifying the indirect sound component included in the content sound and lowering the sound pressure level of the specified indirect sound component. That is, the indirect sound component removing unit 147A removes the indirect sound component almost completely by lowering the sound pressure level of the indirect sound component to zero (nearly zero).

The indirect sound component removing unit 147A may remove (reduce) the indirect sound component to some extent by lowering the sound pressure level of the indirect sound component to some extent. That is, "removing the indirect sound component" includes not only removing the indirect sound component almost completely but also removing (reducing) the indirect sound component to some extent. Here, "to some extent" is a degree that does not make the user feel uncomfortable even if the indirect sound component remains.

The performance sound adjustment unit 131 adjusts the performance sound (S52). Since step S52 is the same as steps S10 and 20 of the first embodiment or the second embodiment, and the performance sound adjustment unit 141 is the same as the performance sound adjustment units 110 and 111 of the first embodiment or the second embodiment. , The description is omitted here.

In FIG. 15, for convenience, steps S50 and S51 and step S52 are shown to be executed in order, but steps S50 and S51 and step S52 are executed in parallel.

The content sound (direct sound component) from which the indirect sound component has been removed is mixed with the performance sound (S53), and the mixed sound is supplied to the indirect sound component generation unit 143 via the preprocessing unit 142. The preprocessing unit 142 executes preprocessing on the mixed sound (S54). Since step S54 is the same as step S11 of the first embodiment and the preprocessing unit 142 is the same as the preprocessing unit 112 of the first embodiment, the description thereof will be omitted here.

The indirect sound component generation unit 143 generates a pseudo indirect sound component corresponding to the mixed sound (direct sound component of the content sound and the direct sound component of the performance sound) (S55). Since step S55 is basically the same as step S34 of the third embodiment and the indirect sound component generation unit 143 is basically the same as the indirect sound component generation unit 123 of the third embodiment, the description thereof is omitted here. To do.

The indirect sound component generated by the indirect sound component generation unit 143 is supplied to the output control unit 145 via the post-processing unit 144. The post-processing unit 144 executes post-processing (S56). Since step S56 is the same as step S13 of the first embodiment and the post-processing unit 144 is the same as the post-processing unit 114 of the first embodiment, the description thereof will be omitted here. As shown in FIG. 14, the content sound from which the indirect sound component has been removed (that is, the direct sound component of the content sound) is also supplied to the output control unit 145. Further, the performance sound (direct sound component) is supplied to the output control unit 145 via the path 149. Route 149 is similar to Route 129 of the second embodiment.

The output control unit 145 mixes the performance sound (direct sound component) supplied via the path 149, the content sound (direct sound component), and the indirect sound component generated by the indirect sound component generation unit 143. The mixed sound is output to the output unit 14 (S57). The mixed sound output to the output unit 14 is emitted by the speaker 6.

According to the sound processing device 1 according to the fifth embodiment described above, the user can enjoy the feeling of playing in a hall or the like as a member of the performer of the music content. Further, according to the sound processing device 1 according to the fifth embodiment, it is possible to match the amounts of the indirect sound components of the user's performance sound and the content sound, and as a result, the user's performance sound and the content sound are combined. The user can fully feel the sense of unity.

As shown in FIG. 14, also in the fifth embodiment, the musical instrument sound or the singing sound of the acoustic musical instrument may be input from the microphone 3. However, in this case, since the performance sound may include an indirect sound component, the performance sound adjustment unit 141 may remove the indirect sound component included in the performance sound.

[Sixth Embodiment] Next, the sixth embodiment will be described. The sixth embodiment is a modification of the fifth embodiment. The sound processing device 1 according to the sixth embodiment removes the indirect sound component of the content sound only when the performance sound is input.

FIG. 16 is a functional block diagram showing a function realized by the sound processing device 1 according to the sixth embodiment, and FIG. 17 is a flow diagram showing a process executed by the sound processing device 1 according to the sixth embodiment. Is. As shown in FIG. 16, the sound processing device 1 according to the sixth embodiment includes a performance sound adjustment unit 141, a pre-processing unit 142, an indirect sound component generation unit 143, a post-processing unit 144, an output control unit 145, and a content decoding unit. 146, content sound adjustment unit 147, and input detection unit 148 are included. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, the above functional block is realized by the CPU 11 controlling the acoustic signal processing unit 15 according to a program.

The sound processing device 1 according to the sixth embodiment is different from the fifth embodiment in that it includes an input detection unit 148 and includes steps S51A and 51B instead of step S51. Hereinafter, the differences from the fifth embodiment will be mainly described.

In the sound processing device 1 according to the sixth embodiment, the input detection unit 148 detects that the performance sound is input from the electronic musical instrument 4, the electric musical instrument 5, or the microphone 3. The indirect sound component removing unit 147A removes the indirect sound component contained in the content sound according to the detection result of the input detection unit 148. Specifically, when it is detected that the performance sound is input (S51A: Yes), the indirect sound component removing unit 147A removes the indirect sound component from the content sound (S51B). On the other hand, when it is not detected that the performance sound is input (S51A: No), the indirect sound component removing unit 147A does not remove the indirect sound component from the content sound. Although omitted in FIG. 17, in this case, steps S52 and S53 are not executed, only the content sound is supplied to the preprocessing unit 142, and the content sound is output to the output unit 14 in step S57.

In the sound processing device 1 according to the sixth embodiment described above, the indirect sound component is removed from the content sound only when the performance sound is input. When the performance sound is not input, it is not necessary to match the amount of the indirect sound component between the performance sound and the content sound, and it is not necessary to remove the indirect sound component from the content sound. In this regard, according to the sound processing device 1 according to the sixth embodiment, when it is not necessary to remove the indirect sound component from the content sound, the process of removing the indirect sound component from the content sound is not executed, so that the sound It becomes possible to reduce the processing load of the processing device 1.

[7th Embodiment] Next, the 7th embodiment will be described. The hardware configuration of the sound processing device 1 according to the seventh embodiment is the same as that of the first embodiment. The playing environment of the user is the same as that of the first embodiment or the second embodiment.

In the fifth and sixth embodiments, the indirect sound component included in the content sound is removed in order to match the amount of the indirect sound component of the content sound with the performance sound, whereas the sound processing according to the seventh embodiment is performed. In the device 1, the indirect sound component is added to the performance sound according to the amount of the indirect sound component of the content sound.

FIG. 18 is a functional block diagram showing a function realized by the sound processing device 1 according to the seventh embodiment. As shown in FIG. 18, the sound processing device 1 according to the seventh embodiment has a performance sound adjustment unit 151, a pre-processing unit 152, an indirect sound component generation unit 153, a post-processing unit 154, an output control unit 155, and a content decoding unit. 156 and indirect sound component amount analysis unit 157 are included. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, the above functional block is realized by the CPU 11 controlling the acoustic signal processing unit 15 according to a program.

FIG. 19 is a flow chart showing a process executed by the sound processing device 1 according to the seventh embodiment. Hereinafter, the functions of the functional blocks will be described with reference to FIG.

First, the content decoding unit 156 converts the multi-channel content sound input from the content playback device 2 into a PCM signal by format decoding (S60). Since step S60 is basically the same as step S30 of the third embodiment and the content decoding unit 156 is basically the same as the content decoding unit 126 of the third embodiment, the description thereof will be omitted here.

The indirect sound component amount analysis unit 157 analyzes the amount of the indirect sound component contained in the content sound (S61). For example, the indirect sound component amount analysis unit 157 analyzes the number and magnitude (sound pressure level) of indirect sound components included in the content sound. Various known methods can be adopted as a method for analyzing the amount of indirect sound components contained in the content sound.

The performance sound adjustment unit 151 adjusts the performance sound (S62). Step S62 is basically the same as steps S10 and S20 of the first embodiment or the second embodiment, and the performance sound adjusting unit 151 is basically the same as the performance sound adjusting units 101 and 111 of the first embodiment or the second embodiment. Is similar.

However, the performance sound adjustment unit 151 of the seventh embodiment also plays a role of adjusting the performance sound in order to match the characteristics of the performance sound and the content sound. That is, the performance sound adjustment unit 151 includes the indirect sound component addition unit 151A, and in step S62, the indirect sound component addition unit 151A adds the indirect sound component corresponding to the performance sound to the performance sound. In particular, the indirect sound component addition unit 151A sets the amount of the indirect sound component added to the performance sound based on the analysis result of the indirect sound component amount analysis unit 157. That is, the indirect sound component addition unit 151A sets the number and size of the indirect sound components added to the performance sound according to the number and size of the indirect sound components included in the content sound. That is, the indirect sound component addition unit 151A sets the number and size of the indirect sound components added to the performance sound to the same degree as the number and size of the indirect sound components included in the content sound.

In the sound processing device 1 according to the seventh embodiment, the content sound and the performance sound to which the indirect sound component is added by the indirect sound component addition unit 151A are mixed (S63), and the mixed sound is supplied to the preprocessing unit 152. Will be done. Then, based on the mixed sound, processing by the pre-processing unit 152, the indirect sound component generation unit 153, and the post-processing unit 154 is executed (S64, S65, S66). Steps S64 to S66 are basically the same as steps S33 to S35 of the third embodiment, and the pre-processing unit 152, the indirect sound component generation unit 153, and the post-processing unit 154 are the pre-processing unit 122 of the third embodiment. Since it is basically the same as the indirect sound component generation unit 123 and the post-processing unit 124, the description thereof will be omitted here.

The performance sound to which the indirect sound component is added by the indirect sound component addition unit 151A is supplied to the output control unit 155 via the path 159. Route 159 is similar to Route 119 of the second embodiment.

The output control unit 155 mixes the content sound, the performance sound to which the indirect sound component is added by the indirect sound component addition unit 151A, and the indirect sound component generated by the indirect sound component generation unit 153, and mixes the mixed sound. Output to the output unit 14 (S67). The mixed sound output to the output unit 14 is emitted by the speaker 6.

According to the sound processing device 1 according to the seventh embodiment described above, the user can enjoy the feeling of playing in a hall or the like as a member of the performer of the music content. Further, according to the sound processing device 1 according to the seventh embodiment, it is possible to match the amounts of the indirect sound components of the user's performance sound and the content sound, and as a result, the user's performance sound and the content sound are combined. The user can fully feel the sense of unity.

As shown in FIG. 18, in the seventh embodiment as well, the musical instrument sound or the singing sound of the acoustic musical instrument may be input from the microphone 3. However, in this case, since the performance sound included from the microphone 3 may contain an indirect sound component in advance, the performance sound adjustment unit 151 once removes the indirect sound component included in the performance sound, and then the indirect sound component. The indirect sound component may be added to the performance sound by the addition unit 151A.

[Eighth Embodiment] Next, the eighth embodiment will be described. The eighth embodiment is a modification of the seventh embodiment. In the sound processing device 1 according to the eighth embodiment, the method of adding the indirect sound to the performance sound is changed according to the type of the performance sound.

FIG. 20 is a functional block diagram showing a function realized by the sound processing device 1 according to the eighth embodiment, and FIG. 21 is a flow diagram showing a process executed by the sound processing device 1 according to the eighth embodiment. Is. As shown in FIG. 20, the sound processing device 1 according to the eighth embodiment has a performance sound adjustment unit 151, a preprocessing unit 152, an indirect sound component generation unit 153, a post processing unit 154, an output control unit 155, and a content decoding unit. 156, an indirect sound component amount analysis unit 157, and a performance sound type identification unit 158 are included. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, the above functional block is realized by the CPU 11 controlling the acoustic signal processing unit 15 according to a program.

The sound processing device 1 according to the eighth embodiment is different from the seventh embodiment in that the performance sound type specifying unit 158 is included and steps S62A and S62B are included instead of step S62. Hereinafter, the differences from the seventh embodiment will be mainly described.

In the sound processing device 1 according to the eighth embodiment, the performance sound type specifying unit 158 specifies the type of the input performance sound (step S62A). For example, the performance sound type specifying unit 158 determines whether or not the input performance sound is a musical instrument sound. When the input performance sound is a musical instrument sound, the performance sound type specifying unit 158 specifies the type of the musical instrument sound. That is, the performance sound type specifying unit 158 specifies which of the plurality of types of musical instruments (for example, a guitar, a violin, a piano, etc.) the input performance sound is. Further, for example, the performance sound type specifying unit 158 determines whether or not the input performance sound is a singing sound. As a method for specifying the type of performance sound, various known methods can be adopted.

Further, the indirect sound component addition unit 151A of the eighth embodiment is based on not only the analysis result of the indirect sound component amount analysis unit 157 but also the specific result of the performance sound type identification unit 158, and the indirect sound corresponding to the performance sound. A component is added to the performance sound (S62B). That is, the indirect sound component addition unit 151A sets the indirect sound component to be added to the performance sound based not only on the analysis result of the indirect sound component amount analysis unit 157 but also on the specific result of the performance sound type identification unit 158. To do.

Since the radiation characteristics of the performance sound in the actual acoustic space differ depending on the type of the performance sound, the indirect sound component addition unit 151A adds the indirect sound to the performance sound based on the radiation characteristics different for each type of the performance sound. Set the ingredients.

For example, since the instrument sound of a guitar tends to be radiated in the front direction as compared with other directions, when the performance sound is the instrument sound of a guitar, the indirect sound component addition unit 151A is set to the channel corresponding to the front direction. On the other hand, an indirect sound component (reverberation component, etc.) is added. Alternatively, the indirect sound component addition unit 151A makes the amount of the indirect sound component added to the channel corresponding to the front direction larger than the amount of the indirect sound component added to the other channels.

Further, for example, since the instrument sound of the violin tends to be radiated upward as compared with other directions, when the performance sound is the instrument sound of the violin, the indirect sound component addition unit 151A is a channel corresponding to the upward direction. An indirect sound component (reverberation component, etc.) is added to the sound. Alternatively, the indirect sound component addition unit 151A makes the amount of the indirect sound component added to the channel corresponding to the upward direction larger than the amount of the indirect sound component added to the other channels.

Further, for example, since the singing sound tends to be radiated in the front direction as compared with other directions, when the playing sound is a singing sound, the indirect sound component addition unit 151A is indirect with respect to the channel corresponding to the front direction. Add sound components (reverberation components, etc.). Alternatively, the indirect sound component addition unit 151A makes the amount of the indirect sound component added to the channel corresponding to the front direction larger than the amount of the indirect sound component added to the other channels.

According to the sound processing device 1 according to the eighth embodiment described above, it is possible to add an indirect sound component to the playing sound based on the radiation characteristics of the playing sound of the user, which is more natural indirect. Sound components can be added to the playing sound.

[Ninth Embodiment] Next, the ninth embodiment will be described. The hardware configuration of the sound processing device 1 according to the ninth embodiment is the same as that of the first embodiment. The playing environment of the user is the same as that of the first embodiment or the second embodiment.

In the sixth to eighth embodiments, the amount of the indirect sound component is matched between the performance sound and the content sound, whereas in the sound processing device 1 according to the ninth embodiment, the performance sound and the content sound are used to produce a timbre. It is designed to match.

FIG. 22 is a functional block diagram showing a function realized by the sound processing device 1 according to the ninth embodiment. As shown in FIG. 22, the sound processing device 1 according to the ninth embodiment includes a performance sound adjustment unit 161, a preprocessing unit 162, an indirect sound component generation unit 163, a post processing unit 164, an output control unit 165, and a content decoding unit. 166, the first tone color analysis unit 167, and the second tone color analysis unit 168 are included. These functional blocks are realized by the CPU 11 and the acoustic signal processing unit 15. For example, the above functional block is realized by the CPU 11 controlling the acoustic signal processing unit 15 according to a program.

FIG. 23 is a flow chart showing a process executed by the sound processing device 1 according to the ninth embodiment. Hereinafter, the functions of each functional block will be described with reference to FIG. 23.

First, the content decoding unit 166 converts the multi-channel content sound input from the content playback device 2 into a PCM signal by format decoding (S70). Since step S70 is basically the same as step S30 of the third embodiment and the content decoding unit 166 is basically the same as the content decoding unit 126 of the third embodiment, the description thereof will be omitted here.

The second timbre analysis unit 168 analyzes the timbre of the content sound (S71). For example, when the content sound includes a plurality of types of musical instrument sounds, the second timbre analysis unit 168 identifies the type of musical instrument sound included in the performance sound among the plurality of types of musical instrument sounds, and the musical instrument sound. Analyze the tone of. Further, for example, when the performance sound includes a singing sound, the second timbre analysis unit 168 analyzes the timbre of the singing sound included in the content sound. Various known methods can be adopted as a method for identifying the musical instrument sound or the singing sound included in the content sound and a method for analyzing the timbre of the musical instrument sound or the singing sound.

The first timbre analysis unit 167 analyzes the timbre of the performance sound (S72). The first timbre analysis unit 167 identifies a musical instrument sound or a singing sound included in the performance sound, and analyzes the timbre of the musical instrument sound or the singing sound. Various known methods can be adopted as a method for analyzing the timbre of the performance sound.

In FIG. 23, for convenience, steps S70 and 71 and step S72 are shown to be executed in order, but steps S70 and S71 and step S72 are executed in parallel.

The performance sound adjustment unit 161 adjusts the performance sound (S73). Step S62 is basically the same as steps S10 and S20 of the first embodiment or the second embodiment, and the performance sound adjusting unit 161 is basically the same as the performance sound adjusting units 101 and 111 of the first embodiment or the second embodiment. Is similar.

However, the performance sound adjustment unit 161 of the ninth embodiment also plays a role of adjusting the performance sound in order to match the characteristics of the performance sound and the content sound. That is, the performance sound adjustment unit 161 includes the tone color adjustment unit 161A, and in step S73, the tone color adjustment unit 161A is based on the comparison between the analysis result of the first tone color analysis unit 167 and the analysis result of the second tone color analysis unit 168. To adjust the tone of the performance sound.

For example, the first timbre analysis unit 167 obtained an analysis result that the violin sound (violin instrument sound) contained in the performance sound had many high-frequency components, and the high-frequency component of the violin sound contained in the content sound was obtained. When the second tone color analysis unit 168 obtains the analysis result that the amount is small, the tone color adjustment unit 161A reduces the high frequency component of the violin sound contained in the performance sound. In short, when the amount of the specific band component of the instrument sound contained in the performance sound and the amount of the specific band component of the same type of instrument sound contained in the content sound are different, the specific band component and the content of the instrument sound included in the performance sound. The tone color adjusting unit 161A adjusts a specific band component of the instrument sound included in the performance sound in order to set it to the same level as the high frequency component of the same type of instrument sound included in the sound.

In the sound processing device 1 according to the ninth embodiment, the content sound and the performance sound whose timbre is adjusted by the timbre adjusting unit 161A are mixed (S74), and the mixed sound is supplied to the preprocessing unit 162. Then, based on the mixed sound, processing by the pre-processing unit 162, the indirect sound component generation unit 163, and the post-processing unit 164 is executed (S75, S76, S77). Steps S75 to S77 are basically the same as steps S33 to S35 of the third embodiment, and the pre-processing unit 162, the indirect sound component generation unit 163, and the post-processing unit 164 are the pre-processing units 122 of the third embodiment. Since it is basically the same as the indirect sound component generation unit 123 and the post-processing unit 124, the description thereof will be omitted here.

The performance sound whose timbre is adjusted by the timbre adjusting unit 161A is supplied to the output control unit 165 via the path 169. Route 169 is similar to Route 119 of the second embodiment.

Similar to the output control unit 125 of the third embodiment, the output control unit 165 is the performance sound supplied via the path 169 (the performance sound whose timbre is adjusted by the timbre adjustment unit 161A) and the post-processing unit 164. The supplied content sound and indirect sound component are mixed, and the mixed sound is output to the output unit 14 (S78). The mixed sound output to the output unit 14 is emitted by the speaker 6.

According to the sound processing device 1 according to the ninth embodiment described above, the user can enjoy the feeling of playing in a hall or the like as a member of the performer of the music content. Further, according to the sound processing device 1 according to the ninth embodiment, it is possible to match the tones of the user's performance sound and the content sound, and as a result, the user feels a sense of unity between the user's performance sound and the content sound. Can be fully felt.

Instead of the tone color adjusting unit 161A that adjusts the tone color of the performance sound, the tone color that adjusts the tone color of the content sound based on the comparison between the analysis result of the first tone color analysis unit 167 and the analysis result of the second tone color analysis unit 168. An adjustment unit may be provided.

In this tone adjustment unit, for example, the first tone color analysis unit 167 obtains an analysis result that the high frequency component of the violin sound contained in the performance sound is large, and the high frequency component of the violin sound contained in the content sound is obtained. When the analysis result that the amount is small is obtained by the second tone color analysis unit 168, the high frequency component of the violin sound contained in the content sound may be increased.

Further, in this tone adjustment unit, for example, the first tone color analysis unit 167 obtains an analysis result that the performance sound has a large amount of high frequency components of the violin sound (musical instrument sound of the violin), and the content sound is the violin sound. When the second tone color analysis unit 168 obtains the analysis result that there are many high-frequency components of the guitar sound, which is a different musical instrument sound, the high-frequency components of the guitar sound contained in the content sound are reduced. May be good.

It should be noted that both the tone color adjusting unit 161A for adjusting the tone color of the performance sound and the tone color adjusting unit for adjusting the tone color of the content sound may be provided. For example, the first tone color analysis unit 167 obtains an analysis result that there are many high-frequency components of the violin sound contained in the performance sound, and the second tone color analysis unit 168 obtains the height of the violin sound contained in the content sound. When the analysis result that the range component is small is obtained, the high range component of the violin sound contained in the performance sound and the high range component of the violin sound contained in the content sound are set to the same level in the performance sound. The high-frequency component of the violin sound included and the high-frequency component of the violin sound included in the performance sound may be adjusted respectively.

[Modified Example] The present invention is not limited to the first to ninth embodiments described above.

For example, a plurality of the first to ninth embodiments may be combined.

Further, for example, the above description is based on the premise that the sound processing device 1 is an AV receiver, and the functions described above can be realized by using the AV receiver, but the sound processing device 1 is an AV. A device other than the receiver may be used, and the functions described above may be realized by a device other than the AV receiver. For example, the sound processing device 1 may be built in the speaker. Further, for example, the sound processing device 1 may be realized by a desktop computer, a laptop computer, a tablet computer, a smartphone, or the like.

[Additional Notes] As can be understood from the description of the embodiments described above, various technical ideas including the inventions described below are disclosed in the present specification.

In the sound processing device according to the present invention, in order to match the characteristics of the input means that receives the input of the user's performance sound, the performance sound, and the content sound that is the sound obtained based on the content data, the performance sound and the performance sound An adjustment means for adjusting at least one of the content sounds, a generation means for generating an indirect sound component corresponding to a sound obtained by mixing the performance sound and the content sound, the performance sound, and the content sound. It includes at least an output control means for outputting a sound formed by mixing the direct sound component and the indirect sound component to the output means.

Further, the sound processing method according to the present invention is an adjustment step of adjusting at least one of the performance sound and the content sound in order to match the characteristics of the user's performance sound and the content sound obtained based on the content data. A generation step of generating an indirect sound component corresponding to a sound obtained by mixing the performance sound and the content sound, the performance sound, at least a direct sound component of the content sound, and the indirect sound component. It includes an output control step for outputting the mixed sound to the output means.

Further, the program according to the present invention is an adjusting means for adjusting at least one of the performance sound and the content sound in order to match the characteristics of the user's performance sound and the content sound obtained based on the content data. A generation means for generating an indirect sound component corresponding to a sound obtained by mixing a performance sound and the content sound, and the performance sound, at least a direct sound component of the content sound, and the indirect sound component are mixed. This is a program for operating a computer as an output control means for outputting a sound to an output means. The information storage medium according to the present invention is a computer-readable information storage medium on which the above program is recorded.

In the above invention, the adjusting means removes the indirect sound component contained in the content sound, and the generating means mixes the playing sound and the content sound from which the indirect sound component has been removed by the adjusting means. The indirect sound component corresponding to the resulting sound may be generated.

In the above invention, the adjusting means may remove the indirect sound component included in the content sound in response to the input of the playing sound.

The present invention includes an analysis means for analyzing the amount of indirect sound components contained in the content sound, and the adjustment means adds an indirect sound component corresponding to the performance sound to the performance sound. The amount of the indirect sound component added to the performance sound is set based on the analysis result of the analysis means, and the generation means includes the performance sound to which the indirect sound component is added by the adjustment means and the content sound. The indirect sound component corresponding to the sound formed by mixing and may be generated.

The present invention includes means for specifying the type of musical instrument sound included in the playing sound, and the adjusting means is an indirect sound added to the playing sound based on the type of musical instrument sound included in the playing sound. Ingredients may be set.

In the above invention, the means for determining whether or not the performance sound includes the singing sound is included, and the adjusting means is said based on the determination result of whether or not the performance sound includes the singing sound. The indirect sound component to be added to the performance sound may be set.

The present invention includes a first timbre analysis means for analyzing the timbre of the instrument sound included in the performance sound, and a second timbre analysis means for analyzing the timbre of the instrument sound included in the content sound. The adjusting means adjusts at least one of the performance sound and the content sound based on the comparison between the analysis result of the first tone color analysis means and the analysis result of the second tone color analysis means. It may be.

In the above invention, the means for specifying the type of the instrument sound included in the performance sound is included, and the second timbre analysis means is the plurality of types of instruments when the content sound includes a plurality of types of instrument sounds. Of the sounds, the timbre of the type of instrument sound included in the performance sound may be analyzed.

The present invention includes a first timbre analysis means for analyzing the timbre of the singing sound included in the performance sound, and a second timbre analysis means for analyzing the timbre of the singing sound included in the content sound. The adjusting means adjusts at least one of the performance sound and the content sound based on the comparison between the analysis result of the first tone color analysis means and the analysis result of the second tone color analysis means. It may be.

1 sound processing device, 2 content playback device, 3 microphone, 4 electronic device, 5 electrical device, 6,6A, 6B, 6C, 6D, 6E speaker, 11 CPU, 12 memory, 13 input section, 14 output section, 15 sound Signal processing unit, 16 Video signal processing unit, 101,111,121,131,141,151,161 Performance sound adjustment unit, 102,112,122,132,142,152,162 Preprocessing unit, 103,113,123 , 133, 143, 153, 163 Indirect sound component generation unit, 104, 114, 124, 134, 144, 154, 164 Post processing unit, 105, 115, 125, 135, 145, 155, 165 Output control unit, 119, 129,139,149,159,169 paths, 126,136,146,156,166 Content decoding unit, 136A Specific component removal unit, 147 Content sound adjustment unit, 147A Indirect sound component removal unit, 148 Input detection unit, 151A Indirect Sound component addition unit, 157 indirect sound component analysis unit, 158 performance sound type identification unit, 161A sound color adjustment unit, 167 first sound color analysis unit, 168 second sound color analysis unit, U user.

Claims (12)

An input means that accepts the input of the user's performance sound,
An adjusting means for adjusting at least one of the performance sound and the content sound in order to match the characteristics of the performance sound and the content sound which is a sound obtained based on the content data.
A generation means for generating an indirect sound component corresponding to a sound obtained by mixing the performance sound and the content sound, and
An output control means that outputs a sound obtained by mixing the performance sound, at least a direct sound component of the content sound, and the indirect sound component to the output means.
Only including,
The adjusting means removes the indirect sound component contained in the content sound, and removes the indirect sound component.
The generation means generates an indirect sound component corresponding to a sound formed by mixing the performance sound and the content sound from which the indirect sound component has been removed by the adjustment means.
A sound processing device characterized by this. An input means that accepts the input of the user's performance sound,
An adjusting means for adjusting at least one of the performance sound and the content sound in order to match the characteristics of the performance sound and the content sound which is a sound obtained based on the content data.
A generation means for generating an indirect sound component corresponding to a sound obtained by mixing the performance sound and the content sound, and
An output control means that outputs a sound obtained by mixing the performance sound, at least a direct sound component of the content sound, and the indirect sound component to the output means.
The A including sound processing equipment,
Further, it includes an analysis means for analyzing the amount of indirect sound components contained in the content sound.
The adjusting means adds an indirect sound component corresponding to the playing sound to the playing sound, and the amount of the indirect sound component added to the playing sound is based on the analysis result of the analysis means. Set,
The generation means generates an indirect sound component corresponding to a sound obtained by mixing the performance sound to which the indirect sound component is added by the adjustment means and the content sound.
A sound processing device characterized by this. In the sound processing apparatus according to claim 1 ,
The adjusting means removes the indirect sound component contained in the content sound in response to the input of the playing sound.
A sound processing device characterized by this. In the sound processing apparatus according to claim 2 ,
Including means for identifying the type of musical instrument sound included in the performance sound, including
The adjusting means sets an indirect sound component to be added to the playing sound based on the type of musical instrument sound included in the playing sound.
A sound processing device characterized by this. In the sound processing apparatus according to claim 2 or 4 .
Includes means for determining whether or not include the song唱音the performance sound,
The adjusting means sets an indirect sound component to be added to the playing sound based on a determination result of whether or not the playing sound includes the singing sound.
A sound processing device characterized by this. In the sound processing apparatus according to any one of claims 1 to 5 .
A first timbre analysis means for analyzing the timbre of the musical instrument sound included in the performance sound,
A second timbre analysis means for analyzing the timbre of the musical instrument sound included in the content sound is included.
The adjusting means adjusts at least one of the performance sound and the content sound based on the comparison between the analysis result of the first tone color analysis means and the analysis result of the second tone color analysis means. ,
A sound processing device characterized by this. In the sound processing apparatus according to claim 6 ,
Including means for identifying the type of musical instrument sound included in the performance sound, including
When the content sound includes a plurality of types of musical instrument sounds, the second timbre analysis means analyzes the timbre of the type of musical instrument sound included in the performance sound among the plurality of types of musical instrument sounds.
A sound processing device characterized by this. In the sound processing apparatus according to any one of claims 1 to 5 .
A first timbre analysis means for analyzing the timbre of the singing sound included in the performance sound,
A second timbre analysis means for analyzing the timbre of the singing sound included in the content sound is included.
The adjusting means adjusts at least one of the performance sound and the content sound based on the comparison between the analysis result of the first tone color analysis means and the analysis result of the second tone color analysis means. ,
A sound processing device characterized by this. An adjustment step of adjusting at least one of the performance sound and the content sound in order to match the characteristics of the user's performance sound and the content sound obtained based on the content data.
A generation step of generating an indirect sound component corresponding to a sound obtained by mixing the performance sound and the content sound, and
An output control step that outputs a sound obtained by mixing the performance sound, at least the direct sound component of the content sound, and the indirect sound component to the output means.
Only including,
In the adjustment step, the indirect sound component contained in the content sound is removed.
In the generation step, the indirect sound component corresponding to the sound formed by mixing the performance sound and the content sound from which the indirect sound component has been removed in the adjustment step is generated.
A sound processing method characterized by that. An adjusting means for adjusting at least one of the performance sound and the content sound in order to match the characteristics of the user's performance sound and the content sound obtained based on the content data.
A generation means for generating an indirect sound component corresponding to a sound obtained by mixing the performance sound and the content sound, and
An output control means that outputs a sound obtained by mixing the performance sound, at least a direct sound component of the content sound, and the indirect sound component to the output means.
A program for causing a computer to function as,
The adjusting means removes the indirect sound component contained in the content sound, and removes the indirect sound component.
The generation means generates an indirect sound component corresponding to a sound formed by mixing the performance sound and the content sound from which the indirect sound component has been removed by the adjustment means.
A program characterized by that. An adjustment step of adjusting at least one of the performance sound and the content sound in order to match the characteristics of the user's performance sound and the content sound obtained based on the content data.
A generation step of generating an indirect sound component corresponding to a sound obtained by mixing the performance sound and the content sound, and
An output control step that outputs a sound obtained by mixing the performance sound, at least the direct sound component of the content sound, and the indirect sound component to the output means.
The A including sound processing method,
Further, it includes an analysis step of analyzing the amount of indirect sound components contained in the content sound.
In the adjustment step, an indirect sound component corresponding to the performance sound is added to the performance sound, and the amount of the indirect sound component added to the performance sound is based on the analysis result of the analysis step. Set,
In the generation step, an indirect sound component corresponding to a sound obtained by mixing the performance sound to which the indirect sound component is added in the adjustment step and the content sound is generated.
A sound processing method characterized by that. An adjusting means for adjusting at least one of the performance sound and the content sound in order to match the characteristics of the user's performance sound and the content sound obtained based on the content data.
A generation means for generating an indirect sound component corresponding to a sound obtained by mixing the performance sound and the content sound, and
An output control means that outputs a sound obtained by mixing the performance sound, at least a direct sound component of the content sound, and the indirect sound component to the output means.
A program for causing a computer to function as,
Further, the analysis means for analyzing the amount of indirect sound components contained in the content sound is included.
The adjusting means adds an indirect sound component corresponding to the playing sound to the playing sound, and the amount of the indirect sound component added to the playing sound is based on the analysis result of the analysis means. Set,
The generation means generates an indirect sound component corresponding to a sound obtained by mixing the performance sound to which the indirect sound component is added by the adjustment means and the content sound.
A program characterized by that.

Images