JP2017220798A - Information processing device, sound processing method and sound processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide sound which does not make a user feel discomfort in content providing.SOLUTION: An information processing device for processing sound that is outputted in prescribed content is provided. The information processing device includes: an encode processing unit for generating a center signal from an addition value of a first signal and a second signal of stereo sound included in the sound that is outputted, and generating a peripheral signal from a difference value between the first signal and the second signal; a filter processing unit for generating corrected center signal by removing sound of a prescribed band from the generated center signal; and a decode processing unit for decoding the first signal and the second signal on the basis of the generated corrected center signal and peripheral signal.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an information processing apparatus, a sound processing method, and a sound processing program.

  In an audio device including a speaker or the like, there are various sounds such as BGM, dialogue, and sound effects that are reproduced. Here, depending on the type of sound output in parallel, there is a relative priority, and the volume or the like should be adjusted. For example, in the situation where BGM is output constantly, if the line is output intermittently, the BGM level is adjusted to be lowered according to the line output level in order to make the line easy to hear. The ducking process is performed (see, for example, Patent Document 1 and Patent Document 2).

  An example of the ducking process is a volume ducking process. In volume ducking processing, dialogue is important in the sound to be played back, so when dialogue is output, surrounding sounds (sound effects, BGM, etc.) other than the dialogue you are interested in are lowered and the dialogue is easy to hear. To do. For example, there is a method called volume ducking, in which the volume of the BGM is lowered according to the start of the dialogue and the volume of the BGM is raised according to the end of the dialogue.

  Also, in television broadcast content, pre-production is performed in which the reproduced sound is adjusted in advance so that the sound flowing along with the video can be easily heard. In pre-production, adjustments of dialogue and surrounding BGM and sound effects are performed in advance before broadcasting.

Japanese Patent Laying-Open No. 2015-201125 JP 2006-325207 A

  However, volume ducking has a drawback in that if the interruption of the dialogue continues, the volume is frequently raised and lowered, making it difficult to listen. Moreover, when the audio content changes according to the progress of the game as in a game, pre-production cannot be performed as in the case of television broadcast content. For this reason, it has been difficult to output optimal sound in content that is interactively processed according to the progress, such as a game.

  Therefore, in one aspect, an object of the present invention is to provide a sound that does not make a user feel uncomfortable in providing content.

  In one proposal, an information processing apparatus for processing a sound output with a predetermined content, wherein a central signal is obtained from an addition value of a first signal and a second signal of a stereo sound included in the output sound. An encoding processing unit that generates and generates a peripheral signal from a difference value between the first signal and the second signal, and a filter that generates a corrected central signal by removing a predetermined band of sound from the generated central signal An information processing apparatus is provided that includes a processing unit and a decoding processing unit that restores the first signal and the second signal based on the generated corrected central signal and peripheral signal.

  According to one aspect, it is possible to provide a sound that does not make the user feel uncomfortable in providing content.

The figure which shows an example of the process which a compressor performs. The figure which shows an example of the ducking method. The figure which shows an example of a function structure of the information processing apparatus concerning one Embodiment. The figure which shows an example of the hardware constitutions of the information processing apparatus concerning one Embodiment. The figure which shows an example of the hardware constitutions of MS ducking part concerning one Embodiment. The figure which shows an example of a function structure of the MS ducking part concerning one Embodiment. The figure which shows an example of a function structure of the panda packing part concerning one Embodiment. The figure for demonstrating the volume control by the panda decking concerning one Embodiment. The flowchart which shows an example of the sound process concerning one Embodiment. The flowchart which shows an example of the MS ducking process concerning one Embodiment. The flowchart which shows an example of the panda decking process concerning one Embodiment. The figure which shows an example of the control according to pan of the surround sound concerning one Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, the duplicate description is abbreviate | omitted by attaching | subjecting the same code | symbol.

[compressor]
The compressor 5 for compressing the loudness will be described with reference to FIG. As shown in FIG. 1A, the compressor 5 inputs a sound signal, and the gain (volume) shown in FIG. 1B according to the signal intensity (amplitude peak, etc.) of the waveform of the input signal. Has the ability to apply adjustments and compress the loudness of the sound.

  For example, when the parameter includes a threshold value and a compression rate, when the signal strength of the input signal exceeds the threshold value, the gain is adjusted so that the compression rate is applied to the excess amount. For example, assuming that the threshold value is 0.3 and the compression ratio is ½, and the signal strength becomes 0.5, the signal is attenuated to 80% so that the excess 0.2 becomes ½.

[Ducking]
When the main sound is played, ducking is the process of reducing the volume of other sounds to make them stand out. An example of the ducking process is a volume ducking process. The volume ducking process is a method of lowering the volume of BGM, sound effects, etc. by a certain amount during audio reproduction such as conversation. In the ducking method shown in FIG. 2A, the volume of the target sound is reduced by a certain amount when the playback of the conversational voice data is started, and the volume of the target sound is restored when the playback of the conversational voice data is completed.

  FIG. 2B shows a configuration example of automatic volume ducking by the compressor 5. For example, a side chain type compressor can be used for the compressor 5. The compressor 5 inputs a sound signal such as BGM to be ducked, inputs a conversation voice signal that the viewer wants to listen to, calculates an applied gain from the input conversation voice signal, Apply calculated gain compression. Thereby, for example, in a radio broadcast, the volume of the BGM is automatically reduced according to the volume of the voice while the speaker is speaking, so that the speaker's voice can be easily heard. The gain calculation input is called a side chain input. There is also a ducking method in which the compressor type volume ducking is improved without using the side chain type compressor 5.

  However, volume ducking has a drawback in that if conversational voice interruption continues, the volume is frequently raised and lowered, making it difficult to hear. On the other hand, the human auditory sense can analyze the difference due to the component distribution for each frequency and the phase difference depending on the direction, etc., and recognize the sound source by separating it from the mixed sound. In other words, the human auditory organ can separate and listen to sounds that are far apart even if many sounds overlap.

  For example, in order to be able to hear clearly only in conversation voices played from the center direction (center direction), only the components in the center direction of sounds other than conversation voices are processed, and sounds other than conversation voices are processed. Can be recognized separately from the conversational voice. That is, if sounds other than the conversation voice from the same direction as the conversation voice can be reduced, the conversation voice can be easily heard without changing the overall volume. As a result, it is possible to provide content such as a game that is less conscious of volume increase / decrease than the above volume ducking, can easily hear important sounds, and does not impair the sense of immersion.

  Therefore, the information processing apparatus according to an embodiment described below performs MS ducking processing. In the MS ducking process, the mid component of multiple sounds such as stereo sound and surround sound is reduced and the side component is increased to cancel the center sound such as stereo sound and surround sound. Separate multiple sounds left and right. According to this, it is possible to minimize a change in sound pressure even during ducking, and it is possible to provide a sound that is easy to hear at a relatively low processing cost.

  In addition, the information processing apparatus according to an embodiment described below performs panda ucking processing. In the panda decking process, on the assumption that the conversational voice is played when the pan in the center direction (center direction) is 0, as the pan approaches 0, the volume of the monaural sound produced by 3D is reduced. Only sounds that overlap the direction of the conversation voice can be reduced.

  Pan means a position where sound can be heard. In general, a sound source (object that emits sound) having a position (direction) represents panning (= localization: spatial position expression) by changing a ratio of sending monaural waveform signals to a plurality of speakers. Below, the information processing apparatus 10 which performs MS ducking and panda ducking is demonstrated.

[Information processing device]
First, an example of a functional configuration of the information processing apparatus 10 according to an embodiment of the present invention will be described with reference to FIG. The information processing apparatus 10 according to an embodiment can output sound, such as a sound device, a game device, a personal computer, a tablet device, a smartphone, a portable music playback device, and a wearable display device such as an HMD (Head Mount Display). It can be applied to any electronic device.

  In the present embodiment, a game machine that is an example of the information processing apparatus 10 will be described. When the information processing apparatus 10 outputs a conversation (line) of a character according to the situation during game execution, the information processing apparatus 10 is used for ducking a BGM other than the conversation, a stereo sound such as a sound effect or a surround sound, a monaural sound such as 3D The ducking method according to the embodiment is used. Note that the information processing apparatus 10 is not limited to a game machine, and may be any machine that processes sound output with predetermined content. The predetermined contents include games, TV broadcast dramas, and the like that output sounds such as heroine conversations.

  The information processing apparatus 10 includes a reception unit 11, a game execution unit 12, an MS ducking unit 13, a panda ducking unit 14, a sound processing unit 15, a graphic processing unit 16, a communication unit 17, a display unit 18, a sound output unit 19, and a storage unit. 20

  The reception part 11 receives operation with respect to the game of a player. The operations performed by the player using an input device such as a controller include, for example, operations for operating characters such as heroes and heroines appearing in the game, operations for starting or stopping the game, and the like. The game execution unit 12 executes a desired game by causing the CPU (see FIG. 4) of the information processing apparatus 10 to execute the game processing program 121 stored in the storage unit 20.

The MS ducking unit 13 performs ducking of stereo sound or surround sound by MS processing. The MS ducking unit 13 can be applied to a sound having two or more channels. A sound with 2 channels is a stereo sound, and a sound with 3 or more channels is a surround sound. That is, the MS ducking unit 13 cannot be applied to monaural sound. The MS ducking unit 13 can be applied to a multi-channel sound after a plurality of sounds are mixed. The MS ducking unit 13 applies the following MS ducking to the sound to be ducked during conversational voice reproduction.
(1) The MS ducking unit 13 performs MS encoding processing based on the input L (Left) signal and R (Right) signal. The MS ducking unit 13 generates an M (Mid) signal and an S (Side) signal from the L signal and the R signal by MS encoding processing.
(2) The MS ducking unit 13 applies filter processing to the M signal. Since the sound to be clearly heard is a human voice, the filter process cuts a designated band including the band of the human voice from BGM and sound effects.
(3) The MS ducking unit 13 restores the L signal and the R signal through the MS decoding process. The sound composed of the restored L signal and R signal is such that a predetermined band is cut as it approaches the center direction.

  The panda ducking unit 14 performs ducking in the panning process. In particular, in the present embodiment, the pan ducking unit 14 performs ducking while adjusting the left and right pan. The panda ducking unit 14 is applicable to monaural sound ducking. The panda decking unit 14 calculates the left and right pans based on the position information where the in-game sound effect is sounded, and applies the left and right pans during sound reproduction. The direction in which the sound can be heard is panning.

  In the case of a 3D sound, the panda decking unit 14 calculates the front and rear pans simultaneously with the left and right pans. However, in the MS ducking process according to the present embodiment, the front and rear pans are not directly related to the ducking process. The panducking unit 14 reduces the volume of the sound to be ducked during conversational voice reproduction as the left and right pan values approach the center.

  Note that the panda packing unit 14 does not perform filter processing. In the case of monaural signal ducking, it is not practical to perform filtering. This is because, unlike a stereo signal or a surround sound, a monaural sound is not a signal in which a large number of signals are mixed. Therefore, when filtering is performed, the same number of filters as the number of monaural sounds are required.

  The sound processing unit 15 synthesizes the ducked sound and the conversation voice, and the sound output unit 19 outputs the synthesized sound. As a result, the sound output unit 19 can process and output sound such as a conversation of a game character into a sound that is easy to hear together with sound effects and BGM according to the progress of the game. In left and right panning, the dialogue does not need to be swung to the left or right, even if it is a monaural sound, and is output with the pan located at the center. The information processing apparatus 10 according to the present embodiment executes sound processing including ducking processing by causing the CPU (see FIG. 4) to execute the sound processing program 122 stored in the storage unit 20.

  The communication unit 17 performs communication or voice communication with other game devices or portable devices. When the graphic processing unit 16 outputs a drawing command corresponding to the progress of the game, the graphic processing unit 16 outputs a video signal to the display unit 18. The display unit 18 displays a game image as the game progresses.

  Note that FIG. 3 is a block diagram focusing on functions, and each unit indicated by these functional blocks can be realized by hardware alone, software alone, or a combination of hardware and software.

[Hardware configuration of information processing device]
Next, an example of the hardware configuration of the information processing apparatus 10 according to an embodiment of the present invention will be described with reference to FIG. The information processing apparatus 10 according to the present embodiment includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, and an HDD (Hard Disk Drive) 24. Further, the information processing apparatus 10 according to the present embodiment includes a graphic card 25, an external I / F (interface) 26, a communication I / F 27, an input I / F 28, a display 29, a speaker 30, the encoder / decoder 12, and a filter 13. Have. Each part is mutually connected by a bus.

  The ROM 22 is a nonvolatile semiconductor memory that can retain internal data even when the power is turned off. The ROM 22 stores programs and data. The RAM 23 is a volatile semiconductor memory that temporarily stores programs and data.

  The HDD 24 is a non-volatile storage device that stores programs and data. The programs stored in the HDD 24 include basic software and application software that control the entire information processing apparatus 10. Various databases may be stored in the HDD 24. In the present embodiment, the HDD 24 stores various programs such as a game processing program 121 and a sound processing program 122.

  The CPU 21 implements overall control of the information processing apparatus 10 and functions installed in the information processing apparatus 10 by reading programs and data from the ROM 22 and the HDD 24 onto the RAM 23 and executing the various processes described above.

  The external I / F 26 is an interface that connects the information processing apparatus 10 to an external apparatus. The external device includes a recording medium 26a. Thereby, the information processing apparatus 10 can read and write the recording medium 26 a via the external I / F 26. Examples of the recording medium 26a include a CD (Compact Disk), a DVD (Digital Versatile Disk), an SD memory card (SD Memory card), or a USB memory (Universal Serial Bus memory).

  For example, the information processing apparatus 10 can be loaded with a recording medium 26 a in which programs such as a game processing program 121 and a sound processing program 122 are stored. These programs are read by the external I / F 26 and read into the RAM 23.

  The CPU 21 processes the various programs loaded in the RAM 23 and instructs the graphic card 25 to output an image according to the progress of the game. The graphic card 25 performs image processing of the game to be displayed on the screen according to the instruction, and causes the display 29 to draw the game image. One frame time of an image output from the graphic card 25 is, for example, 1/30 to 1/60 seconds. The graphic card 25 executes drawing of one image for each frame. That is, an image of 30 to 60 frames per second is drawn.

  The display 29 may be equipped with a touch panel. Thereby, an input operation can be performed by touching the screen without using the controller 1. In this case, the input information of the touch position detected by the touch panel is stored in the RAM 23, and the CPU 21 executes various calculation processes based on the input information stored in the RAM 23.

  The communication I / F 27 is an interface that connects the information processing apparatus 10 to a network. In addition, the communication I / F 27 may have a function of performing communication and voice communication with other game devices and portable devices via a communication unit having an antenna.

  The input I / F 28 is an interface connected to the controller 1. The user can operate the game by operating the operation button 2 and the direction key 3 of the controller 1 to cause the character to perform a predetermined action. The input I / F 28 causes the RAM 23 to store input information based on an input operation performed by the user using the controller. The CPU 21 executes various calculation processes relating to the action of the game character based on the input information stored in the RAM 23.

  The functions of the reception unit 11 and the game execution unit 12 can be realized by a process that the game processing program 121 installed in the information processing apparatus 10 causes the CPU 21 to execute. The functions of the panda decking unit 14 and the sound processing unit 15 can be realized by processing executed by the CPU 21 by the sound processing program 122 installed in the information processing apparatus 10.

  The function of the graphic processing unit 16 can be realized using the graphic card 25, for example. The function of the communication part 17 is realizable by the communication interface 27, for example. The function of the display unit 18 can be realized by the display 29, for example. The function of the sound output unit 19 can be realized by the speaker 30, for example.

  The encoder / decoder 12 performs an encoding process for generating an M signal and an S signal from the L signal and the R signal in an MS ducking process, and a decoding for restoring the L signal and the R signal from the M ′ signal and the S signal obtained by processing the M signal. Process.

The filter 13 is a filter that removes a signal in a predetermined band from the M signal.
(MS ducking)
An example of the hardware configuration of the MS ducking unit 13 is shown in FIG. 5, and an example of the functional configuration of the MS ducking unit 13 is shown in FIG. The MS ducking unit 13 ducks stereo sound or surround sound such as BGM or sound effect. The stereo sound and the surround sound are themselves a collection of many sounds, and are different from those in which the monaural sound has a position as in the panda deck described later. Since the surround sound is included in the stereo sound, a case where the stereo sound is input will be described below.

  The MS encoder 12a shown in FIG. 5 inputs the stereo sound L signal and R signal, performs an encoding process, and generates and outputs an M signal and an S signal. Specifically, the MS encoder 12a generates an M signal according to the following equation (1), and generates an S signal according to the equation (2).

M = L + R (1)
S = L−R (2)
The L signal is an example of a first signal of stereo sound, and the R signal is an example of a second signal of stereo sound. Other examples of the first signal and the second signal include the upper and lower signals output from the surround system, the signals on the upper right and the lower left, the signals on the upper left and the lower right, and the like. It is done.

  Further, the M signal and the M ′ signal obtained by processing the M signal are an example of the center signal of the stereo sound, and the S signal of the stereo sound is an example of the peripheral (side) signal. The M ′ signal is an example of a corrected center signal obtained by processing the M signal.

  The M signal is generated from the added value of the L signal and the R signal, and the S signal is generated from the difference value between the L signal and the R signal. In other words, the M signal is generated by adding the left (L signal) and right (R signal) sounds, and the central (Middle) sound component is a dominant signal. The S signal is generated by subtracting the other sound from one sound of the L signal and the R signal, and is a signal in which a lateral sound component deviating from the center is dominant. In other words, the M signal is a signal that has more central components of stereo sound than the horizontal component (or vertical component), and the S signal is a signal that has more horizontal components (or vertical component) of stereo sound than the central component. By dividing the L signal and the R signal into the M signal and the S signal in this manner, only the M signal and only the S signal can be processed in a predetermined manner.

  Human voice is included in the band of 300 Hz to 3 kHz. Therefore, the M signal output from the MS encoder 12a is passed through a low-pass filter (hereinafter referred to as “LPF 13a”) to be a signal in a band of 300 Hz or less. Separately, the M signal is passed through a high-pass filter (hereinafter referred to as “HPF13b”) to obtain a signal in a band of 3 kHz or higher. The M ′ signal, which is a signal obtained by synthesizing the signal in the band of 300 Hz or less output from the LPF 13a and the signal in the band of 3 kHz or more output from the HPF 13b, is input to the MS decoder 12b. The M ′ signal is a signal obtained by removing sounds in a band of 300 Hz to 3 kHz, that is, in a band including human voices from the input stereo sound (L signal and R signal). That is, the M ′ signal is a sound (BGM, sound effect, etc.) obtained by hollowing out the human voice band from the input sound. Note that the filter process is not passed when ducking is not in operation. In this case, the M ′ signal is the M signal itself and is input to the MS decoder 12b.

  The MS decoder 12b receives the M ′ signal and the S signal, performs a decoding process, and restores the L signal and the R signal. Thereby, the L signal and the R signal from which the sound in the band including the human voice is removed are output. Specifically, the MS decoder 12b generates an L signal according to the following equation (3), and generates an R signal according to the equation (4).

L = (M ′ + S) ÷ 2 (3)
R = (M′−S) ÷ 2 (4)
The sound processing unit 15 synthesizes predetermined speech (such as speech) with the restored L signal and R signal. The sound output unit 19 outputs the synthesized sound. There is no other sound in the band of 300 Hz to 3 kHz (including speech bands such as lines) of the restored L signal and R signal. For this reason, in the synthesized stereo sound, the dialogue is easy to hear and becomes a sound adjusted with the surrounding sounds. As a result, in the information processing apparatus 10 that performs an operation in response to an interactive operation from the user, it is easy to hear important sounds such as lines, and sounds adjusted with surrounding sounds such as BGM are generated in real time and output as content Can be provided to the user as a sound to play.

  In the present embodiment, sound effects that are important for the progress of the game, such as the sound of a gun when the main character fires a gun in an important scene, are sounds that do not want to be cut even in a band other than 300 Hz to 3 kHz. In this case, the filter processing of the M signal may be performed using a filter that does not remove the band of the sound that is not desired to be removed from the M signal.

  The MS ducking unit 13 includes an encoding processing unit 32, a filter processing unit 33, and a decoding processing unit 34 illustrated in FIG. The encoding processing unit 32 generates an M signal by adding the stereo L signal and the R signal included in the sound output by the predetermined content, and the encoding processing unit 32 subtracts the R signal from the L signal. An S signal is generated.

  The filter processing unit 33 generates an M ′ signal obtained by removing a predetermined band of sound from the generated M signal. The decode processing unit 34 restores the L signal and the R signal based on the generated M ′ signal and S signal.

The function of the encoding processing unit 32 can be realized by the MS encoder 12a. The function of the filter processing unit 33 can be realized by the LPF 13a and the HPF 13b. The function of the decoding processing unit 34 can be realized by the MS decoder 12b.
(Pundakking)
An example of the functional configuration of the panda decking unit 14 is shown in FIG. The panda decking unit 14 has functions of a pan calculation unit 35 and a volume correction unit 36. The pan calculator 35 pans the input monaural sound left and right. The volume correction unit 36 performs correction to reduce the volume as the pan approaches the center. The sound processing unit 15 synthesizes a predetermined sound (such as speech) with the corrected monaural sound. The sound output unit 19 outputs the synthesized sound.

The pan calculator 35 performs panning sound volume calculation based on the following equation. At this time, the bread range is -1.0 to 1.0, and the center of the bread is 0.0.
The left and right volume of the pan can be calculated using the following equations (5) and (6).
Left pan volume = cos (π × (pan + 1) ÷ 4) (5)
Right pan volume = sin (π × (pan + 1) ÷ 4) (6)
According to this, as shown in the graph showing the relationship between the left and right pans and the volume during the non-ducking operation in FIG. 8A, the sound output unit 19 when panning in the left and right directions with the center direction of the pan being zero. Outputs the right volume shown in FIG. 8A from the right speaker, and outputs the left volume shown in FIG. 8A from the left speaker. That is, during the non-ducking operation, the sound volume is output by applying the volume value of the panning calculation result as it is. When the pan is in the center direction, that is, 0, the left volume and the right volume are 0.707.

  During the ducking operation, the volume correction unit 36 performs correction to reduce the volume as the pan approaches the center. The correction of the volume varies depending on the formula to be applied. For example, the volume correction unit 36 may perform correction to reduce the volume by multiplying the absolute values of the right volume and left volume of the left and right pans. Further, for example, the volume correction unit 36 may perform correction to reduce the volume by subtracting the volume of the right volume of the left and right pan and the volume of the lower left volume from the volume of the left and right.

  FIG. 8B and FIG. 8C are graphs showing the relationship between the left and right pan and the volume during the ducking operation. FIG. 8B shows an example in the case of the ducking operation (weak), and FIG. 8C shows an example in the case of the ducking operation (strong). In the case of the ducking operation (strong) in FIG. 8C, the volume when the pan approaches 0 (center) or 0 is smaller than in the ducking operation (weak) in FIG. 8B. In the ducking operation of FIG. 8B, the left volume and the right volume when the pan is in the center direction, that is, 0, become smaller than 0.707. At the time of the ducking operation in FIG. 8C, the left volume and the right volume in the pan direction are 0.

  If the pan is in the center, it will be heard as the center sound. In this embodiment, when the pan approaches or approaches the center, the volume is reduced or reduced to zero. As a result, when the pan approaches or approaches the center, BGM and sound effects can be reduced or reduced to zero, and the human voice band can be hollowed out. As a result, by synthesizing a human voice with a signal obtained by hollowing out the human voice band, it is possible to generate a signal in which the dialogue is easy to hear and the surrounding sounds are adjusted.

  The premise when sound processing is started will be briefly described. If there is a waveform of the sound material that you want to play during the game, the sound processing unit 15 determines how to play it (volume, pan, etc.), processes each waveform based on the parameters, and uses the mixer to sound Is synthesized. The synthesized sound is finally output from the speaker 30. For example, sound effects with strong interactive elements are processed by changing parameters such as volume and pan in real time before being input to the mixer. On the other hand, BGM or the like that has a waveform processed by pre-production is input to the mixer with almost the same sound or a sound whose volume is changed.

  Since the pandacking process is performed at the waveform processing stage, the application target is limited to sound effects and the like. The MS ducking process is performed on the multi-channel sound signal after the target sounds are collected by the mixer.

[Sound processing method]
Next, an example of sound processing using the information processing apparatus 10 according to the present embodiment will be described with reference to FIG. When this process is started, the sound processing unit 15 determines whether or not the input sound is a monaural sound (step S10). If the sound processing unit 15 determines that the sound is not a monaural sound, the sound processing unit 15 determines whether it is an MS ducking target (step S12). If it is determined that the target is MS ducking, the MS ducking processing unit 13 executes MS ducking processing (step S14), and proceeds to step S16. On the other hand, if it is determined in step S12 that it is not an MS ducking target, the process immediately proceeds to step S16. The MS ducking process will be described later with reference to the flowchart of FIG.

  In step S10, when determining that the sound processing unit 15 is a monaural sound, the sound processing unit 15 determines whether it is a panda hacking target (step S18). When it is determined that the target is panda hacking, the panda hacking processing unit 14 executes the panda hacking process (step S20) and proceeds to step S16. On the other hand, if it is determined in step S18 that it is not a panda hacking target, the process immediately proceeds to step S16. The panda decking process will be described later with reference to the flowchart of FIG.

  In step S <b> 16, the sound processing unit 15 performs a mixing process for synthesizing a speech with the processed ambient sound. Next, the sound processing unit 15 outputs the synthesized sound (step S22) and ends this process.

In this manner, the audio signal after the MS ducking process or the panda ducking process is synthesized including the dialogue (bus routing) in the mixing process.
(MS ducking)
The MS ducking process called in step S14 in FIG. 9 will be described with reference to FIG. When the process of FIG. 10 is started, the reception unit 11 inputs the L signal and the R signal of stereo sound (step S40). Next, the encoding process part 32 encodes L signal and R signal, and produces | generates M signal and S signal (step S42). The encoding processing unit 32 generates an M signal by the following expression (1) and generates an S signal by the expression (2).

M = L + R (1)
S = L−R (2)
Next, the filter processing unit 33 filters the M signal using the LPF 13a that passes a component of 300 Hz or less (step S44). Further, the filter processing unit 33 filters the M signal using the HPF 13b that passes a component of 3 kHz or higher (step S44).

  Next, the filter processing unit 33 synthesizes the filtered signals to generate an M ′ signal (step S46). Next, the decoding processing unit 34 decodes the M signal and the S signal, restores the L signal and the R signal (step S48), and ends this processing.

L = (M ′ + S) ÷ 2 (3)
R = (M′−S) ÷ 2 (4)
(Pundakking)
Next, the panda packing process called in step S20 of FIG. 9 will be described with reference to FIG. When the process of FIG. 11 is started, the reception unit 11 inputs a monaural sound signal (step S50). Next, the pan calculation unit 35 calculates the volume corresponding to the pan (step S52).

  The bread calculator 35 determines whether the bread has approached the center (step S54). If the pan calculation unit 35 determines that the pan is not close to the center, the pan calculation unit 35 keeps the calculated volume (step S56) and proceeds to step S60. On the other hand, when determining that the pan has approached the center, the pan calculation unit 35 makes the volume smaller than the calculated volume (step S58), and ends the process.

  As described above, according to the sound processing according to the present embodiment, MS ducking and panda ducking can be performed in parallel. For example, stereo sound and surround sound are processed by the MS ducking method, and monaural sound is processed by the panda ducking method to hollow out the human voice band (300 Hz to 3 kHz band). Let

  In MS ducking, the MS ducking unit 13 removes the 300 Hz to 3 kHz band of the stereo sound, and synthesizes speech such as speech to the removed sound. As a result, in the synthesized stereo sound or the like, the speech is easy to hear, and the sound is adjusted with the surrounding sound. Thereby, in the information processing apparatus 10 that performs an operation in response to an interactive operation from the user, important sounds such as lines can be easily heard, and sounds adjusted with surrounding sounds such as BGM can be generated in real time.

  In panda hacking, the panda hacking unit 14 corrects the volume so that BGM and sound effects are reduced when the pan approaches the center or near the center, and outputs speech such as speech from the center of the pan. Is easy to hear and can generate ambient sounds and adjusted sounds in real time. It should be noted that the fact that the pan approaches the center may be, for example, when it enters a region of ３ from the center of the left and right pans or when it enters a region of ¼ from the center of the left and right pans.

In particular, in panda hacking, the sound is difficult to change from the original sound. Therefore, MS ducking is preferably used for BGM in which there is no sound at the center, and panda ducking is preferably used for 3D (3D coordinate position designation). In either case, the voice band of the speaker can be processed into a signal deleted from the original sound, and the voice of the speaker can be easily passed. Thereby, it is possible to provide a sound that does not make the user feel uncomfortable in providing the content. In addition, according to the ducking process according to the present embodiment, compared to volume ducking, the central sound (that is, a sound important for the progress of the game such as dialogue) is passed better without changing the overall volume feeling. be able to.
(Surround sound processing)
In the surround processing for processing the surround sound, the front / rear volume ratio of the front / rear pan is calculated together with the left / right pan, and the volume values of all the surround channels are determined by multiplying them. Further, the left and right sound volumes are calculated by adding interior processing. The interior processing is to reduce the left / right volume difference and the front / rear volume difference when the distance between the sound source position and the listener position is smaller than the determined interior radius.

  The case of 5.1ch surround sound will be briefly supplemented using FIG. When processing surround sound, the front left speaker A, the front right speaker B, the front center speaker C, the LFE (Low Frequency Effect) speaker D, the surround left speaker E, and the surround right speaker F There is a signal channel.

  Since the MS process (Mid / Side process) is a process for a stereo waveform signal (that is, an L signal and an R signal), sounds output from the speakers of the front and surround stereo pairs are separately processed. Further, although there is no left and right in the front center and LFE, since it can be considered that there is only sound of Mid / Side M signal, only filter processing is applied. In summary, the MS ducking process for 5.1ch surround signals is as follows.

Input front left and front right signals → MS ducking processing → Output signals to front left and front right speakers A and B Input surround left and surround right signals → MS ducking processing → Surround left and surround right speakers E , F Output signal Front center signal → Filter processing only → Output signal to front center speaker C Input LFE signal → Filter processing only → Output signal to LFE speaker D Main game sound Before explaining surround sound, here is an example of stereo sound. In the case of stereo sound, the destination speakers are the left and right speakers A and B, and the left and right directions are expressed by changing the value of pan (representing the left / right ratio, the range of −1 to 1). Let the center of the bread be 0.0.

When a monaural sound is played with the following formula (formulas (5) and (6) described above) for a given pan, the sense of volume is the same regardless of the pan. However, a stereo list ring environment is a condition.
Left speaker volume (left pan volume) = cos (π × (pan + 1) ÷ 4) (5)
Light speaker volume (right pan volume) = sin (π × (pan + 1) ÷ 4) (6)
When expanding to the surround sound used in most games, the basic concept is the same as the stereo sound, and the concept of front and rear pan is added in addition to the left and right pan shown in FIG. These two pans are determined by the direction of the sound source position (360-degree direction on the horizontal plane).

When the direction of the sound source is between the front speakers, the left and right pans are in the range of −1 to 1, and the front and rear pans are fixed to −1. When the direction of the sound source is between the side speakers, the left and right pans are fixed. Is in the range of −1 to 1 in that ratio, and the front and rear pans are fixed to 1 When the direction of the sound source is between the front left and side left, the left and right pans are fixed to −1, and the front and rear pans are in the ratio of −1 to 1 When the direction of the sound source is between the front light and the side light, the left and right pans are fixed at 1, and the front and rear pans are in the range of −1 to 1 in proportion.

The left / right volume ratio and the front / rear volume ratio are the same as the following formulas as in the case of stereo sound.
Left pan volume = Cos (π x (left and right pan + 1) ÷ 4)
Right pan volume = Sin (π x (left and right pan + 1) ÷ 4)
Front pan volume = Cos (π x (front and rear pan + 1) ÷ 4)
Rear pan volume = Sin (π x (front and rear pan + 1) ÷ 4)
From this calculation result, the volume ratio of each speaker is determined as follows.
Front Left Speaker Volume = Left Pan Volume x Forward Pan Volume Front Right Speaker Volume = Right Pan Volume x Forward Pan Volume Surround Left Speaker Volume = Left Pan Volume x Rear Pan Volume Surround Right Speaker Volume = Right Pan Volume x Rear Pan Volume The panda decking according to the embodiment performs processing for correcting the calculation formulas of the left pan volume and the right pan volume, and outputs the surround sound from each speaker without performing the volume correction otherwise.

  The information processing apparatus, the sound processing method, and the sound processing program have been described in the above embodiment. However, the information processing apparatus, the sound processing method, and the sound processing program according to the present invention are not limited to the above embodiment. Various modifications and improvements are possible within the scope of the invention. In addition, when there are a plurality of the above-described embodiments and modifications, they can be combined within a consistent range.

10: Information processing device 11: Reception unit 12: Game execution unit 12a: MS encoder 12b: MS decoder 13a: LPF
13b: HPF
13: MS ducking unit 14: Panducking unit 15: Sound processing unit 16: Graphic processing unit 17: Communication unit 18: Display unit 19: Sound output unit 20: Storage unit 21: CPU
22: ROM
23: RAM
24: HDD
25: Graphic card 26: External I / F
26a: Storage medium 27: Communication I / F
28: Input I / F
28a: Memory card 29: Display 30: Speaker 32: Encoding processing unit 33: Filter processing unit 34: Decoding processing unit 35: Pan calculation unit 36: Volume correction unit 121: Game processing program 122: Sound processing program

Claims (7)

An information processing apparatus for processing sound output with predetermined content,
A central signal is generated from the sum of the first signal and the second signal of the stereo sound included in the sound to be output, and a peripheral signal is generated from the difference value between the first signal and the second signal Encoding processing unit to
A filter processing unit that generates a corrected central signal obtained by removing a predetermined band of sound from the generated central signal;
A decoding processor that restores the first signal and the second signal based on the generated corrected central signal and peripheral signal;
An information processing apparatus. The predetermined band is a band including an audio band important for the content.
The information processing apparatus according to claim 1. A sound processing unit that synthesizes audio important to the content with the restored first signal and second signal;
A sound output unit for outputting the synthesized sound,
The information processing apparatus according to claim 1 or 2. A pan calculation unit for calculating a volume corresponding to left and right pans indicating a left / right ratio of a monaural sound included in the sound to be output;
A volume correction unit that corrects the volume so as to be smaller than the calculated volume as the left and right pan approaches the center;
The information processing apparatus according to any one of claims 1 to 3. A sound processing unit that synthesizes important audio to the content to the corrected monaural sound;
A sound output unit for outputting the synthesized sound;
The information processing apparatus according to claim 4. A sound processing method in which a computer executes processing for processing sound output with predetermined content,
A central signal is generated from the sum of the first signal and the second signal of the stereo sound included in the sound to be output, and a peripheral signal is generated from the difference value between the first signal and the second signal And
A corrected central signal is generated by removing a predetermined band of sound from the generated central signal,
Restoring the first signal and the second signal based on the generated corrected central signal and peripheral signal;
A sound processing method comprising: A sound processing program in which a computer executes processing for processing sound output with predetermined content,
A central signal is generated from the sum of the first signal and the second signal of the stereo sound included in the sound to be output, and a peripheral signal is generated from the difference value between the first signal and the second signal And
A corrected central signal is generated by removing a predetermined band of sound from the generated central signal,
Restoring the first signal and the second signal based on the generated corrected central signal and peripheral signal;
A sound processing program that causes a computer to execute processing.

Images