JP2008039958A - Sound source processing apparatus and method, and electronic equipment utilizing them - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound source processing apparatus capable of attaining flexible effect processing. <P>SOLUTION: A sound source LSI generates a voice waveform data to be reproduced, on the basis of an input data. A DSP 32 generates a waveform data for each voice according to a parameter data. An output section 49 sequentially outputs the waveform data of all voices from a main path Pm, and sequentially outputs the waveform data of predetermined voice from a path Pefx for effects. A mixer processing section 50 separately combines the waveform data which are output from the main path Pm and the path Pefx for effects, and outputs them as a main audio signal Sm and an audio signal Sefx for effects. An effect processing section 34 applies a predetermined effect processing at least on the audio signal Sefx for effects, and thereafter, combines it with the main audio signal Sm, and outputs the composite signal as the voice waveform data to be reproduced. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a sound source processing technique for generating audio data based on data described by the MIDI (Musical Instrument Digital Interface) standard or the like.

As a method for controlling an electronic musical instrument that generates various audio signals based on pre-sampled sound waveform data, control based on the MIDI standard is known. In the MIDI standard, a channel message for instructing note-on, note-off, etc. is prepared for each unit called voice, and waveform data is read according to this channel message to reproduce the sound intended by the creator of the MIDI data. For example, Patent Documents 1 and 2 describe related technologies.
JP 2001-265352 A Japanese Patent Laid-Open No. 9-237090

  In recent years, sound reproduction using such a MIDI sound source is also performed in electronic devices such as mobile phone terminals. In mobile phone terminals, etc., due to power consumption and space constraints, a high clock count cannot be used as in the case of using a dedicated MIDI sound source, and the number of processing processors and processing power that can be used are also low. It will be restricted. Therefore, when high-quality sound source processing is performed on a mobile phone terminal, it is necessary to perform the processing efficiently.

  In audio playback using MIDI, effects processing such as reverb or chorus that adds reverberation effect to the audio signal to be played, in order to enhance the sense of reality or to reproduce the music close to the intention of the music producer, In some cases, an effect process such as an equalizer for changing frequency characteristics is performed.

  When it is desired to perform desired effect processing for each timbre, it is necessary to prepare a plurality of sound sources and to prepare an effect device for each sound source that outputs the timbre for which effect processing is desired. However, since the number of processors and the operation speed are limited in a sound source processing device mounted on a mobile phone terminal or the like, it is difficult to provide a plurality of sound sources or to prepare a plurality of effect devices. . For this reason, sound source processing devices installed in conventional mobile phone terminals and the like perform effect processing uniformly only for all timbres, and it is not possible to perform effect processing for individual timbres. could not.

  The present invention has been made in view of such problems, and an object thereof is to provide a sound source processing apparatus capable of performing flexible effect processing.

  According to an aspect of the present invention, a sound source processing device that generates and outputs audio waveform data to be reproduced based on input data is provided. This sound source processing device sequentially analyzes data expressed in units of voice and outputs it as parameter data in units of voice, generates waveform data for each voice according to the parameter data, The digital signal processor that sequentially outputs the waveform data of all voices from the effect path, and the waveform data output from the main path and the effect path separately from the digital signal processing apparatus that sequentially outputs the waveform data of the predetermined voice. A mixer processing unit that synthesizes and outputs the result as a main audio signal and an effect audio signal, and at least a predetermined effect process is performed on the effect audio signal, and then is synthesized with the main audio signal and output as audio waveform data to be reproduced. An effect processing unit.

  According to this aspect, in addition to the main audio signal obtained by synthesizing the waveform data of all voices, an effect audio signal obtained by synthesizing only the waveform data of the voice to be subjected to predetermined effect processing is separately generated, thereby enabling flexible effect processing. Is possible.

  The effect processing unit may include a first signal processing unit that performs effect processing on the main audio signal and a second signal processing unit that performs effect processing on the effect audio signal.

  The first signal processing unit and the second signal processing unit may perform pipeline processing every predetermined unit time. Furthermore, the mixer processing unit and the first signal processing unit may perform pipeline processing for each predetermined unit time.

To this end, the effect processing unit may further include a memory that holds the main audio signal and the effect audio signal output from the mixer processing unit. The first signal processing unit reads the main audio signal from the memory in a certain unit time, performs a predetermined effect process, and then writes it back to the memory. The second signal processing unit reads the effect from the memory in the next unit time. The audio signal may be read out, subjected to predetermined effect processing, and then synthesized with the main audio signal and output.
By performing pipeline processing using two signal processing units, the processing efficiency can be increased. Moreover, the independence of the signal processing of the first signal processing unit and the second signal processing unit can be enhanced by performing pipeline processing via the memory.

The second signal processing unit rewrites the audio signal obtained as a result of performing predetermined effect processing on the effect audio signal in a certain unit time into the memory, and the first signal processing unit performs the next unit time. The audio signal written by the second signal processing unit may be read from the memory, and a predetermined effect process may be performed on the audio signal.
In this case, the effect processing in the first signal processing unit can be performed on the effect audio signal.

When the first signal processing unit performs a plurality of effect processing on the main audio signal, and the time required for the effect processing is longer than the unit time, the first signal processing unit A part of the plurality of effect processes is applied to the main audio signal, and the second signal processing unit reads the main audio signal from the memory in the next unit time, and applies the rest of the plurality of effect processes to the main audio signal. The effect audio signal subjected to the effect processing may be synthesized with the main audio signal.
In this case, when the frequency of the clock signal supplied to the sound source processing device is lowered, the pipeline processing breaks down by passing a part of the processing of the first signal processing unit to the second signal processing unit. Can be prevented.

  The main path may include a stereo main path that outputs stereo waveform data and an effect main path that outputs monaural waveform data. The first signal processing unit of the effect processing unit may add a reverberation effect to the waveform data output from the stereo main path using the waveform data output from the effect main path.

  The digital signal processing device may output the waveform data of the voice designated by the parameter data from the central processing unit from the effect path.

  The sound source processing device may be integrated on a single semiconductor substrate. “Integrated integration” includes the case where all of the circuit components are formed on a semiconductor substrate and the case where the main components of the circuit are integrated. A resistor, a capacitor, or the like may be provided outside the semiconductor substrate. By integrating as one LSI, the circuit area can be reduced and the characteristics of the circuit elements can be kept uniform.

  Another embodiment of the present invention is an electronic device. The electronic apparatus includes the above-described sound source processing device and a host processor that outputs data describing a sound waveform to be reproduced to the sound source processing device.

  According to still another aspect of the present invention, a sound source processing method for generating and outputting audio waveform data to be reproduced based on input data is provided. This sound source processing method is generated by sequentially analyzing data expressed in units of voice and outputting the data as parameter data in units of voice, and generating waveform data for each voice according to the parameter data. Among the waveform data, the step of sequentially outputting the waveform data of all voices, the step of sequentially outputting the waveform data of the predetermined voice among the generated waveform data, and the waveform data of the predetermined voice, Performing a predetermined effect process; and synthesizing the waveform data subjected to the predetermined effect process with the waveform data of all voices.

  According to this aspect, by generating separately the waveform data of all the voices and the waveform data of the voices to be subjected to predetermined effect processing, flexible effect processing is possible.

  Still another embodiment of the present invention also relates to a sound source processing method. This method is a sound source processing apparatus that generates and outputs audio waveform data to be reproduced, and sequentially outputs data describing characteristics of an audio signal to be reproduced, and waveform data corresponding to the data sequentially. A step of generating, synthesizing all of the waveform data and outputting as a main audio signal, a step of synthesizing waveform data corresponding to a predetermined tone among the waveform data and outputting as an audio signal for an effect, and an effect Performing a predetermined effect process on the audio signal, and synthesizing the effect audio signal subjected to the effect process with the main audio signal and outputting the synthesized audio signal as audio waveform data to be reproduced.

  It should be noted that any combination of the above-described constituent elements, and those obtained by replacing constituent elements and expressions of the present invention with each other among methods, apparatuses, systems, etc. are also effective as an aspect of the present invention.

  According to the sound source processing device of an aspect of the present invention, flexible effect processing is possible.

  The present invention will be described below based on preferred embodiments with reference to the drawings. The same or equivalent components, members, and processes shown in the drawings are denoted by the same reference numerals, and repeated descriptions are omitted as appropriate. Further, the embodiments are illustrative rather than limiting the invention, and all features and combinations thereof described in the embodiments are not necessarily essential to the invention.

  In the figure, each element described as a functional block for performing various processes can be configured with a CPU, a memory, and other LSIs in terms of hardware, and loaded into the memory in terms of software. Realized by programs. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and is not limited to any one. In addition, the functional blocks that execute different processes do not necessarily need to be configured by different arithmetic processing units, and the same arithmetic processing unit may execute a plurality of functions at different timings.

  FIG. 1 is a block diagram showing a configuration of a tone generator LSI 100 according to the embodiment. The sound source LSI 100 according to the present embodiment includes a MIDI (Musical Instrument Data Interface) sound source, and has a function of generating an audio signal corresponding to a MIDI message input from the outside and outputting the sound signal to the outside. For example, the tone generator LSI 100 is mounted on an electronic device such as a mobile phone terminal, and is used to play a ringing melody or the like.

  First, data transmission / reception with peripheral circuits including the tone generator LSI 100 will be described. FIG. 2 is a block diagram showing a configuration of the electronic device 200 on which the sound source LSI 100 is mounted. The electronic device 200 is a mobile phone terminal, and includes an operation unit 2, a host processor 4, a memory 5, an LCD monitor 6, a communication processing unit 8, an audio output unit 10, and a sound source LSI 100.

  The operation unit 2 includes a button for the user to input a telephone number and the like. The communication processing unit 8 is a communication unit that executes processing necessary for communication. Specifically, the communication processing unit 8 detects an incoming call from an external telephone or server, or transmits to an external telephone or server. Here, the term “incoming call” includes not only incoming calls but also incoming calls of packet communication from a server via a network. The same applies to outgoing calls.

  The communication processing unit 8 downloads the incoming melody from the external server via the network. At this time, in order to reduce the data transmission amount, the communication processing unit 8 downloads a music file in which music data is described. The music file describing the music data is typically a MIDI file, and may be constituted by the GM standard that is now a de facto industry standard. Note that music files described in other standards may be used, and in any case, there is an advantage that the data amount is small as compared with the case where the waveform data of music is downloaded. The downloaded music file is stored in the memory 5. In the following, as an example, the case where the ring tone data is described in the MIDI standard will be described. Note that the music file may be stored in the memory 5 not via the network but via another storage medium.

  The host processor 4 is a unit that comprehensively controls the entire processing of the electronic device 200, and corresponds to, for example, a baseband IC. The memory 5 is connected to the host processor 4 via a bus and stores various data. When there is an incoming call, the host processor 4 reads the music file stored in the memory 5 and outputs a MIDI message to the tone generator LSI 100. The tone generator LSI 100 generates voice waveform data based on the received MIDI message and outputs the voice waveform data from the voice output unit 10. The audio output unit 10 includes an amplifier that amplifies an input signal, a speaker, and an earphone.

  First, before describing the embodiments in detail, some terms are defined and explained. In this embodiment, “voice” refers to a unit for expressing one sound, for example. A channel is a concept often compared to a musical instrument player, and one channel represents one musical instrument. There is an upper limit to the number of channels, and the number of instruments that can be played simultaneously is limited. The “voice” is assigned to one of a plurality of “channels” for each tone color by assigner processing. For example, when a “voice” has a piano tone, the assigner process assigns this voice to channel 1 and when “voice” has a trumpet tone, this voice Another channel, for example, is assigned to channel 2. “Parameter data” is data used inside the sound source processing device, and includes, for example, data describing in which address the tone color designated by the voice is stored in the waveform memory at the subsequent stage. .

  In the present embodiment, the music file is described in, for example, an SMF (Standard MIDI File) format. The SMF is configured for each voice with data specifying a time between events called delta time and data including a MIDI message (hereinafter referred to as message data MS) as one unit. “Voice” is also called a note, and is a unit for expressing one sound. By describing tone color, pitch, strength, length, etc. for each “voice”, a certain sound can be expressed. The tone generator LSI 100 analyzes MIDI messages sequentially input from the host processor 4 and generates an audio signal to be reproduced.

Returning to FIG. 1, the configuration of the tone generator LSI 100 that generates and outputs audio waveform data to be reproduced based on the input message data will be described.
The tone generator LSI 100 includes an interface (hereinafter abbreviated as I / F) 20, a tone generator block 22, and a digital-analog converter (hereinafter abbreviated as DAC) 24, and is integrated as a functional IC on a single semiconductor substrate.
The I / F 20 is an interface circuit that controls communication between the host processor 4 and the sound source block 22 in FIG. 2, and message data MSG from the host processor 4 is input to the I / F 20.
The sound source block 22 analyzes the message data MSG and generates a sound waveform. For example, simultaneous pronunciation of 64 sounds (voices) is possible. As will be described later, PCM compressed wave data is stored in a ROM (Load Only Memory) inside the sound source block 22. The sound source block 22 performs assigner function, analysis of message data MSG, conversion control, and time management of access to the sound source, and sets data necessary for performance at a desired timing.

The sound source block 22 includes a CPU (Central Processing Unit) 30, a DSP (Digital Signal Processing Unit) 32, and an effect processing unit 34.
The CPU 30 sequentially analyzes the message data MSG expressed in units of voice, specifies the message type and parameters, performs assigner processing, and obtains information obtained as a result of the analysis (hereinafter simply referred to as parameter data PRM). Is output to the DSP 32.

  The DSP 32 reads the wave data from the internal ROM based on the parameter data PRM output from the CPU 30, and generates the waveform data for each voice by performing pitch envelope processing, filter envelope processing, and amplifier envelope processing. “Pitch envelope processing”, “Filter envelope processing”, and “Amplifier envelope processing” are used to change the pitch of the sound, the cutoff frequency of the filter, and the strength of the sound over time. This is a process for getting close to the timbre. Further, the DSP 32 performs a mixing process for synthesizing an audio signal of a maximum of 64 voices and outputs it to the effect processing unit 34 at the subsequent stage. The effect processing unit 34 is a block that adds chorus, reverb and other acoustic effects. The data output from the effect processing unit 34 is converted into an analog signal by the DAC 24 and output to the audio output unit 10.

The tone generator LSI 100 according to the present embodiment includes two signal processing blocks, a CPU 30 and a DSP 32. Hereinafter, data processing in these two signal processing blocks will be described in detail.
FIG. 3 is a block diagram showing a detailed configuration of the CPU 30 and the DSP 32. The CPU 30 and the DSP 32 are connected via the parameter bank 36.

  Here, a unit of arithmetic processing in the tone generator LSI 100 will be described. The waveform data reproduced by the MIDI data is sampled at the sampling time Ts. The tone generator LSI 100 processes data for a maximum of 64 voices during the sampling time Ts. A time obtained by multiplying the sampling time Ts by N times (N is an integer, for example, 128, 256, 512, 1024, etc.) is called a frame time Tf. In the present embodiment, the CPU 30 outputs the parameter data PRM generated by analyzing the message data MSG to the parameter bank 36 for each frame, with one frame as a unit of processing.

  First, data transmission / reception between the CPU 30 and the DSP 32 will be described. In the present embodiment, the CPU 30 and the DSP 32 execute pipeline processing in units of frames. The parameter bank 36 is a dual-bank memory in which the memory space is divided into two areas (banks) of the 0th bank BANK0 and the first bank BANK1, and the CPU 30 writes the parameter data PRM to the parameter bank 36. For each frame, the first bank and the second bank are switched.

  The DSP 32 reads and processes the data stored in the bank opposite to the bank used by the CPU 30. That is, in the DSP 32, the parameter data PRM stored in the first bank BANK1 is read during the period when the 0th bank BANK0 is used by the CPU 30, and the first bank BANK0 is used by the CPU 30 during the period when the first bank BANK0 is used. The parameter data PRM stored in the bank BANK1 is read.

  FIG. 4 is a time chart of frame processing by the CPU 30 and the DSP 32. The CPU 30 writes the data of the nth frame to the 0th bank BANK0 in the period Tn, writes the data of the (n + 1) th frame to the first bank BANK1 in the next period Tn + 1, and the (( n + 2) The frame data is sequentially written into the 0th bank BANK0. On the other hand, the DSP 32 receives the (n−1) th frame data stored in the first bank BANK1 of the parameter bank 40 in the period Tn, and the nth frame data stored in the 0th bank BANK0 in the period Tn + 1. In the period Tn + 2, the data of the (n + 1) th frame stored in the 0th bank BANK0 is sequentially read.

  In this way, the dual bank parameter bank 36 in which the memory space is divided is arranged between the CPU 30 and the DSP 32, and writing and reading are alternately performed using the opposite banks, so that the CPU 30 and the DSP 32 are connected. Thus, pipeline processing in units of frames is realized. This pipeline processing increases the independence of the CPU 30 and the DSP 32, enables efficient processing, and increases the degree of design freedom within each block.

  Returning to FIG. 3, the internal configuration and function of the DSP 32 will be described. The DSP 32 includes a parameter bank interface (hereinafter simply referred to as I / F) 40, a first arithmetic processing unit 42, a wave memory 44, a work memory 46, a second arithmetic processing unit 48, a mixer processing unit 50, and a timing controller 52. .

  The I / F 40 switches the access bank of the parameter bank 36 for each frame and reads data. The first arithmetic processing unit 42 and the second arithmetic processing unit 48 sequentially read out the parameter data PRM written in the parameter bank 36 in units of voice for each sampling time Ts and perform pipeline processing. Hereinafter, this pipeline processing will be described. The parameter for each voice is referred to as a voice parameter VP.

  The wave memory 44 stores wave data of various timbres encoded by PCM (Pulse Code Modulation). The first arithmetic processing unit 42 reads the parameter VP for one voice from the parameter bank 40, and reads the waveform data stored at the address specified in the read data from the wave memory 44. Further, the first arithmetic processing unit 42 performs pitch envelope processing on the read waveform data. The first calculation processing unit 42 temporarily stores data being calculated in the work memory 46 and stores data after the calculation is completed. The first arithmetic processing unit 42 completes data processing for one voice within a predetermined time (hereinafter referred to as voice time Tv). The first arithmetic processing unit 42 sequentially reads out voice parameters for each voice time, and executes arithmetic processing according to the parameters. The wave memory 44 may store data encoded by ADPCM (Adaptive Differential PCM).

  Data obtained as a result of the arithmetic processing by the first arithmetic processing unit 42 is sequentially written in the working memory 46 at different addresses. Here, the working memory 46 has a dual port, the first port is connected to the first arithmetic processing unit 42, and the second port is connected to the second arithmetic processing unit 48. With this configuration, the work memory 46 can simultaneously access data from two blocks. The second arithmetic processing unit 48 reads out the voice parameter VP from the parameter bank 36 and reads out the data processed by the first arithmetic processing unit 42 from the work memory 46 during the previous voice time. The second arithmetic processing unit 48 performs filter envelope processing and amplifier envelope processing on the data read from the work memory 46 in accordance with the voice parameter, and outputs the result to the subsequent mixer processing unit 50. The second arithmetic processing unit 48 completes the data processing for one voice within the voice time.

  The mixer processing unit 50 performs a mixing process for synthesizing voice data output in order from the second arithmetic processing unit 48, and outputs the sampling time Ts as a unit when synthesis of one sample, that is, 64 voices is completed.

  FIG. 5 is a diagram showing a flow of pipeline processing for each voice in the DSP 32. The first arithmetic processing unit 42 processes one voice at a time. The second arithmetic processing unit 48 executes predetermined processing with a delay of one voice time from the first arithmetic processing unit 42. The mixer processing unit 50 further executes predetermined processing with a delay of one voice time. The pipeline processing for each voice by the first arithmetic processing unit 42 and the second arithmetic processing unit 48 is time-managed by the timing controller 52.

  FIG. 6 is a diagram showing a state of memory access by the DSP 32 during pipeline processing in units of voices. In the period Tv1, the first arithmetic processing unit 42 refers to the address Pe1 in the parameter bank 36, reads out a parameter related to the voice V1, and executes arithmetic processing using the address W1 in the work memory 46. In the subsequent period Tv2, the first arithmetic processing unit 42 refers to the address Po2 in the parameter bank 36, reads out a parameter related to the voice V2, and executes arithmetic processing using the address W2 in the work memory 46.

  On the other hand, in the period Tv2, the second arithmetic processing unit 48 refers to the address Po2 in the parameter bank 36 and reads parameters related to the voice V2. In this period Tv2, the parameter bank 36 is accessed from the first arithmetic processing unit 42 and the second arithmetic processing unit 48. Here, the addresses of the parameter bank 36 are distinguished by odd numbers (odd) and even numbers (even), and the data relating to voices (for example, V1 and V2) that are continuously processed have different odd and even addresses. Is stored as follows. This is to prevent the first arithmetic processing unit 42 and the second arithmetic processing unit 48 from accessing the same address. In the period Tv2, the second arithmetic processing unit 48 performs a predetermined process using the address W2 of the work memory 46 in which the data processed by the first arithmetic processing unit 42 in the period Tv1 is stored. The second arithmetic processing unit 48 performs the same processing for the next voice V2 in the subsequent period T3.

Thus, in the present embodiment, pipeline processing for each voice is executed inside the DSP 32.
In the present embodiment, processing contents are allocated to the first arithmetic processing unit 42 and the second arithmetic processing unit 48 so that predetermined processing is completed in one voice time Tv. When the wave data is encoded by ADPCM, a large amount of arithmetic processing is required for the decoding processing. Therefore, the decoding processing and the amplifier envelope processing are allocated to the first arithmetic processing unit 42, and the remaining processing is performed first. 2 is preferably allocated to the arithmetic processing unit 48. However, the allocation of processing by the first arithmetic processing unit 42 and the second arithmetic processing unit 48 is not limited to this, and may be set according to the processing amount, the number of clocks, the bus width, and the like.

  Further, the DSP 32 of the tone generator LSI 100 according to the present embodiment is provided with a dual port memory that can be simultaneously accessed from both the first arithmetic processing unit 42 and the second arithmetic processing unit 48. As a result, pipeline processing can be performed appropriately by sequentially switching the memory access area for each voice. This provides more reliable processing than simply providing a buffer such as a FIFO.

  Next, a configuration for executing flexible effect processing in the tone generator LSI 100 will be described. FIG. 7 is a block diagram showing the configuration of the DSP 32 and the effect processing unit 34. For the DSP 32, only the output unit 49 and the mixer processing unit 50 provided in the final stage of the second arithmetic processing unit 48 are shown. From the output unit 49, waveform data in units of voices processed by the first calculation processing unit 42 and the second calculation processing unit 48 of FIG.

  The output unit 49 distributes the input waveform data in units of voice to a plurality of paths, and outputs it to the subsequent mixer processing unit 50. The output unit 49 includes main paths Pm_L, Pm_R, Pm_CRS, Pm_REV, and an effect path Pefx as a plurality of paths. The output unit 49 sequentially outputs the waveform data of all input voices from the main paths Pm_L, Pm_R, Rm_CRS, and Pm_REV. The output unit 49 sequentially outputs waveform data of a predetermined voice from the effect path Pefx.

  The waveform data generated by the second arithmetic processing unit 48 of FIG. 3 is a monaural signal, and the signal is distributed to the L channel and the R channel by the stereo distribution unit 51, which are output from the main paths Pm_L and Pm_R, respectively. The Further, the waveform data of the input monaural signal is output as it is from the main paths Pm_CRS and Pm_REV. The monaural signals output from the main paths Pm_CRS and Pm_REV are used for effect processing for adding reverberant sounds such as chorus and reverb, respectively.

  Further, the output unit 49 selects only waveform data of a predetermined voice from the effect path Pefx and sequentially outputs it. The selector 54 is provided for selecting a predetermined voice. The voice to be selected by the selector 54 is designated by the parameter data PRM output from the CPU 30 in FIG.

  The mixer processing unit 50 separately synthesizes (mixes) the waveform data for one frame time Tf output from the plurality of paths of the output unit 49 for each path, and outputs the combined data to the effect processing unit 34. Signals obtained by synthesizing waveform data output from the main paths Pm_L, Pm_R, Pm_CRS, and Pm_REV are referred to as main audio signals Sm_L, Sm_R, Sm_CRS, and Sm_REV, respectively. These signals are collectively referred to as the main audio signal Sm as necessary. To do. A signal obtained by synthesizing the waveform data output from the effect path Pefx is referred to as an effect audio signal Sefx.

  The effect processing unit 34 performs predetermined effect processing on at least the audio signal for effect Sfx, synthesizes it with the main audio signals Sm_L and Sm_R, and outputs it as audio waveform data Sout to be reproduced.

  Specifically, the effect processing unit 34 performs the following processing. The effect processing unit 34 includes a memory 60, a first signal processing unit 61, and a second signal processing unit 62. Audio signals Sm_L, Sm_R, Sm_CRS, and Sm_REV output from the DSP 32 are written in the memory 60. This data is updated every frame time Tf.

  The first signal processing unit 61 performs effect processing on the main audio signals Sm_L and Sm_R. The effect process performed by the first signal processing unit 61 is an effect process for adding a reverberant sound such as a chorus CRS or a reverb REV. The first signal processing unit 61 multiplies the main audio signal Sm_CRS by a predetermined coefficient, gives a predetermined delay time, and superimposes it with the main audio signals Sm_L and Sm_R to add a chorus effect. The first signal processing unit 61 adds a reverb effect by multiplying the main audio signal Sm_REV by a predetermined coefficient and giving a predetermined delay time and superimposing the main audio signal Sm_L and Sm_R.

  Further, the first signal processing unit 61 may perform an equalizer process EQ. The equalizer process EQ is a process for amplifying and attenuating a predetermined frequency component of the audio signal, and can be executed using a digital filter. The first signal processing unit 61 performs the processing instructed by the parameter data PRM among the chorus CRS, the reverb REV, and the equalizer processing EQ on the main audio signal Sm. When performing a plurality of processes, the first signal processing unit 61 performs a chorus CRS, a reverb REV, and an equalizer EQ in order. Of course, the first signal processing unit 61 may perform other effect processing in addition to or instead of these effect processing.

  Further, the second signal processing unit 62 performs predetermined effect processing on the audio signal for effects Sfx. For example, the second signal processing unit 62 performs a process of distorting audio data such as overdrive OVD. Since the effect audio signal Sfx is a signal synthesized only for a predetermined voice, the second signal processing unit 62 can perform effect processing on a desired voice.

  FIG. 8 is a block diagram showing a detailed configuration of the effect processing unit 34. The first signal processing unit 61 includes addition units 70 and 71 and a reverberation effect processing unit 72 as functional blocks. FIG. 8 shows a case where the reverberation effect processing unit 72 of the first signal processing unit 61 performs chorus processing CRS.

  The first signal processing unit 61 reads the main audio signals Sm_L, Sm_R, Sm_CRS and the audio signal DSP2_CRS for chorus processing generated by the second signal processing unit 62 from the memory 60 at a certain frame time Tf. The audio signal DSP2_CRS is data prepared to give reverberation to the signal subjected to the effect processing by the second signal processing unit 62.

  The reverberation effect processing unit 72 of the first signal processing unit 61 receives the audio signal Sm_CRS and the audio signal DSP2_CRS. The reverberation effect processing unit 72 multiplies these signals by a predetermined coefficient α, gives a delay τ, and outputs the signals as L channel and R channel stereo signals. The L channel adding unit 70 adds the main audio signal Sm_L and the output data of the L channel of the reverberation effect processing unit 72. Similarly, the R channel adding unit 71 adds the main audio signal Sm_R and the output data of the R channel of the reverberation effect processing unit 72. Output data from the L channel adder 70 and the R channel adder 71 is written to the memory 60 as data DSP1_L and DSP1_R.

The second signal processing unit 62 includes an effect processing unit 80, a stereo distribution unit 81, an L channel addition unit 82, and an R channel addition unit 83. The second signal processing unit 62 reads out the effect audio signal Sefx and the data DSP1_L and DSP1_R from the memory 60. The effect processing unit 80 performs predetermined effect processing on the effect audio signal Sfx. The output of the effect processing unit 80 is output by the stereo distribution unit 81 as data Sfx_DSP2_L and Sfx_DSP2_R of the L channel and the R channel.
Further, the output of the effect processing unit 80 is written in the memory 60 as the above-described data DSP2_CRS in order to add a reverberation effect in a monaural state.

  The L channel addition unit 82 adds the output data DSP1_L of the first signal processing unit 61 and the output data DSP2_L of the L channel of the stereo distribution unit 81. Similarly, the R channel addition unit 83 adds the output data DSP1_R of the first signal processing unit 61 and the output data DSP2_R of the R channel of the stereo distribution unit 81. The output data of the L channel addition unit 82 and the R channel addition unit 83 are output as final stereo data OUT_L and OUT_R of the effect processing unit 34.

  Although not shown in FIG. 8, the main audio signal Sm_REV is written in the memory 60 by the mixer processing unit 50. Furthermore, the audio signal DSP2_REV for reverberation processing is output from the effect processing unit 80 to the memory 60 in the same manner as the audio signal DSP2_CRS. In this case, when the reverberation process is performed, the first signal processing unit 61 may read the audio signals Sm_REV and DSP2_REV instead of the audio signals Sm_CRS and DSP2_CRS and add a reverb effect.

  FIG. 9 is a time chart of signal processing in the effect processing unit 34 of FIG. In the present embodiment, the first signal processing unit 61 and the second signal processing unit 62 execute pipeline processing every predetermined unit time. The predetermined unit time is set to a time for updating data written in the memory 60, that is, a frame time Tf.

At the frame time Tf (n), the mixer processing unit 50 synthesizes the main audio signals Sm_L (n), Sm_R (n), Sm_CRS (n), Sm_REV (n), and the effect audio signal Sefx (n), and stores the memory. 60 is written.
At the frame time Tf (n), the first signal processing unit 61 generates audio signals DSP1_L (n) and DSP1_R (n) and writes them in the memory 60.
Further, in the frame time Tf (n), the second signal processing unit 62 uses the audio signal DSPx_n () for the effect audio signal Sfx (n−1) of the previous frame time Tf (n−1). n), DSP2_R (n) and audio signals DSP2_CRS (n), DSP2_REV (n) are generated and written to the memory 60.
The generation of these data has already been described.

  In the subsequent frame time Tf (n + 1), the first signal processing unit 61 reads the main audio signals Sm_L (n), Sm_R (n), Sm_CRS (n), and Sm_REV (n). The first signal processing unit 61 uses the audio signal Sm_CRS (n) in the first half of the frame time Tf (n + 1) to add the chorus effect CRS to the audio signals Sm_L (n) and Sm_R (n). Subsequently, the reverb effect REV is added to the audio signals Sm_L (n) and Sm_R (n) using the audio signal Sm_REV (n) in the second half of the frame time Tf (n + 1). The audio signals DSP1_L (n + 1) and DSP1_R (n + 1) generated by the first signal processing unit 61 are written in the memory 60.

  On the other hand, at the frame time Tf (n + 1), the second signal processing unit 62 receives audio signals DSP1_L (n) and DSP1_R (n) that are outputs of the first signal processing unit 61 at the previous frame time Tf (n). , The effect audio signal Sefx (n) generated at the previous frame time Tf (n) and the audio signal DSP2_L (n) that is the output of the second signal processing unit 62 at the previous frame time Tf (n). , DSP2_R (n) is read out.

  The second signal processing unit 62 adds the audio signals DSP1_L (n) and DSP1_R (n) to DSP2_L (n) and DSP2_R (n), respectively, in the first half (OUTPUT) of the frame time Tf (n + 1), and the final output Are output as audio signals OUT_L (n) and OUT_R (n). This process is performed by the L channel adder 82 and the R channel adder 83 in FIG.

  Subsequently, in the second half of the frame time Tf (n + 1), the second signal processing unit 62 performs predetermined effect processing on the signal Sefx (n) and converts it to a stereo signal, thereby generating signals DSP1_L (n + 1) and DSP1_R (n + 1). Is generated. Further, the monaural signals DSP2_CRS (n + 1) and DSP2_REV (n + 1) are written in the memory 60. This processing is performed by the effect processing unit 80 and the stereo distribution unit 81 shown in FIG.

  In this way, by using the first signal processing unit 61 and the second signal processing unit 62 and performing effect processing by pipeline processing, the processing efficiency can be improved. In addition, by performing pipeline processing via the memory 60, the independence of the signal processing of the first signal processing unit 61 and the second signal processing unit 62 can be enhanced.

  Furthermore, the first signal processing unit 61 performs pipeline processing with the mixer processing unit 50. Therefore, the mixer processing unit 50, the first signal processing unit 61, and the second signal processing unit 62 perform pipeline processing with the frame time Tf as a unit. As a result, the processing efficiency can be further increased.

  In the present embodiment, signals that have been subjected to predetermined effect processing in the second signal processing unit 62 are written back to the memory 60 as audio signals DSP2_CRS and DSP2_REV, which are monaural signals for chorus and reverb, In contrast, the first signal processing unit 61 performs effect processing. As a result, a chorus effect and a reverb effect can be added to the effect audio signal Sfx.

  When the first signal processing unit 61 performs a plurality of effect processes on the main audio signal Sm, if the time required for the effect process is longer than the frame time Tf which is a unit time, the first signal process The unit 61 performs a part of the plurality of effect processes on the main audio signal Sm at a certain frame time Tf, and the second signal processing unit 62 reads the main audio signal Sm from the memory 60 at the next frame time. Further, the remainder of the plurality of effect processes may be applied to the main audio signal Sm, and the main audio signal Sm may be synthesized with the audio signals DSP2_L and R that have been subjected to the effect process by the effect processing unit 80.

  In this case, when the frequency of the clock signal supplied to the tone generator LSI 100 is lowered, a part of the processing of the first signal processing unit 61 is handed over to the second signal processing unit 62, thereby causing the pipeline processing to fail. Can be prevented.

  The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the embodiment, the case where the tone generator LSI 100 is mounted on a mobile phone terminal has been described. However, the application of the present invention is not limited to this, and can naturally be used for a MIDI tone generator dedicated device or the like. .

  In the embodiment, the MIDI sound source has been described as an example. However, the present invention can also be applied to equivalent sound sources that exist in the past or that will be developed in the future.

  Although the present invention has been described based on the embodiments, it should be understood that the embodiments merely illustrate the principles and applications of the present invention, and the embodiments are defined in the claims. Needless to say, many modifications and arrangements can be made without departing from the spirit of the present invention.

It is a block diagram which shows the structure of the sound source LSI which concerns on embodiment. It is a block diagram which shows the structure of the electronic device by which sound source LSI is mounted. It is a block diagram which shows the detailed structure of CPU and DSP. It is a time chart of the frame processing by CPU and DSP. It is a figure which shows the flow of the pipeline process of the voice unit in DSP inside. It is a figure which shows the mode of the memory access by DSP in the case of the pipeline process of a voice unit. It is a block diagram which shows the structure of DSP and an effect process part. It is a block diagram which shows the detailed structure of an effect process part. It is a time chart of the signal processing in the effect processing part of FIG.

Explanation of symbols

2 ... operation unit, 4 ... host processor, 5 ... memory, 6 ... LCD monitor, 8 ... communication processing unit, 10 ... audio output unit, 22 ... sound source block, 24 ... DAC, 30 ... CPU, 32 ... DSP, 34 ... effect processing unit, 36 ... parameter bank, 42 ... first arithmetic processing unit, 44 ... wave memory, 46 ... Working memory, 48 ... Second arithmetic processing unit, 49 ... Output unit, 50 ... Mixer processing unit, 52 ... Timing controller, 60 ... Memory, 61 ... First signal processing unit, 62 ... second signal processing unit, 70 ... L channel addition unit, 71 ... R channel addition unit, 72 ... reverberation effect processing unit, 80 ... effect processing unit 81 ... stereo distribution unit, 82 ... L Channel adding unit, 83 ··· R channel addition unit, 100 ... sound source LSI, 200 ··· electronic devices.

Claims (13)

A sound source processing device that generates and outputs audio waveform data to be reproduced based on input data,
A central processing unit that analyzes data expressed in voice units in order and outputs the data as parameter data in voice units;
A digital signal processing device that generates waveform data for each voice according to the parameter data, sequentially outputs waveform data of all voices from the main path, and sequentially outputs waveform data of predetermined voices from the effect path;
Waveform data output from the main path and the effect path are separately combined, and a mixer processing unit that outputs the main audio signal and the effect audio signal;
An effect processing unit for performing at least predetermined effect processing on the audio signal for effect, then synthesizing with the main audio signal and outputting as audio waveform data to be reproduced;
A sound source processing apparatus comprising:   The effect processing unit includes a first signal processing unit that performs an effect process on at least the main audio signal, and a second signal processing unit that performs an effect process on at least the effect audio signal. The sound source processing device according to claim 1.   The sound source processing apparatus according to claim 2, wherein the first signal processing unit and the second signal processing unit perform pipeline processing every predetermined unit time.   The sound source processing device according to claim 3, wherein the mixer processing unit and the first signal processing unit perform pipeline processing for each predetermined unit time. The effect processing unit further includes a memory that holds the main audio signal and the effect audio signal output from the mixer processing unit,
The first signal processing unit reads the main audio signal from the memory in a certain unit time, performs a predetermined effect processing, and then writes the memory again.
The second signal processing unit reads out the effect audio signal from the memory in a next unit time, performs predetermined effect processing, and then synthesizes and outputs the synthesized signal with the main audio signal. Item 4. The sound source processing device according to item 3. The second signal processing unit rewrites the audio signal obtained as a result of performing the predetermined effect processing on the effect audio signal in a certain unit time in the memory again.
The first signal processing unit reads the audio signal written by the second signal processing unit from the memory in the next unit time, and performs predetermined effect processing also on the audio signal. The sound source processing device according to claim 5. The main path is
Stereo main path that outputs stereo waveform data;
A main path for effects that outputs monaural waveform data,
Including
The first signal processing unit of the effect processing unit adds a reverberation effect to the waveform data output from the stereo main path using the waveform data output from the effect main path. The sound source processing device according to claim 2. When the first signal processing unit performs a plurality of effect processing on the main audio signal, when the time required for the effect processing is longer than the unit time,
The first signal processing unit performs a part of a plurality of effect processes on the main audio signal in a certain unit time,
The second signal processing unit reads the main audio signal from the memory in the next unit time, applies the rest of the plurality of effect processing to the main audio signal, and applies effect processing to the main audio signal. 6. The sound source processing apparatus according to claim 5, wherein the effect audio signal is synthesized.   The digital signal processing device outputs the waveform data of the voice designated by the parameter data from the central processing unit from the effect path as the waveform data of the predetermined voice. The sound source processing device according to claim 8.   9. The sound source processing device according to claim 1, wherein the sound source processing device is integrated on a single semiconductor substrate. A sound source processing device according to any one of claims 1 to 8,
A host processor for outputting data describing a sound waveform to be reproduced to the sound source processing device;
An electronic device comprising: A sound source processing method for generating and outputting audio waveform data to be reproduced based on input data,
Analyzing the data expressed in voice units in order and outputting as parameter data in voice units;
Generating waveform data for each voice according to the parameter data;
A step of sequentially outputting waveform data of all voices of the generated waveform data;
A step of sequentially outputting waveform data of a predetermined voice among the generated waveform data;
Applying predetermined effect processing to the waveform data of the predetermined voice;
Synthesizing the waveform data subjected to the predetermined effect processing with the waveform data of all the voices;
A sound source processing method comprising: Sequentially outputting data describing the characteristics of the audio signal to be reproduced;
Sequentially generating waveform data corresponding to the data;
Synthesizing all of the waveform data and outputting as a main audio signal;
Synthesizing waveform data corresponding to a predetermined tone among the waveform data, and outputting as an effect audio signal;
Applying predetermined effect processing to the audio signal for effects;
Synthesizing the effect audio signal subjected to the effect processing with the main audio signal, and outputting it as audio waveform data to be reproduced;
A sound source processing method comprising:

Images