JP2014229965A - Sound source position display device, sound source position display method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve a sound source position display device which makes the position of each sound source, changing in time series, quite obvious.SOLUTION: A CPU 10 acquires the localization information pan from the power spectra of L channel and R channel, obtained by performing frequency analysis (fast Fourier transformation) of the acoustic waveform data of L channel and R channel, and displays the work spectra, having display colors differentiated depending on the localization information pan thus acquired, on a screen in the time series order from the head to the tail of a melody, thus making the position of each sound source, changing in time series, quite obvious.

Description

  The present invention relates to a sound source position display device, a sound source position display method, and a program that make the position of a sound source in a stereo sound field clear.

  A technique for visualizing and displaying the position of a sound source in a stereo sound field is known. For example, Patent Document 1 discloses a level distribution for each frequency band in a two-dimensional plane (localization axis-frequency axis plane) determined by localization information for each frequency band extracted from an input musical sound signal. A technique is disclosed in which the existence area of a sound source such as a vocal or a musical instrument is visually recognized by expressing the level axis (height direction) on the screen in a distinguishable manner by expressing the level axis (height direction) with a level axis that is the height direction. ing.

JP 2011-158664 A

  By the way, in the technique disclosed in Patent Document 1, since the level distribution for each frequency band is expressed in the height direction of a two-dimensional plane (localization axis-frequency axis plane), vocal and instrument unit existence areas are grouped. Displayed on the screen. For this reason, there exists a problem that the position of each sound source which changes in time series according to the input acoustic signal cannot be made clear at a glance.

  The present invention has been made in view of such circumstances, and it is an object of the present invention to provide a sound source position display device, a sound source position display method, and a program capable of making the position of each sound source changing in time series clear. Yes.

  In order to achieve the above object, a sound source position display device according to the present invention includes a power acquisition unit that acquires a power spectrum from acoustic waveform data, and a localization information acquisition unit that acquires localization information from the power spectrum acquired by the power acquisition unit. And display means for displaying a power spectrum having a different display color according to the localization information acquired by the localization information acquisition means.

  In the sound source position display method of the present invention, the power spectrum is acquired from the acoustic waveform data, the localization information is acquired from the acquired power spectrum, and the power spectrum with different display colors is displayed according to the acquired localization information. It is characterized by that.

  Furthermore, in the program of the present invention, the computer acquires a power acquisition step for acquiring a power spectrum from acoustic waveform data, a localization information acquisition step for acquiring localization information from the power spectrum acquired in the power acquisition step, and the localization information And a display step of displaying a power spectrum having a different display color according to the localization information acquired in the acquisition step.

  In the present invention, the position of each sound source that changes in time series can be made clear at a glance.

1 is a block diagram showing an overall configuration of a sound source position display device 100 according to a first embodiment of the present invention. 3 is a memory map showing a configuration of a work area WE of a RAM 12; It is a figure which shows an example of the hue table CT. It is a figure which shows another example of hue table CT. It is a figure which shows another example of hue table CT. It is a flowchart which shows operation | movement of a main routine. It is a flowchart which shows the operation | movement of an analysis process. It is a flowchart which shows the operation | movement of a display process. It is a flowchart which shows operation | movement of a color value calculation process. It is a flowchart which shows operation | movement of a color value calculation process. It is a figure which shows an example of the sound source position display in 1st Embodiment. It is a memory map which shows the structure of the work area WE of RAM12 in 2nd Embodiment. It is a figure which shows an example of the color correction table RT of 2nd Embodiment. It is a figure which shows another example of the color correction table RT of 2nd Embodiment. It is a flowchart which shows the operation | movement of the display process by 2nd Embodiment. It is a flowchart which shows the operation | movement of the color correction process by 2nd Embodiment. It is a figure which shows an example of the sound source position display in 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
A. Configuration FIG. 1 is a block diagram showing an overall configuration of a sound source position display device 100 according to a first embodiment of the present invention. The sound source position display device 100 shown in this figure includes a CPU 10, ROM 11, RAM 12, input means 13, display means 14, sound output means 15, and sound input means 16.

  The CPU 10 controls each part of the apparatus according to a command input by the user via the input unit 13. The types of commands input by the user include an end command for instructing the end of processing, a playback command for instructing execution of a playback function, an acoustic input command for instructing execution of an acoustic input function, and a performance instructing execution of a performance input function There are an input command and a sound source position display command for instructing execution of a sound source position display function according to the gist of the present invention.

  The reproduction function is a function for reproducing and outputting from the sound output means 15 acoustic waveform data captured by an acoustic input function described later and performance data (MIDI data) captured by a performance input function described later. The sound input function samples the sound signal of the stereo component picked up as a stereo sound field by the sound input means 14 and stores the sound waveform data (L channel, R channel) obtained thereby in the waveform data area of the RAM 12. It is a function to do. The performance input function is a function for capturing performance data (MIDI data) input from an external MIDI instrument via the input means 13 and storing it in the performance data area of the RAM 12.

  The sound source position display function obtains a power spectrum (L channel and R channel) by performing frequency analysis (Fast Fourier Transform FFT) on the acoustic waveform data (L channel, R channel) stored in the RAM 12 by the acoustic input function. In addition, the localization information pan is acquired from the power spectra of the L channel and the R channel, and the display color of the power spectrum is changed according to the acquired localization information pan, and the time axis (horizontal axis) and the frequency axis (vertical axis) are changed. This is a function for displaying on the display screen. In this way, the power spectrum whose display color changes according to the localization information pan is displayed on the screen in time series order, so that the position of each sound source that changes in time series can be made clear at a glance.

  The ROM 11 stores various programs (including a main routine, analysis processing, display processing, and color value calculation processing described later) that implement the above-described playback function, sound input function, performance input function, and sound source position display function. The RAM 12 stores acoustic waveform data (L channel, R channel) captured by the acoustic input function and a data area (not shown) for storing performance data (MIDI data) captured by the performance input function. A work area WE for temporarily storing various register / flag data is provided as a work area of the CPU 10.

  Here, with reference to FIG. 2, main register data (variables) temporarily stored in the work area WE provided in the RAM 12 will be described. FIG. 2 shows register data (variables) used mainly when realizing the sound source position display function. The register doData [n] stores a power spectrum value for one frame. However, the power spectrum value of the R channel is stored in the first half (0 to iFrame2-1) of the register doData [n], and the power of the L channel is stored in the second half (iFrame2 to iFrame2 × 2) of the register doData [n]. The spectral value is stored.

  The register d stores a monaural power spectrum value. The register dfr stores a frequency (described later) corresponding to the pointer j. A half of the frame length (analysis window width) is stored in the register iFrame2. The register doFs stores the sampling frequency of the acoustic waveform data (L channel, R channel). The number of frames corresponding to the display width on the time axis (the width of the display area) is stored in the register timen. The register iHeight stores the height of the display area (the range of the vertical axis for logarithmically displaying the frequency).

  The register width stores the display width (number of display pixels) of the time axis. The register res100cent stores the number of display pixels corresponding to a semitone corresponding to the display resolution. The register pan stores localization information obtained by multiplying the difference between the power spectra of the left and right channels by the sum of the power spectra of both the left and right channels. Hereinafter, the value of the register pan is referred to as localization information pan. The localization information pan is “−1.0” at the left end Left, “0.0” at the center Center, and “1.0” at the right end Right.

  The registers pan, peakPosShfitR, peakPosShiftL, brightness, brightnessOffset set the hue characteristics of the hue table CT. The hue table CT converts the color (hue) when the power spectrum is displayed in color according to the localization information pan. That is, this is a data table for reading corresponding color values (r (red), g (green), b (blue)) using the localization information pan as a read address.

  Here, with reference to an example of the hue table CT illustrated in FIG. 3, the registers peakPosShfitR, peakPosShiftL, brightness, and brightnessOffset for setting the hue characteristics of the hue table CT will be described. The hue table CT shown in FIG. 3 is provided with color functions (R function, G function, B function) that determine color values (r, g, b) according to the localization information pan. Note that the color values (r, g, b) illustrated in FIG. 3 are normalized with “1” as the maximum value.

  In the example illustrated in FIG. 3, the R function includes two straight lines having the maximum color value r at the center (localization information pan = 0). That is, the slope from the value specified by the register brightness (or the offset value specified by the register brightnessOffset) to the center (positioning information pan = 0) color value r “1” at the left end (the localization function pan = −1.0). And the color value r in the center (orientation information pan = 0) from the value designated by the register brightness (or the offset value designated by the register brightnessOffset) at the right end (orientation information pan = 1.0). And a second straight line having an inclination reaching “1”.

  The G function is composed of first and second straight lines having the maximum value “1” as the color value g at the localization position specified by the value of the register peakPosShfitR and having the same inclination as the R function. The first straight line constituting the G function has a definition area (localization information pan) corresponding to a value range from a color value g (maximum value) “1” to “0”. On the other hand, the second straight line constituting the G function has a value range corresponding to the definition range from the localization position specified by the value of the register peakPosShfitR to the right end (localization information pan = 1.0).

  The B function is composed of first and second straight lines having the maximum value “1” as the color value b at the localization position specified by the value of the register peakPosShfitL and having the same slope as the R function. The first straight line constituting the B function has a value range corresponding to the definition range from the localization position designated by the value of the register peakPosShfitL to the left end (localization information pan = −1.0). On the other hand, the second straight line constituting the B function has a definition area (localization information pan) corresponding to a color value b (maximum value) “1” to “0”.

  In the hue table CT having such hue characteristics, for example, when the value of the localization information pan to be input is “0.5”, the color values (r1, g1, b1) is output. The hue characteristics of the hue table CT are adjusted according to the type of music (acoustic signal) to be analyzed. Specifically, the maximum value positions of the G function and the B function are made different from the slopes of the first and second straight lines. By changing the maximum value position of the G function and the B function, the display color of the power spectrum can be made different, and thereby the sound source position can be displayed so as to emerge.

  The slopes of the first and second straight lines in the G function and the B function are adjusted according to the spread of the sound source. If the slopes of the first and second straight lines are steep, it will not be possible to capture a sound source with a large spread, such as a piano. While capturing a sound source with a broad width, an inclination that can capture a sound source with a narrow width is empirically found.

  For example, when the slopes of the first and second straight lines constituting the RGB functions are made larger than the hue table CT shown in FIG. 3, the hue table CT having the hue characteristics shown in FIG. 4 is obtained. Further, in the hue table CT shown in FIG. 4, the maximum value of the G function is set to the left end (localization information pan = −1.0), and the maximum value of the B function is set to the right end (localization information pan = 1.0). Is a hue table CT of hue characteristics shown in FIG.

  Next, the configuration of the embodiment will be described with reference to FIG. 1 again. In FIG. 1, an input means 13 includes various operators for inputting the various commands described above (end command, playback command, sound input command, performance input command and sound source position display command), a performance input keyboard and MIDI interface ( (Not shown).

  Based on the display control signal supplied from the CPU 10, the display unit 14 displays the power spectrum whose display color changes according to the localization information pan on the screen in chronological order. As will be described later, the horizontal axis of the display screen is a time axis, and the vertical axis is a logarithmic frequency axis. The length of the time axis corresponds to one piece of music to be analyzed, and the range of the logarithmic frequency axis is a display area of 88 × resolution res100 cent or more for convenience because the frequency of a semitone in an 88-key piano is resolution rescent It becomes.

  The sound output unit 15 includes a sound source device, a D / A converter, a noise removal filter, an amplifier, and a stereo reproduction speaker. When the sound output means 15 reproduces sound waveform data, the sound waveform data (L channel, R channel) read from the data area of the RAM 12 under the control of the CPU 10 is converted into an analog signal format by the D / A converter. Then, the sound is output from the stereo reproduction speaker via a noise removal filter and an amplifier. On the other hand, when the sound output means 15 reproduces the performance data, the tone generator generates musical sound waveform data based on the performance data read from the data area of the RAM 12 under the control of the CPU 10, and this musical sound waveform data is converted into D / D. After conversion to an analog signal format by the A converter, sound is output from a stereo reproduction speaker via a noise removal filter and an amplifier.

  The acoustic input means 16 includes a stereo recording microphone, an amplifier, and an A / D converter. The sound input means 16 samples a stereo component sound signal collected as a stereo sound field under the control of the CPU 10, and stores sound waveform data (L channel, R channel) obtained thereby in the waveform data area of the RAM 12. Remember.

B. Operations Next, operations of the main routine, analysis processing, display processing, and color value calculation processing executed by the CPU 10 of the sound source position display device 100 configured as described above will be described with reference to FIGS.

(1) Operation of Main Routine FIG. 6 is a flowchart showing the operation of the main routine. When the sound source position display device 100 is powered on (powered on), the CPU 10 executes a main routine shown in FIG. 6 and proceeds to step SA1 to initialize each part of the device. Subsequently, in steps SA2 to SA6, the type of command (end command, playback command, sound input command, performance input command, and sound source position display command) input by the user operation via the input means 13 is determined. If no command is input, the determination results in steps SA2 to SA6 are all “NO”, and the apparatus waits in a state waiting for command input.

  If a playback command is input while waiting for command input, the determination result in step SA3 is “YES”, the process proceeds to step SA7, and the sound captured in step SA8 (acoustic input processing) described later is acquired. After executing the reproduction process of reproducing and outputting the waveform data and the performance data (MIDI data) acquired in step SA9 (performance input process) described later from the sound output means 15, the process returns to step SA2.

  If the sound input command is input, the determination result in step SA4 is “YES”, and the process proceeds to step SA8, where the sound input means 14 is used to sample the stereo component sound signal collected as a stereo sound field. After executing the acoustic input process for storing the acoustic waveform data (L channel, R channel) obtained in this manner in the waveform data area of the RAM 12, the process returns to step SA2.

  If a performance input command is input, the determination result in step SA5 is “YES”, and the process proceeds to step SA9, where performance data (MIDI data) input from an external MIDI instrument is input via the input means 13. After performing the performance input process stored in the performance data area of the RAM 12, the process returns to step SA2.

  When the sound source position display command is input, the determination result in step SA6 is “YES”, the process proceeds to step SA10, and the acoustic waveform data (L channel, R) stored in the RAM 12 in the acoustic input process (step SA6). Channel).

  Next, in step SA11, frequency analysis (fast Fourier transform FFT) is performed on the read acoustic waveform data (L channel, R channel) to obtain a power spectrum (L channel and R channel), and the acquired L channel and R channel are obtained. An analysis process for acquiring the localization information pan from the power spectrum is executed.

  Subsequently, in step SA12, display processing for changing the display color of the power spectrum according to the acquired localization information pan and displaying it on the display screen with the time axis (horizontal axis) and the logarithmic frequency axis (vertical axis) is performed. After execution, the process returns to step SA2. When the end command is input, the determination result in step SA2 is “YES”, and this routine is ended.

(2) Analysis Processing Operation Next, the analysis processing operation will be described with reference to FIG. When this process is executed via step SA11 (see FIG. 6) of the main routine described above, the CPU 10 proceeds to step SB1 shown in FIG. 7 and resets the pointer i to zero. When the pointer i is “0”, the acoustic waveform data of the L channel is designated, and when the pointer i is “1”, the acoustic waveform data of the R channel is designated.

  Subsequently, in step SB2, it is determined whether or not the pointer i is smaller than “2”, that is, whether or not the analysis of the acoustic waveform data of both the left and right channels has been completed. If the pointer i is smaller than “2” and the analysis of the acoustic waveform data of the left and right channels has not been completed, the determination result is “YES”, the process proceeds to step SB3, and the pointer j is reset to zero.

  The pointer j represents the number of analysis frames cut out (the number of overlaps). In the present embodiment, the acoustic waveform data of one channel is duplicated in two frames (j = 0, 1). Therefore, in this step SB4, it is determined whether or not the analysis of the acoustic waveform data of one channel has been completed.

  If the pointer i is smaller than “2” and the analysis of the acoustic waveform data of one channel is not completed, the determination result is “YES”, and the flow proceeds to Step SB5. In step SB5, the pointer j-th frame is cut out from the acoustic waveform data of the channel specified by the pointer i, and in the subsequent step SB6, it is multiplied by a window function (for example, Hanning window function).

  Next, in step SB7, FFT (Fast Fourier Transform) is performed on the acoustic waveform data cut out by the pointer j-th frame and multiplied by the window function. In subsequent step SB8, the real part and the imaginary number obtained by FFT are applied. The power spectrum value is obtained by calculating the root sum square. In step SB9, the acquired power spectrum value is stored in the register doData of the RAM 12. Subsequently, the process proceeds to step SB10, the pointer j is incremented and incremented, and the process returns to step SB4 described above.

  If the value of the incremented pointer j is smaller than “2”, that is, if the analysis of the acoustic waveform data of one channel has not been completed, the determination result in Step SB4 is “YES”, and Step SB5 described above is performed. Repeat SB10. This completes the acquisition of the power spectrum value for the L-channel acoustic waveform data.

  Now, suppose that the acquisition of the power spectrum value of the acoustic waveform data of the L channel is completed in this way, and the value of the pointer j advanced in accordance with this has become “2”. Then, the determination result in step SB4 becomes “NO”, the process proceeds to step SB11, the pointer i is incremented and stepped, and then the process returns to step SB2. If the value of the incremented pointer i is smaller than “2”, that is, if the analysis of the acoustic waveform data of both the left and right channels has not been completed, the determination result in step SB2 is “YES”, and the process proceeds to step SB3. The pointer j is reset to zero.

  Thereafter, the R channel acoustic waveform data is repeatedly executed from the R channel acoustic waveform data by repeatedly executing the processing of steps SB4 to SB10 in order to perform power spectrum analysis for overlapping and cutting out in two frames (j = 0, 1). The acquired power spectrum value is stored in the register doData of the RAM 12. Then, when the value of the pointer j incremented by repeating the processing of steps SB4 to SB10 becomes “2”, the determination result of step SB2 becomes “NO”, and this processing ends.

  In this way, in the analysis process, the acoustic waveform data (L channel, R channel) is read from the RAM 12 and subjected to frequency analysis (fast Fourier transform FFT), and the square sum of squares of the real part and imaginary part obtained by FFT is calculated. Thus, power spectrum values (L channel and R channel) are acquired and stored in the register doData. Note that the L channel power spectrum value is stored in the first half of the register doData, and the R channel power spectrum value is stored in the second half.

(3) Display Processing Operation Next, the display processing operation will be described with reference to FIG. FIG. 8 is a flowchart showing the operation of the display process. When this process is executed via step SA12 (see FIG. 6) of the main routine described above, the CPU 10 proceeds to step SC1 shown in FIG. 7 and resets the counter i to zero. The counter i counts the number of frames from “0” to “display area width time” on the horizontal axis.

  Subsequently, in step SC2, it is determined whether or not the number of frames is smaller than the “display area width time” on the horizontal axis, that is, whether or not the display has been completed up to the display area width time. If the display area width timen has not been displayed, the determination result is “YES”, and the flow proceeds to step SC3.

  In step SC3, the counter j is reset to zero. The counter j accumulates from “0” to the height iHeight of the display area on the vertical axis. Next, in step SC4, whether or not the value of the counter j is smaller than the display area height iHeight, that is, in the frame (time axis) designated by the counter i, the display is finished up to the height iHeight of the vertical display area. Determine whether or not.

If the display has not been completed until the height iHeight of the display area on the vertical axis, the determination result in step SC4 is “YES”, and the flow proceeds to step SC5. In step SC5, the value of the counter j corresponding to the scale on the vertical axis is converted into a frequency based on the following equation (1) and stored in the register dfr.
drf ← 440 × 2 ^ ((j / res100cent−69 + 21) / 12) (1)
In the above equation (1), “2 ^” represents a power of 2, and res100cent represents the number of display pixels per semitone. j / res100cent converts the position on the vertical axis (the value of counter j) into a MIDI note number. Therefore, the above equation (1) converts the converted MIDI note number (the value of the counter j) into a frequency.

Next, in step SC6, an index index for designating a frame is calculated according to the following equation (2).
index ← (iFrame2 × dfr) / (doFs / 2) (2)
In the above equation (2), iFrame2 is half the length of one frame, dfr is the frequency converted value of the position on the vertical axis (value of counter j) calculated in step SC5, and doFs is the sampling frequency.

Subsequently, in step SC7, the average value of the power spectra of the L channel and the R channel in the frame specified by the index index, that is, the monaural power d is calculated according to the following equation (3).
d ← (doDate [index] +
doDate [index + iFrame2]) / 2 (3)

Next, in step SC8, based on the following equation (4), the difference value between the L channel power spectrum and the R channel power spectrum is divided by the sum (absolute value) of the L channel and R channel power spectra. Localization information pan is calculated.
pan ← (doData [index] −doDate [index + iFrame2]) / ABS (doData [index] + doDate [index + iFrame2]) (4)

In step SC9, the color value conversion coefficient icolindex corresponding to the logarithmic value log (d) of the monaural power d is calculated according to the following equation (5).
icolindex ← −iZoom + (icoln + iZoom) × log (d) / maxlog (5)
In the above equation (5), iZoom and icoln are color value conversion constants.

  Thus, when the localization information pan and the color value conversion coefficient icolindex are obtained in step SC8 and step SC9, the color value calculation process is executed via step SC10. In the color value calculation process, as will be described later, the localization information pan is converted into a color value (r, g, b) by referring to the hue table CT (see FIG. 3) and stored in the register icol (r, g, b). Store.

  In step SC11, the register icol (r, g, b) obtained in step SC10 is set in the bitmap register bitmap [i, j × width] for screen drawing and displayed on the screen. The width is the number of display pixels on the vertical axis. In step SC11, a pitch guideline is drawn for each octave.

  Subsequently, in step SC12, the counter j is incremented and incremented, and then the process returns to step SC4. Thereafter, by repeating the above steps SC4 to SC12 until the value of the counter j exceeds the height iHeight of the display area, the display color changes according to the localization information pan in the frame (time axis) designated by the counter i. The power spectrum is displayed on the vertical axis (logarithmic frequency axis).

  When the display is completed up to the height iHeight of the display area on the vertical axis, the determination result in step SC4 is “NO”, the process proceeds to step SC13, the counter i is incremented, and the above-mentioned step is performed. Return processing to SC2. Thereafter, by repeating the above steps SC3 to SC13 until the value of the incremented counter i reaches the “display area width time” on the horizontal axis (time axis), the power spectrum in which the display color changes according to the localization information pan. Are displayed in chronological order from the beginning of the song to the end of the song. When the value of the incremented counter i exceeds the “display area width timen” on the horizontal axis (time axis), the determination result in step SC2 is “NO”, and the present process ends.

(4) Operation of Color Value Calculation Process Next, the operation of the color value calculation process will be described with reference to FIG. FIG. 9 is a flowchart showing the operation of color value processing. When this processing is executed via the above-described display processing step SC10 (see FIG. 8), the CPU 10 proceeds to step SD1 shown in FIG. 9, and stores the color value conversion coefficient icolindex calculated in the above-described display processing in the register. Replace with index. Hereinafter, the value of the register index is referred to as a color value conversion coefficient index.

  Subsequently, in step SD2 and subsequent steps, the color value conversion area of the hue table CT (see FIG. 3) is determined based on the value of the localization information pan, and the color function (R function, G function, B function) in the determined color value conversion area. ), The localization information pan is converted into color values (r, g, b). Hereinafter, the operation will be described separately for the case where the color value conversion area is “pan ≧ 0” and the case where “pan <0”.

a. When the color value conversion area is pan ≧ 0 If the color value conversion area is pan ≧ 0, the determination result in step SD2 is “YES”, the process proceeds to step SD3, and the localization information pan is obtained based on the following equation (6). The color value g is calculated by substituting it into the G function.
g ← ((1- (pan + peakPosShiftR)) × brightness + brightnessOffset-brightness) × index / icoln (6)
In the above equation (6), the color value conversion coefficient index / color value conversion constant icoln is a multiplication coefficient that determines the color value width.

Next, in step SD4, the color value r is calculated by substituting the localization information pan into the R function based on the following equation (7).
r ← ((1−pan) × brightness + brightnessOffset) × index / icoln (7)

Subsequently, in step SD5, it is determined whether or not the localization information pan is "1-peakPosShiftL" or less. If the localization information pan is equal to or less than “1-peakPosShiftL”, the determination result is “YES”, the process proceeds to step SD6, and the color value b is calculated by substituting the localization information pan into the B function based on the following equation (8). The process proceeds to step SD8.
b ← ((pan-peakPosShiftL-0) × brightness + brightnessOffset) × index / icoln (8)

On the other hand, if the localization information pan is larger than “1-peakPosShiftL”, the determination result in step SD5 is “NO”, and the process proceeds to step SD7, where the localization information pan is substituted into the B function based on the following equation (9). The color value b is calculated and the process proceeds to step SD8.
b ← ((1- (pan-peakPosShiftL)) × brightness + brightnessOffset) × index / icoln (9)

  In step SD8, the color values (r, g, b) obtained from the hue table CT (see FIG. 3) in accordance with the localization information pan in steps SD2 to SD7 are stored in the register icol, and this processing is completed. .

b. When the color value conversion area is pan <0 If the color value conversion area is pan <0, the determination result in step SD2 is “NO”, and the process proceeds to step SD9 shown in FIG. Based on the localization information pan based on the B function, the color value b is calculated.
b ← ((1 + pan-peakPosShiftL) × brightness + brightnessOffset-brightness) × index / icoln (10)

Subsequently, in step SD11, the localization value pan is substituted into the R function based on the following equation (11) to calculate the color value r.
r ← ((1 + pan) × brightness + brightnessOffset) × index / icoln (11)

Next, in step SD11, it is determined whether or not the localization information pan is “−1 + peakPosShiftR” or more. If the localization information pan is equal to or greater than “−1 + peakPosShiftR”, the determination result is “YES”, and the process proceeds to step SD12 to calculate the color value g by substituting the localization information pan into the G function based on the following equation (12). The process proceeds to step SD8.
g ← ((0− (pan + peakPosShiftR)) × brightness + brightnessOffset) × index / icorn (12)

On the other hand, if the localization information pan is smaller than “−1 + peakPosShiftR”, the determination result in step SD11 is “NO”, and the process proceeds to step SD13, where the localization information pan is substituted into the G function based on the following equation (13). The value g is calculated and the process proceeds to step SD8.
g ← ((1+ (pan + (1-peakPosShiftR))) × brightness + brightnessOffset) × index / icoln (13)

  In step SD8, the color values (r, g, b) obtained from the hue table CT (see FIG. 3) in accordance with the localization information pan in steps SD9 to SD13 are stored in the register icol, and this processing is completed. .

  As described above, according to the first embodiment, the localization information pan is obtained from the power spectra of the L channel and the R channel obtained by performing frequency analysis (fast Fourier transform FFT) on the acoustic waveform data of the L channel and the R channel. Since the power spectrum obtained by changing the display color according to the acquired localization information pan is displayed on the screen in time series from the beginning of the song to the end of the song, for example, as shown in the display screen example shown in FIG. It becomes possible to make the position of each sound source that changes to clear at a glance.

  Note that FIG. 11 shows monochrome display because color display cannot be performed, but actually the display color of the power spectrum is made different according to the localization information pan based on the above-described hue table CT (see FIG. 3). Therefore, the position of each sound source that changes in time series depending on which color (which position) is displayed becomes clear at a glance.

  In the first embodiment described above, the acoustic waveform data (L channel, R channel) stored in the waveform data area of the RAM 12 is a processing target. However, the present invention is not limited to this, and the sound is collected as a stereo sound field. Of course, it is also possible to display the sound source position changing in real time by performing power spectrum analysis and display processing on the acoustic waveform data (L channel, R channel) obtained by sampling the stereo component acoustic signal.

  In the present embodiment, the display color of the power spectrum is varied according to the localization information pan based on the hue table CT (see FIG. 3). However, the present invention is not limited to this, and the table further converts the localization information pan to lightness. In addition, a table for converting the localization information pan to saturation is provided, and by combining them, the display color, brightness, and saturation of the power spectrum are made different according to the localization information pan, thereby further clearly expressing the position of the sound source. Embodiments are possible.

[Second Embodiment]
Next, a second embodiment will be described. Since the sound source position display device 100 according to the second embodiment has the same configuration as that of the first embodiment shown in FIG. 1, the description thereof is omitted. The second embodiment is different from the first embodiment in that a color correction table RT is provided. Here, the configuration of the color correction table RT will be described with reference to FIGS.

  First, FIG. 12 is a diagram showing a configuration of main register data (variables) temporarily stored in the work area WE provided in the RAM 12 of the sound source position display device 100 according to the second embodiment. In this figure, the difference from the first embodiment is that a color correction table RT and registers emphasizeLR, eLRRCratio, and trapezoid that hold values for specifying the color correction characteristics of the color correction table RT are provided.

  The color correction table RT converts the localization information pan to a correction coefficient emphasize of color values (r, g, b). The correction coefficient emphasize is multiplied by the color value (r, g, b) converted according to the localization information pan by the above-described hue table CT.

  Here, with reference to an example of the color correction table RT shown in FIG. 13, the registers emphasizeLR, eLRRCratio, and trapezoid that hold values for specifying the color correction characteristics of the color correction table RT will be described. The color correction table RT includes a trapezoidal correction function that converts the localization information pan into a correction coefficient emphasize. This trapezoidal correction function is specified by the half width of the upper base stored in the register trapezoid, the center position of the upper base stored in the register emphasizeLR, and the inclination of the leg (hypotenuse) stored in the register eLRRCratio.

  That is, in the trapezoidal correction function, a correction coefficient emphasize of “1” is generated corresponding to the localization information pan in the range included in the upper base. In other words, the localization information pan in the range included in the upper base is not subjected to color value correction, and is a straight line having the right leg inclination from the upper base to the lower base (or the inclination of the left leg from the upper base to the lower base). In the localization information pan in the range corresponding to (a straight line having), a correction coefficient emphasize that linearly attenuates the color value is generated. Note that the inclination of the left leg (left oblique side) is opposite to the inclination of the right leg.

  The color correction table RT illustrated in FIG. 13 is an example in which the value of the register eLRRCatio indicating the inclination of the right leg (right oblique side) is “0” or less. When the inclination of the right leg is larger than “0”, as shown in FIG. 14, the correction function has a shape obtained by vertically inverting the trapezoid shown in FIG. Specifically, a correction coefficient emphasize of “0” is generated for the localization information pan in the range included in the lower base range. That is, for the localization information pan in the range included in the lower base, the color value is set to “0”. Correction for linearly increasing the color value in the localization information pan in the range corresponding to the straight line having the right leg inclination from the lower base to the upper base (or the straight line having the left leg inclination from the lower base to the upper base) The coefficient emphasize is generated.

  By using the color correction table RT having such correction characteristics, the color value corresponding to the localization information pan included in the specific range is emphasized, so that the display color of the power spectrum positioned in the specific range is emphasized. This makes it possible to display the sound source position so as to emerge.

  Hereinafter, the display processing operation of the second embodiment using such a color correction table CT will be described. FIG. 15 is a flowchart showing the operation of the display process according to the second embodiment. The operation of the second embodiment shown in this figure is different from the display process of the first embodiment shown in FIG. 8 in that the color correction process of step SE14 is provided after the color value calculation process of step SE10. It is in. Since steps SE1 to SE13 in the display process of the second embodiment are the same as steps SC1 to SC13 in the display process of the first embodiment described above, the description thereof is omitted.

  FIG. 16 is a flowchart showing the operation of color correction processing according to the second embodiment. When the color correction process is executed via step SE14 of the display process according to the second embodiment illustrated in FIG. 15, the CPU 10 proceeds to step SF1 illustrated in FIG. 16, and is stored in the register emphasize LRC from the localization information pan. It is determined whether or not the value obtained by subtracting the upper center position is “0.0” or more. In the following, the description of the operation will be divided into a case where pan-emphasize LRC ≧ 0.0 and a case where pan-emphasize LRC <0.

a. In the case of pan-emphasize LRC ≧ 0.0 In this case, the determination result in step SF1 is “YES”, the process proceeds to step SF2, and the inclination of the leg (hypotenuse) stored in the register eLRRCratio is “0.0” or more. Judge whether there is. If the inclination of the leg (the hypotenuse) is equal to or greater than “0.0”, the determination result in step SF2 is “YES”, and the process proceeds to step SF3. In step SF3, the localization information pan is substituted into the color correction table RT (trapezoidal correction function) based on the following equation (14) to calculate the correction coefficient emphasize, and then the process proceeds to step SF8.
emphasize ← 1.0-eLRRCratio ×
(Pan-trapezoid-emphasize LRC) (14)

On the other hand, if the inclination of the leg (slope) is smaller than “0.0”, the determination result in step SF2 is “NO”, and the process proceeds to step SF4. In step SF4, the localization information pan is substituted into the color correction table RT (trapezoidal correction function) based on the following equation (15) to calculate the correction coefficient emphasize, and then the process proceeds to step SF8.
emphasize ← 0.0-eLRRCratio ×
(Pan-trapezoid-emphasize LRC) (15)

b. In the case of pan-emphasize LRC <0, in this case, the determination result in step SF1 is “NO”, the process proceeds to step SF5, and the inclination of the leg (hypotenuse) stored in the register eLRRCratio is “0.0” or more. Judge whether or not. If the inclination of the leg (slope) is “0.0” or more, the determination result in step SF5 is “YES”, and the process proceeds to step SF6. In step SF6, the localization information pan is substituted into the color correction table RT (trapezoidal correction function) based on the following equation (16) to calculate the correction coefficient emphasize, and then the process proceeds to step SF8.
emphasize ← 1.0-eLRRCratio ×
(Emphasize LRC-pan-trapezoid) (16)

On the other hand, if the inclination of the leg (the hypotenuse) is smaller than “0.0”, the determination result in step SF5 is “NO”, and the process proceeds to step SF7. In step SF7, the localization information pan is substituted into the color correction table RT (trapezoidal correction function) based on the following equation (17) to calculate the correction coefficient emphasize, and the process proceeds to step SF8.
emphasize ← 0.0-eLRRCratio ×
(Emphasize LRC-pan-trapezoid-) (17)

  In step SF8, if the calculated correction coefficient emphasize is within the range of “0.0” to “1.0”, the calculated correction coefficient emphasize is used as it is, but smaller than “0.0”. In this case, the correction coefficient emphasize is set to a lower limit value of “0.0”, while when it is larger than “1.0”, the correction coefficient emphasize is set to an upper limit value of “1.0”. Next, in step SF9, the color value (r, g, b) obtained by the color value calculation process described above is multiplied by the correction coefficient emphasize to generate a corrected color value (r, g, b). In SF10, the generated correction color value (r, g, b) is stored in the register icol, and this process is finished.

  As described above, according to the second embodiment, the localization information pan is obtained from the power spectra of the L channel and the R channel obtained by performing frequency analysis (fast Fourier transform FFT) on the acoustic waveform data of the L channel and the R channel. When the power spectrum is displayed from the beginning of the song to the end of the song while changing the display color of the power spectrum according to the obtained localization information pan, and further emphasizing the power spectrum of the color value corresponding to the localization information pan included in the specific range Since the screens are displayed in the order of series, for example, the position of each sound source that changes in time series can be made clear at a glance as in the display screen example shown in FIG.

  Note that FIG. 17 shows monochrome display because color display cannot be performed. Actually, however, the display color of the power spectrum is made different according to the localization information pan based on the above-described hue table CT (see FIG. 3). Since the power spectrum of the color value corresponding to the localization information pan included in a specific range is emphasized based on the color correction table RT (see FIG. 13), the color (which position) is displayed in chronological order. The position of each sound source that changes is clear at a glance.

  Further, in the second embodiment described above, the correction color value (r, g, b) is obtained by uniformly multiplying the color value (r, g, b) obtained by the color value calculation process by the correction coefficient emphasize. However, the present invention is not limited to this. For example, a correction table for each color value r, color value g, and color value b is provided, and each color value obtained from each correction table according to the localization information pan is provided. It is also possible to obtain the correction color value (r, g, b) by multiplying the corresponding color value (r, g, b) by the correction coefficient emphasize, respectively. In this case, a desired sound source position can be clearly expressed by highlighting or suppressing display of a power spectrum of a specific color corresponding to the localization information pan included in a specific range. .

  As mentioned above, although one Embodiment of this invention was described, this invention is not limited to it, It is included in the invention described in the claim of this-application, and its equivalent range. Hereinafter, each invention described in the scope of claims at the beginning of the present application will be additionally described.

(Appendix)
[Claim 1]
Power acquisition means for acquiring a power spectrum from acoustic waveform data;
Localization information acquisition means for acquiring localization information from the power spectrum acquired by the power acquisition means;
A sound source position display device comprising: display means for displaying a power spectrum having a display color different according to the localization information acquired by the localization information acquisition means.

[Claim 2]
The display unit includes a hue conversion unit that converts the localization information acquired by the localization information acquisition unit into a hue, and displays a power spectrum corresponding to the localization information with the hue converted by the hue conversion unit. The sound source position display device according to claim 1, wherein

[Claim 3]
2. The sound source position display device according to claim 1, wherein the hue conversion unit includes a color function that generates three primary color values (r, g, b) according to the localization information acquired by the localization information acquisition unit. .

[Claim 4]
The sound source position display device according to claim 1, wherein the display means further comprises a lightness conversion means for converting the localization information acquired by the localization information acquisition means into a lightness.

[Claim 5]
The sound source position display device according to claim 1, wherein the display unit further includes a saturation conversion unit that converts the localization information acquired by the localization information acquisition unit into saturation.

[Claim 6]
The sound source position display device according to claim 1, wherein the display means displays a power spectrum in which a display color is changed according to the localization information acquired by the localization information acquisition means on a screen in time series order.

[Claim 7]
The display means further comprises color correction means for correcting the hue converted by the hue conversion means when the localization information acquired by the localization information acquisition means is included in a predetermined range. Item 1. A sound source position display device according to item 1.

[Claim 8]
The color correction unit emphasizes the hue converted by the hue conversion unit when the localization information acquired by the localization information acquisition unit is included in a specific range. Sound source position display device.

[Claim 9]
A sound source position display method comprising: acquiring a power spectrum from acoustic waveform data; acquiring localization information from the acquired power spectrum; and displaying a power spectrum in which a display color is changed according to the acquired localization information.

[Claim 10]
On the computer,
A power acquisition step of acquiring a power spectrum from the acoustic waveform data;
Localization information acquisition step of acquiring localization information from the power spectrum acquired in the power acquisition step;
And a display step of displaying a power spectrum having a different display color according to the localization information acquired in the localization information acquisition step.

10 CPU
11 ROM
12 RAM
DESCRIPTION OF SYMBOLS 13 Input means 14 Display means 15 Sound output means 16 Sound input means 100 Sound source position display apparatus

Claims (10)

Power acquisition means for acquiring a power spectrum from acoustic waveform data;
Localization information acquisition means for acquiring localization information from the power spectrum acquired by the power acquisition means;
A sound source position display device comprising: display means for displaying a power spectrum having a display color different according to the localization information acquired by the localization information acquisition means.   The display unit includes a hue conversion unit that converts the localization information acquired by the localization information acquisition unit into a hue, and displays a power spectrum corresponding to the localization information with the hue converted by the hue conversion unit. The sound source position display device according to claim 1, wherein   2. The sound source position display device according to claim 1, wherein the hue conversion unit includes a color function that generates three primary color values (r, g, b) according to the localization information acquired by the localization information acquisition unit. .   The sound source position display device according to claim 1, wherein the display means further comprises a lightness conversion means for converting the localization information acquired by the localization information acquisition means into a lightness.   The sound source position display device according to claim 1, wherein the display unit further includes a saturation conversion unit that converts the localization information acquired by the localization information acquisition unit into saturation.   The sound source position display device according to claim 1, wherein the display means displays a power spectrum in which a display color is changed according to the localization information acquired by the localization information acquisition means on a screen in time series order.   The display means further comprises color correction means for correcting the hue converted by the hue conversion means when the localization information acquired by the localization information acquisition means is included in a predetermined range. Item 1. A sound source position display device according to item 1.   The color correction unit emphasizes the hue converted by the hue conversion unit when the localization information acquired by the localization information acquisition unit is included in a specific range. Sound source position display device.   A sound source position display method comprising: acquiring a power spectrum from acoustic waveform data; acquiring localization information from the acquired power spectrum; and displaying a power spectrum in which a display color is changed according to the acquired localization information. On the computer,
A power acquisition step of acquiring a power spectrum from the acoustic waveform data;
Localization information acquisition step of acquiring localization information from the power spectrum acquired in the power acquisition step;
And a display step of displaying a power spectrum having a different display color according to the localization information acquired in the localization information acquisition step.

Images