JP4079260B2 - Music mixing apparatus, method and program - Google Patents

Description

[0001]
[Industrial application fields]
The present invention relates to a music mixing apparatus, method and program, and a template extraction method and program, and in particular, for example, processing acoustic data composed of a plurality of tracks to generate mixed down acoustic data mixed down to a smaller number of tracks. The present invention relates to a music mixing apparatus, method, and program, and template extraction method and program that can easily realize various designs related to music (music) mixing.
[0002]
[Prior art]
An example of this type of conventional music mixing device is disclosed in Patent Document 1, for example. The device of Patent Document 1 is a device dedicated to mixing (so-called multi-track digital recorder / mixer). For example, an audio signal for each part such as an instrument such as a drum or a guitar or a vocal is divided into a plurality of tracks on a built-in hard disk. Recorded (memorized). These track data are mixed down by the operation of a user (for example, a mixing engineer). Mixdown refers to an operation or operation for processing / mixing sound materials that have been recorded in a plurality of tracks while processing / mixing them, and collecting them into fewer tracks.
[0003]
For example, in this dedicated device, the user manually operates various operation means such as a volume fader and a pan control knob for each track provided on the panel while reproducing a necessary track. In response to the user's operation, the built-in CPU, DSP, and the like perform effect applying processing and sound image localization processing (panning) on each track data, and these are mixed and finally mixed into, for example, two stereo tracks. Audio data of one downed music is generated. This data becomes master data for a CD press, for example.
[0004]
On the other hand, recent commercial music mixing can be performed using a general-purpose computer such as a personal computer (PC), for example, by converting the processing performed in the above-described dedicated device into a software program. It is getting more. As this type of software or application, for example, “PRO TOOLS (registered trademark) LE” (manufactured by Digidesign Japan or Avid Japan Co., Ltd.) is well known. In addition, since a manual (manual) of this software is available on the Internet, this is cited as Non-Patent Document 1.
[0005]
In a PC in which such a program is installed, that is, a music mixing apparatus, audio data (acoustic data or the like) for each part is recorded (stored) in a plurality of tracks, for example, on a hard disk. Then, the user operates an input device such as a mouse to reproduce necessary tracks, for example, adjustment of volume faders and pan control sliders on the GUI displayed on the display, and selection of various plug-ins (effect application processing programs). And by setting parameters of the plug-in and the like, mixed down data is created.
[0006]
The mixing of music or music varies depending on the implementation method and the mixing engineer who performs the implementation. The result appears as a large difference in hearing, for example, giving the listener the impression that it is a different piece of music even if it is the same sound material. In other words, the personality characteristics of the person appear in the mixdown design. This can also be seen from the fact that a song produced by a particular producer is often commercially successful, or that a remix version by another engineer is more commercially successful than the original song.
[0007]
[Patent Document 1]
JP 2001-43604 A (FIG. 1 etc.)
[Non-Patent Document 1]
“Pro Tools Reference Guide Version 5.3 for HD Systems on Macintosh or Windows / Version 5.1.1 for MIX or LE Systems on Macintosh or Windows”, [online], Digidesign Japan, [searched November 14, 2002], Internet < URL: http://www.degidesign.co.jp/japan/serv/pdf/pt_hd_guide_i/PT5.3_Ref_Guide_E.pdf>
[0008]
[Problems to be solved by the invention]
However, effective mixing is not easy. In the prior art, there is a problem that good quality master data cannot be obtained if mixing is performed by a person who lacks mixing technology. It was especially difficult for amateur-level people to achieve a mixdown as designed or intended. For this reason, a mixing engineer training system has been developed that supports the training by providing feedback based on the acoustic analysis of the master data (analysis of sound image localization, frequency characteristics, etc.). It is cumbersome for amateurs like to go in.
[0009]
Therefore, a main object of the present invention is to provide a music mixing apparatus, a method and a program, and a template extraction method and a program capable of improving productivity in music mixing.
[0010]
[Means for Solving the Problems]
A first aspect of the present invention is a music mixing apparatus that processes sound data composed of a plurality of tracks based on a user operation and generates mixed down sound data mixed down to a smaller number of tracks. Template storage means for storing a mixdown design template including design information related to the mixdown process associated with each music group corresponding to the music group, music group setting means for setting a music group in acoustic data to be applied to the template, and track A template application means for generating mixdown sound data by applying a mixdown design template to sound data by making the music groups and music groups correspond to each other is provided. A single device.
[0011]
In the first invention described above, in the mixdown process based on the user operation, the process process information generating means for generating the mixdown process process information for the case acoustic data composed of a plurality of tracks, the case acoustic data, Based on the mixdown process information, the music group extraction means for extracting the music group in the case acoustic data and the design information on the mixdown process in each track and each music group are extracted from the mixdown process information. And a template generation unit that generates a mixdown design template in association with each track and each music group, and the mixdown design template generated by the template generation unit is a template. It may be stored in the over preparative storage means.
[0012]
The second invention includes template storage means for storing a mixdown design template including design information related to a mixdown process associated with at least each track and each music group corresponding to a music component, and is a mixdown target. This is a music mixing method in a music mixing apparatus that processes sound data composed of a plurality of tracks and generates mixed-down sound data having a smaller number of tracks. The method includes the steps of (a) setting a music group in acoustic data in accordance with a user operation, (b) reading a mixdown design template selected by the user operation from a template storage means, and (c) tracks between each other. And generating the mixdown sound data by associating the music groups with each other and applying the mixdown design template read in step (b) to the sound data.
[0013]
The third invention comprises template storage means for storing a mixdown design template including design information related to a mixdown process associated with at least each track and each music group corresponding to a music component, and is a mixdown target This is a music mixing program that is executed in a music mixing apparatus that processes sound data composed of a plurality of tracks and generates mixdown sound data with a smaller number of tracks. This program includes a music group acquisition step of acquiring music group setting information in acoustic data in response to a user operation, and a template reading for reading out a mixdown design template selected by the user operation from a template storage means. A template applying step of generating mixdown sound data is performed by applying the mixdown design template to the sound data in association with the steps and between the tracks and the music groups.
[0014]
4th invention is equipped with the template memory | storage means which memorize | stores the mixdown design template containing the design information regarding a mixdown process, processes the acoustic data which consists of several tracks, and produces | generates the mixdown acoustic data of a smaller track number This is a method for extracting a mixdown design template in a music mixing device. In this method, (a) in the mixdown process based on user operation, the step of generating mixdown process information for the case acoustic data consisting of a plurality of tracks, (b) the case acoustic data and the mixdown process A step of extracting a music group in the case acoustic data based on the process information, and (c) extracting at least each track and design information on the mixdown process in each music group from the mixdown process information. And a step of generating a mixdown design template in association with each music group, and a step of (d) storing the mixdown design template generated in step (c) in the template storage means.
[0015]
The fifth invention includes template storage means for storing a mixdown design template including design information related to the mixdown process, and processes the audio data composed of a plurality of tracks to generate mixdown audio data with a smaller number of tracks. This is a program for extracting a mixdown design template in a music mixing apparatus. This program is a processing process information generating step for generating mixdown process process information for case acoustic data consisting of a plurality of target tracks in a mixdown process based on a user operation. Based on the data and the mixdown process information, the music group extraction step for extracting the music group in the case acoustic data, and the design information on the mixdown process at least for each track and each music group from the mixdown process information A template generation step for extracting and generating a mixdown design template in association with each track and music group, and the mixdown design generated in the template generation step To execute a storage step of storing the Plates in the template storage means.
[0016]
[Action]
The music mixing apparatus includes template storage means, which stores a mixdown design template (hereinafter also simply referred to as “template”) including design information related to the mixdown process. In this music mixing apparatus, acoustic data composed of a plurality of tracks can be mixed down based on a user operation or using a template. When the mixdown is performed, a music group in the target acoustic data is set according to a user operation. Thereby, correspondence with the music group of a template is attained. Here, the music group corresponds to a music component, and can be, for example, a delimiter such as a so-called chorus or A melody. When the template is selected by the user operation, the mixdown sound data is generated by applying the template to the sound data so that the tracks and the music groups are associated with each other. That is, the user can mix down the acoustic data based on the design or characteristics of the selected template.
[0017]
For example, the template stored in the template storage unit may be extracted in a mixdown process based on a user operation. That is, in such a music mixing apparatus, in the mixdown process based on the user operation, the process information generation unit generates the mixdown process information for the case acoustic data. Based on the mixdown case, that is, the case acoustic data and the mixdown process information, the music group extracting means extracts a music group in the case acoustic data. Then, the template generation means extracts mixdown design information for each track and each music group from the mixdown process information, and this extracted information is associated with each track and each music group. A template is generated. The generated template is stored (stored) in the template storage unit. Therefore, the user can extract a template having the design from his / her mixdown session and keep it, and can use the extracted template in another session.
[0018]
Note that the design information included in the template may typically include, for example, at least information on the order of plug-ins used and their parameters, volume, and pan.
[0019]
Further, when applying the template, the acoustic data track and the template track may be associated with each other by their musical function. In this case, even when the template of the same instrument or part as the track of the acoustic data is not included in the template, it is possible to apply design information provided by, for example, a similar track associated with a musical function.
[0020]
Furthermore, the music group extracting means for extracting the music group in the case acoustic data may include, for example, the following means for extracting music group candidates and means for determining the music group. That is, the first candidate extracting means performs a correlation analysis process on each track of the case acoustic data to extract music group candidates. Further, music group candidates are extracted by the second candidate extraction means based on the change in the mixdown process information. Then, the final music group is determined by combining at least the music group candidates extracted by the first candidate extraction unit and the second candidate extraction unit by the music group determination unit.
[0021]
When a template acquisition unit is provided, a template can be acquired from another device and stored in the template storage unit. In this case, for example, a template extracted in another person's mixdown session can be used.
[0022]
【The invention's effect】
According to the present invention, a mixdown design template including the design information can be extracted from the mixdown case, and the template can be used for another mixdown session. Therefore, productivity in music mixing can be improved. In addition, it is possible for an amateur-level person to easily perform various mixdowns.
[0023]
The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.
[0024]
【Example】
A music mixing apparatus 10 of this embodiment shown in FIG. 1 includes a central processing unit (hereinafter referred to as “CPU”) 12. The CPU 12 is also called a computer or a processor, and controls the entire apparatus. For example, a computer such as a personal computer (PC) can be applied to the music mixing apparatus 10.
[0025]
The CPU 12 is connected to a RAM 14, a hard disk device (hereinafter also referred to as “HDD”) 16, a disk I / F 18, an input device 20, a display device 22, an audio I / F 24, a communication I / F, and the like via a bus or the like. Is done.
[0026]
The RAM 14 is used as a work area or buffer area for the CPU 12. The HDD 16 stores a program and various data (including screen data) for performing the music mixing process and the mixdown design template extraction process. Therefore, the CPU 12 loads these programs, data, and the like from the HDD 16 to the RAM 14 and executes processing according to these programs.
[0027]
The disk I / F 18 is connected with a drive for an information recording medium such as an optical disk such as a CD and a DVD, a magnetic disk such as an FD, or a magneto-optical disk, and the CPU 12 controls the drive. For example, when the program and data for the music mixing process described above are recorded on the information recording medium, the program and the like are read via the disk I / F 18 and stored in the HDD 16. In addition, mixed-down master data, mixed-down design template data, and the like may be written to an appropriate information recording medium as necessary.
[0028]
The input device 20 is an operation means operated by a user, and includes a pointing device such as a mouse, a keyboard, and the like. An operation signal corresponding to the operation of the input device 20 is given to the CPU 12. Therefore, the CPU 12 executes a process in accordance with a user instruction given through the input device 20. The display device 22 is, for example, a liquid crystal display (LCD), a CRT, or the like. A display signal is given from the CPU 12 to display various screens.
[0029]
The audio I / F 24 includes, for example, a D / A converter and an A / D converter, and can be connected to a speaker 28, a microphone 30, various musical instruments 32, and the like. The audio I / F 24 receives the audio data (mixed-down acoustic data or the like) generated in the RAM 14 under the control of the CPU 12, and supplies a sound signal corresponding to the audio data to the speaker 28 or a headphone (not shown). Therefore, sound is output from the speaker 28 or the like. The audio I / F 24 receives acoustic signals of parts such as vocals and choruses from the microphone 30 and acoustic signals from various musical instruments (guitar, bass, keyboard, etc.) 32, and audio data (acoustic data or audio data) corresponding to them. MIDI data or the like) can be stored in the HDD 16 by the CPU 12.
[0030]
The communication I / F 26 can be connected to a network such as a LAN or the Internet. Note that the above-described program for music mixing processing or the like may be downloaded from a server or the like on the network.
[0031]
In this embodiment, a description is given assuming a PC or the like on which a music mixing program or the like is installed. However, in another embodiment, for example, a dedicated device in which a program and various data are stored in advance in a built-in ROM or the like. It may be. In this case, the input device 20 may be various buttons, operation keys, etc. including a volume slider, a pan control knob, and the like provided on the surface of the housing.
[0032]
FIG. 2 shows an outline of a mixdown example based on a user operation in the music mixing apparatus 10. The acoustic data is sound material information that is composed of a plurality of tracks and constitutes music, and is stored in the HDD 16 by being divided into a plurality of tracks for each part or instrument. In the example shown in FIG. 2, drum 1 is track 1, bass 2 is track 1, guitar 1 is track 3, guitar 2 is track 4, keyboard 1 is track 5, keyboard 2 is track 6, track 7 is chorus, track 8 is main. Each vocal is stored. The user can set the instrument name or part name of the sound material to be stored as the track name. The music mixing apparatus 10 holds the track name set by the user as musical function information indicating the part or instrument of the track.
[0033]
The acoustic data is time-series data, and in this embodiment, for example, it is configured in the order of intro, A melody, B melody, chorus, A melody, chorus. A music piece is generally composed of these time-series music components or musical phrases (a natural separation of music pieces or a melody that represents a group of musical feelings). In the present specification, these time series music components such as intro, A melody, B melody, and chorus are referred to as a music group.
[0034]
Such acoustic data is subjected to various mixdown processes based on an appropriate mixdown operation by a user operation. On the GUI displayed on the display device 22, for example, icons for various operations for each track, a setting screen displayed as a pop-up, and the like are provided. The user operates the input device 20 to select a desired icon or the like. Do the mixdown work by selecting. For example, the user can specify a plug-in to be used for each track and set parameters of the plug-in. This plug-in is a program for applying various effects to audio data, and includes, for example, a graphic equalizer, a compressor, a reverb, a pitch shift, and the like. The plug-in can be used at a desired position (time zone) of the track as necessary. In addition, a plurality of plug-ins can be used in order for one track. Further, the user can set the volume balance and pan (for example, L / R balance) of each track while changing them over time. Then, for example, mixdown sound data mixed down to stereo 2 tracks (L and R) is generated. The mixdown sound data can be stored as master data.
[0035]
In the mixdown process by the user operation, the CPU 12 creates information on the mixdown process or process based on the user operation on the sound data in the RAM 14 as the mixdown process process data, and associates with the sound data in the HDD 16. Store. Data relating to such a mixdown case, that is, acoustic data consisting of a plurality of tracks as material and mixdown process data for the same are also referred to as mixdown case data.
[0036]
FIG. 3 shows an example of the mixdown process data generated. As can be seen from FIG. 3, the mixdown process data includes respective mixdown process information for each track. The mixdown processing information is time-series information, and includes, for example, the order of plug-ins used, parameters of the plug-ins, information on volume, information on pan, and the like. The mixdown process data includes mixdown process information for a plurality of tracks and mixdown process information for the entire track. These are information relating to a process that is collectively applied to a plurality of tracks and all tracks after, for example, a mixdown process for each track.
[0037]
Then, the music mixing apparatus 10 extracts a mixdown design template (hereinafter also simply referred to as “template”) from the mixdown example as necessary. As described in the section of the prior art, music mixing differs depending on the implementation method and the mixing engineer, and the personality characteristics of the person appear in the mixdown design. Therefore, in this music mixing apparatus 10, the characteristic is extracted from the mixdown example of a certain music, and a template is created.
[0038]
When extracting a template, first, a music group that is a time-series component of the acoustic data is extracted based on the mixdown case data. This is because the characteristics of mixdown tend to be different depending on the music group. In the music group extraction processing, group candidates are extracted by, for example, correlation analysis in the acoustic data of each track, analysis of change points in the mixdown processing information of each track, and correlation analysis of the waveform envelope of the track of the rhythm instrument. The Then, the final music group is determined by combining the results of the candidate extraction processes by weighted voting, for example.
[0039]
Subsequently, mixdown process information for each track and each music group is extracted from the mixdown process data, and further, mixdown process information for a plurality of tracks and the entire music group is extracted. The extracted mixdown processing information is mixdown design information, and mixdown design template data is created by associating the design information with the musical functions of the music group and the track.
[0040]
FIG. 4 shows an example of data of the created mixdown design template. As can be seen from FIG. 4, the mixdown design template data includes mixdown design information for each track. The mixdown design information for each track includes information indicating the part or musical function of the track and mixdown design information for each music group. In FIG. 4, the musical function of the track 1 is a drum, and the music group includes an intro, rust, A melody, B melody, and the like. The mixdown design information in each music group includes information on the types and order of plug-ins used, information on parameters for each plug-in used, information on volume, information on pan, and the like. The mixdown design template data also includes mixdown design information for a plurality of tracks and mixdown design information for the entire track. These also have the same information as the mixdown design information for each track.
[0041]
Further, the music mixing apparatus 10 can perform a mixdown process using a mixdown design template. FIG. 5 shows an outline of the mixdown process performed by applying the mixdown design template.
[0042]
In FIG. 5, the acoustic data to be mixed down is drum 1 for track 1, bass for track 2, guitar 1 for track 3, guitar 2 for track 4, keyboard 1 for track 5, keyboard 2 for track 6, and track 2 for track 7. Main vocals are stored in the chorus and the track 8, respectively. The musical function of each track can be set by the user when recording (storing) acoustic data, for example.
[0043]
The sound data is composed of the music groups arranged in the order of rust, A melody, rust, A melody, B melody, and rust. Which part of the acoustic data is which music group is set by the user before applying the mixdown design template. This makes it possible to apply mixdown design information in the same music group of the template.
[0044]
Then, by applying the mixdown design template, mixdown sound data mixed down to, for example, two stereo tracks is generated.
[0045]
When performing mixdown using a template, a list of mixdown design templates as shown in FIG. 6 is displayed on the display device 22, for example. The user can operate the input device 20 to select a template to be applied from this list. Templates displayed in this list include templates extracted and created by the user in past mixdown sessions.
[0046]
When the selected template is applied and mixed down, matching or association between the musical function of each track of the template and the musical function of the track of the target acoustic data is performed. This is because the mixdown design information of the same or similar musical function is applied to the target acoustic data track. As can be seen from FIG. 4, for example, information indicating the musical function of the track is stored in the template data, so matching is possible.
[0047]
If the template does not store mixdown design information for a track having the same musical function as that of each track of the target acoustic data, for example, a musical function similar to the musical function is stored. Apply mixdown design information for different tracks. That is, for example, a matching table indicating information on musical functions to be associated may be prepared. Thus, for example, strings and keyboards are associated with each other, and drums are associated with other rhythm instruments or percussion instruments (percussion or the like).
[0048]
Then, the mixdown design information of the template is applied to the acoustic data by matching the matched tracks and by matching the same music group. As described above, the mixdown design information includes information on the types and order of plug-ins used, information on parameters for each plug-in, information on volume, information on pan, etc. obtained from the mixdown example. It is. Therefore, by applying the template, mixdown sound data that is mixed down based on the design or characteristics of the mixdown included in the template is generated.
[0049]
In this way, simply by selecting a template, acoustic data that appropriately reflects the characteristics of the mixdown design of the template is created. In other words, the user can select and apply a mixdown design template with features that match the image of the song he is going to make, so that the acoustic data of the material is as he envisioned or close to the image You can easily finish a song with an atmosphere. Therefore, it is not necessary to manually perform operations for mixdown processing for every track and music group, so that productivity in mixdown can be improved.
[0050]
FIG. 7 shows a music mixing operation in the music mixing apparatus 10. When the music mixing program is activated in the music mixing apparatus 10, first, in step S <b> 1, the CPU 12 acquires acoustic data to be mixed down in accordance with a user operation and stores it in the HDD 16. At this time, as described above, musical functions such as musical instruments and parts of the stored acoustic data are set for each track in accordance with a user operation.
[0051]
Next, in step S3, the CPU 12 determines whether or not to mix down using the template. If “NO” in the step S3, that is, if, for example, a template use menu or the like is not selected on a screen (not shown), the CPU 12 performs an operation signal from the input device 20 based on a user operation in a succeeding step S5. In response to this, various processes for normal mixdown are executed.
[0052]
Here, the user operates the input device 20 to select a plug-in to be used for each track, set parameters thereof, control volume, pan, and the like. The CPU 12 generates acoustic data subjected to processing according to the user operation in the RAM 14 and outputs it from the speaker 28 via the audio I / F 24. The user can confirm the quality of his / her mixdown work by the sound reproduced from the speaker 28. In this way, the finally mixed down sound data is generated in the RAM 14 (see also FIG. 2).
[0053]
Further, in step S7, the CPU 12 generates mixdown process data as shown in FIG. 3 including information on the mixdown process or work process in the RAM 14, and in step S9, the mixdown process data is converted into acoustic data. And stored in the HDD 16. The acoustic data and the corresponding mixdown process data can be stored as mixdown case data, for example.
[0054]
In step S11, it is determined whether to extract a template. If “YES” in the step S11, that is, if a template extraction button or the like (not shown) is selected by a user operation, the CPU 12 executes a mixdown design template extracting process in a succeeding step S13. To do. By the template extraction process in step S13, mixdown design template data as shown in FIG. 4 having the characteristics of the mixdown session executed in step S5, for example, is created. This template extraction process is shown in detail in FIG. 8 and will be described later.
[0055]
On the other hand, if “YES” in the step S3, that is, if a template use menu or the like is selected on a screen (not shown), for example, the CPU 12 executes a mixdown process using the mixdown design template in a step S21. . By the processing in step S21, for example, mixed-down acoustic data is generated as shown in FIG. The mix-down process using this template is shown in detail in FIG. 17 and will be described later.
[0056]
If “NO” in the step S11, or when the process of the step S13 or the step S21 is ended, the CPU 12 determines whether or not to store the master data in a step S15. If “YES” in this step S15, that is, if, for example, a master data save button or the like is selected on a screen (not shown), the CPU 12 outputs the mixdown sound data generated in the RAM 14 in a step S17. Stored in the HDD 16 as master data.
[0057]
On the other hand, if “NO” in the step S15, or when the process of the step S17 is ended, the CPU 12 determines whether or not to end the mixdown session in a step S19. If “NO” in the step S19, the process returns to the step S3 to repeat the processing on the acoustic data. If “YES” in the step S19, necessary termination processing is executed to terminate this session.
[0058]
FIG. 8 shows the operation of the mixdown design template extraction process of step S13 in FIG. As shown in FIG. 8, the CPU 12 first obtains mixdown case data in step S31. That is, the CPU 12 reads out the sound data and the mixdown process data for the sound data from the HDD 16 and writes them in a predetermined area of the RAM 14.
[0059]
Next, CPU12 normalizes the volume of each track | truck of acoustic data by step S33. That is, by mixing the volume of each track, it is possible to extract the characteristics of the mixdown process related to the volume.
[0060]
In subsequent step S35, the CPU 12 executes music group extraction processing based on the mixdown case data. The music group in the acoustic data of the case data is extracted by the process of step S35. The process of step S35 is shown in detail in FIG. 9, and the description thereof will be described later.
[0061]
Next, in step S37, the CPU 12 extracts mixdown design information for each track and each music group from the mixdown process data. That is, since the mixdown process data includes mixdown process information for each track, as shown in FIG. 3, the design information regarding the mixdown process in each music group extracted in step S35 from them. Is extracted.
[0062]
In subsequent step S39, the CPU 12 extracts mixdown design information for a plurality of tracks and the entire music group. That is, as shown in FIG. 3, the mixdown process data includes mixdown process information for a plurality of tracks and the whole, and design information regarding the mixdown process in each music group is extracted from these. .
[0063]
Then, in step S41, the CPU 12 associates the extracted mixdown design information with the musical function and music group of the track, thereby creating mixdown design template data as shown in FIG. .
[0064]
In subsequent step S <b> 43, the CPU 12 stores the created mixdown design template data in the HDD 16. At this time, the user can set the name of the mixdown design template, and the template is stored together with the name. The saved template is then displayed in the template list when mixdown processing using the template is performed. When the process of step S43 is completed, the process returns to the flow of FIG. 7 and proceeds to step S15.
[0065]
FIG. 9 shows the operation of the music group extraction process based on the mixdown case data in step S35 in FIG. As shown in FIG. 9, first, in step S51, the CPU 12 extracts a minimum basic beat by histogram processing of volume time-series data in each track.
[0066]
Next, in step S53, the CPU 12 executes a correlation analysis process by shifting the time in units of minimum basic beats with respect to the time frequency map of each track. Thereby, the correlation for each basic beat can be known. For example, FIG. 10 shows the correlation with respect to the first beat of certain track data. Here, the higher the correlation is, the darker it is drawn, and it can be seen that there is a strong correlation when the shift width is 0 (autocorrelation), 10 and 16.
[0067]
In step S55, the CPU 12 creates correlation map data. That is, the correlation map x (i, j) is obtained by repeating the same processing as step S53 for the second and subsequent beats in units of the minimum beat. For example, FIG. 11 shows an example of a correlation map. As is clear from FIG. 11, when there is a repetitive component, an oblique line appears. However, the leftmost diagonal line corresponds to autocorrelation.
[0068]
Subsequently, in step S57, the CPU 12 executes a mask process on the correlation map. By this processing, it is possible to detect a portion where a plurality of continuous movements or patterns exist. For example, a mask process for extracting two continuous patterns is expressed as follows. x (i, j) = 1/2 * x (i, j) + 1/2 * x (i + 1, j + 1)
[0069]
An example of a correlation map subjected to such a masking process for extracting two continuous patterns is shown in FIG. This corresponds to the (1/2, 1/2) product-sum filter processing for the portion indicated by “●” in FIG. In the correlation map of FIG. 12 subjected to this masking process, the curve on the left side shows a projection profile (projection profile), and the number of repeated patterns starting from that line is added for each line, The larger the number, the larger the value.
[0070]
FIG. 13 shows an example of a correlation map subjected to four continuous pattern extraction processing. This corresponds to the product-sum filter processing of (1/4, 1/4, 1/4, 1/4). Furthermore, FIG. 14 shows an example of a correlation map that has been subjected to extraction processing of eight continuous patterns.
[0071]
In step S59 (FIG. 9), the CPU 12 extracts a repeated pattern as a group candidate. That is, first, by looking at the projection profile in the correlation map obtained in the process of step S57 and searching for a local peak having a large value, the start point of the music group candidate is obtained. For example, in the case of the extraction process of 8 continuous patterns in FIG. 14, a peak can be detected at the 9th line, that is, at the 9th beat, and this is the start point. Since the number of repetitions is 8, it is possible to detect the arrangement from the ninth to the eighth as group candidates. And the same group can be selected by searching for the thing with a high correlation value in the line. The basic strength of the group is given by (number of beat units belonging to the group) / (total number of beat units).
[0072]
In step S61, the CPU 12 calculates the correlation value level of the group candidate. That is, as seen from each group candidate, an average of correlation values of other groups determined to be the same is calculated and held as a correlation value level of the group.
[0073]
In subsequent step S63, CPU 12 determines, for example, 2 ⁿ * 3 ^m It is determined whether or not processing has been performed on the data in the sequence (m and n are integers of 0 or more). This 2 ⁿ * 3 ^m Is equivalent to what is seen as a repetition of music.
In another embodiment, it may be determined whether or not processing has been performed on half the number of beat units.
[0074]
If “NO” in the step S63, that is, if the process still remains, the process returns to the step S57, and the process for extracting the continuous pattern of the arrangement is repeated. On the other hand, if “YES” in the step S63, that is, if the processing has already been performed on the necessary number of arranged data, the process proceeds to the step S65.
[0075]
Note that the processing of steps S53 to S63 can be applied to not only the track data of a single tone melody instrument but also the track data of a rhythm instrument or polyphony, so that music group candidates can be extracted.
[0076]
In step S65, the CPU 12 extracts group candidates based on the change point of the mixdown processing information in each track. That is, the volume and pan control for each track, the type of plug-in used and its parameters, etc. tend to be changed at the boundaries of music groups. Therefore, group candidates are extracted by analyzing the mixdown process data and extracting these change points.
[0077]
Subsequently, in step S67, group candidates are extracted from the rhythm instrument track by correlation analysis of the waveform envelope. FIG. 15A shows the envelope of the waveform of the rhythm instrument, and FIG. 15B shows its spectrum map. It can be seen that the envelope of the waveform matches the volume distribution, which is about 0.55 seconds per beat in this example. For rhythm instruments, group candidates are extracted by calculating the similarity of the waveform envelope.
[0078]
As can be seen from the spectrum map of FIG. 15B, it can be seen that there is a correlation with a fine rhythm of one beat or less. Therefore, the spectrum map of a musical instrument or part having a pitch such as vocal or bass contains frequency data of each location, and can be an information source with higher discrimination ability. Therefore, group candidates obtained from the correlation analysis of these tracks can be treated as more weighted or reliable information. For a harmonic instrument such as a guitar, the spectral map may be regarded as a two-dimensional image and correlation analysis may be performed as it is.
[0079]
In the following step S69, the CPU 12 extracts the final music group by combining the results of the group candidate extraction processes. Through the processing up to this point, group candidates based on a plurality of repetition numbers are extracted for each track with a correlation value level. Therefore, for example, for each track, voting is performed by assigning weights to places (time zones) determined to be the same group (substituting a predetermined value for addition). Then, a group candidate having a large value is determined to be a music group. Subsequently, in step S71, the CPU 12 assigns a label to each confirmed music group.
[0080]
For example, FIG. 16 shows a confirmed music group. Each track is divided into group candidates obtained by the respective candidate extraction processes, and a music group is determined by combining and combining them. In FIG. 16, for example, when the track 3 in which the guitar 1 is stored is taken as an example, “T3-1”, “T3-2, T3-3, T3-2, T3-3”, “T3-4”. ”,“ T3-5, T3-6 ”,“ T3-7, T3-8 ”,“ T3-9 ”, and“ T3-2, T3-3, T3-2, T3-3 ” To be separated. The second and seventh groups are the same group that appears repeatedly. Then, labels of A, B, C, D, E, F, and B are set in order for each music group.
[0081]
The setting of the music group label is simply A, B,... In FIG. 16, but the name setting can be performed in step S71. For example, first, “rust” tends to sing with a high vocal sound, and therefore a group in which the vocal is conspicuous can be determined as rust. In addition, since rust tends to be repeated, whether or not it is repeated can be one of the judgment materials. In addition, “Intro” can be set to the group of the start portion of the music. Furthermore, “A melody” is set for the first group other than rust, and “B melody” is set for the second group other than A melody in the portion other than rust. obtain.
[0082]
When the process of step S71 is completed, the process returns to FIG. 8 and proceeds to step S35.
[0083]
FIG. 17 shows the operation of the mixdown process using the mixdown design template in step S21 in FIG. As shown in FIG. 17, first, in step S <b> 81, the CPU 12 acquires information on the musical function of each track of the acoustic data to be mixed down. Since the musical function of each track is set by user input, for example, when recording or storing in the HDD 16, the CPU 12 reads the information and writes it in a predetermined area of the RAM 14. For example, in the example of FIG. 5, the musical functions of tracks 1 to 8 are drum, bass, guitar 1, guitar 2, keyboard 1, keyboard 2, chorus, and main vocal in this order.
[0084]
Next, in step S83, the CPU 12 acquires music group information in the acoustic data to be mixed down. The music group of the target acoustic data is set by user input, and the CPU 12 receives the information and writes it in a predetermined area of the RAM 14. For example, in the example of FIG. 5, the music group of the acoustic data is in the order of chorus, A melody, chorus, A melody, B melody, and chorus. For example, the user can set a music group by dividing a track into a plurality of time zones and inputting a music group name corresponding to each time zone on a music group setting screen (not shown).
[0085]
In subsequent step S85, the CPU 12 displays a list of mixdown design templates as shown in FIG. The user can select a template to be applied from this list by operating the input device 20.
[0086]
In step S87, the CPU 12 determines whether any mixdown design template is selected by a user operation. If “YES” in the step S87, that is, if any template is selected in the list, the CPU 12 reads the data of the selected template from the HDD 16 in a predetermined area of the RAM 14 in a succeeding step S89. Writing, matching or associating each track in the template with each track of the acoustic data to be mixed down by its musical function. Thus, the mixdown design information of the template track having the same or similar musical function is applied to each track of the acoustic data.
[0087]
Subsequently, in step S91, the CPU 12 associates the tracks and music groups matched in step S89 with each other and applies the mixdown design information of the template to the target sound data, thereby mixing down the mixed down sound. Data is generated in a predetermined area of the RAM 14.
[0088]
In step S93, the CPU 12 gives the generated mixdown sound data to, for example, the audio I / F 24 and reproduces it from the speaker 28. Therefore, the user can listen and confirm how the acoustic data is mixed down by applying the selected template.
[0089]
In step S95, the CPU 12 determines whether the template to be applied has been changed by the user. If “YES” in the step S95, that is, if a template change button or the like (not shown) is selected by the user, for example, the process returns to the step S85 to repeat the process.
[0090]
On the other hand, if “NO” in the step S95, that is, if the user does not select the template change, in a step S97, the CPU 12 determines whether or not the mixdown sound data is adjusted by the user. Judging. If “YES” in the step S97, that is, for example, if the user performs an operation for adjusting the acoustic data on a screen (not shown), the CPU 12 outputs the mixdown acoustic data based on the user's adjusting operation in a step S99. Generate, and return to step S93 to repeat the process. In this way, the user can make further fine adjustments to finish the music closer to his / her preference or design.
[0091]
If “NO” in the step S97, that is, if the adjustment operation is no longer performed by the user, the process returns to the flowchart of FIG. 7 and proceeds to the step S15.
[0092]
According to this embodiment, a template can be extracted from an actual mixdown case, and the extracted template can be used for another mixdown session. Therefore, the productivity of mixdown can be improved.
[0093]
Note that the mixdown design template is not limited to those extracted in the past own mixdown session. That is, in another embodiment, a template extracted in another person's mixdown session may be acquired. For example, “design template_producer C (or D)” shown together in the list of FIG. 6 is a template extracted in the mixdown session of producer C (or D). The template of another person can be obtained, for example, by downloading from a distribution server on the Internet via the communication I / F 26, or obtained from an appropriate information recording medium in the HDD 16 via the disk I / F 18 or the like ( Storage).
[0094]
In this way, templates extracted outside of your own mixdown session can be used, so it is easy for amateur users to perform mixdown tasks that were previously difficult, and mixdown of various designs. Can be realized. For example, a template created by a professional mixing engineer or producer can also be used, so it is possible to easily produce another version of the music that has been finished with the professional style or taste, and for example, the atmosphere of the music according to the TPO It is also possible to enjoy something that has never been seen before.
[0095]
On the other hand, the professional side can not only improve the productivity of his production activities, but also introduce new mixdown design templates by distributing them through distribution servers on the Internet or appropriate information recording media, for example. It is also possible to create a business.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an internal configuration of a music mixing apparatus according to an embodiment of the present invention.
FIG. 2 is an illustrative view for explaining an outline of a mixdown example by a user operation.
3 is an illustrative view showing mixdown process data stored in the RAM of FIG. 1 embodiment; FIG.
4 is an illustrative view showing mixdown design template data stored in the RAM of FIG. 1 embodiment; FIG.
FIG. 5 is an illustrative view for explaining an outline of mixdown performed by applying a mixdown design template;
6 is an illustrative view showing one example of a list screen of a mixdown design template displayed on the display device of FIG. 1 embodiment; FIG.
FIG. 7 is a flowchart showing the operation of music mixing processing in the embodiment in FIG. 1;
FIG. 8 is a flowchart showing the operation of the mixdown design template extraction process in FIG. 7;
FIG. 9 is a flowchart showing an operation of music group extraction processing based on the mixdown case data in FIG. 8;
FIG. 10 is an illustrative view for explaining correlation analysis processing in units of basic beats.
FIG. 11 is an illustrative view showing one example of a correlation map obtained by correlation analysis processing in units of basic beats.
12 is an illustrative view showing one example of a correlation map when a mask process for extracting two continuous patterns is performed on the correlation map of FIG. 11;
13 is an illustrative view showing one example of a correlation map when a mask process for extracting four continuous patterns is performed on the correlation map of FIG. 11;
14 is an illustrative view showing one example of a correlation map when mask processing for extracting eight continuous patterns is performed on the correlation map of FIG. 11. FIG.
FIG. 15 is an illustrative view showing acoustic data of a rhythm instrument, where (A) shows an envelope of a waveform and (B) shows a spectrum map.
FIG. 16 is an illustrative view for explaining an outline of music group determination.
FIG. 17 is a flowchart showing an operation of a mixdown process using the mixdown design template in FIG.
[Explanation of symbols]
10: Music mixing device
12 ... CPU
14 ... RAM
16 ... HDD
18 ... Disk I / F
20 ... Input device
22 Display device
24 ... Audio I / F
26 Communication I / F

Claims (10)

In a music mixing apparatus that generates acoustic data composed of a plurality of tracks based on a user operation and generates mixed-down acoustic data that is mixed down to a smaller number of tracks.
A template storage means for storing a mixdown design template including design information relating to a mixdown process associated with each track and each music group corresponding to a music component;
A music group setting means for setting the music group in the acoustic data to be applied as a template; and the mixdown design template is applied to the acoustic data by associating the tracks and the music groups with each other. A music mixing apparatus comprising template application means for generating down sound data. In the mixdown process based on the user operation, processing process information generating means for generating mixdown process process information for the case acoustic data composed of a plurality of tracks,
Based on the case acoustic data and the mixdown process information, music group extracting means for extracting the music group in the case acoustic data, and at least each track and each music group from the mixdown process information Further comprising: template generation means for extracting design information related to the mixdown process in the above and generating the mixdown design template in association with each track and each music group;
The music mixing apparatus according to claim 1, wherein the mixdown design template generated by the template generation unit is stored in the template storage unit. The music mixing apparatus according to claim 1, wherein the design information related to the mixdown process includes at least information about an order of plug-ins to be used and parameters, volume, and pan of the plug-ins. Track association means for associating the tracks of the acoustic data and the tracks of the mixdown design template with their musical functions;
4. The music mixing apparatus according to claim 1, wherein the template application unit associates the tracks associated by the track association unit. The music group extracting means includes
First candidate extraction means for extracting a music group candidate by performing a correlation analysis process on each track of the case acoustic data;
Second candidate extraction means for extracting the music group candidates based on the change in the mixdown process information, and at least each of the music group candidates extracted by the first candidate extraction means and the second candidate extraction means. The music mixing apparatus according to claim 2, further comprising a music group determination unit that determines a final music group when combined. A template acquisition means for acquiring the mixdown design template from another device;
The music mixing apparatus according to claim 1, wherein the mixdown design template acquired by the template acquisition unit is stored in the template storage unit. Acoustic data comprising a plurality of tracks to be mixed down, comprising template storage means for storing a mixdown design template including design information relating to a mixdown process associated with at least each track and each music group corresponding to a music component Is a music mixing method in a music mixing apparatus that generates mixdown sound data with a smaller number of tracks.
(a) setting a music group in the acoustic data in accordance with a user operation;
(b) reading the mixdown design template selected by a user operation from the template storage means; and
(c) generating the mixdown sound data by associating the tracks and the music groups and applying the mixdown design template read in step (b) to the sound data. A music mixing method comprising: Acoustic data comprising a plurality of tracks to be mixed down, comprising template storage means for storing a mixdown design template including design information relating to a mixdown process associated with at least each track and each music group corresponding to a music component Is a music mixing program executed in a music mixing apparatus that generates mixdown sound data with a smaller number of tracks.
In the computer of the music mixing device,
A music group acquisition step of acquiring music group setting information in the acoustic data in response to a user operation;
A template reading step of reading out the mixdown design template selected by a user operation from the template storage means, and applying the mixdown design template to the acoustic data in association with the tracks and the music groups. A music mixing program for executing a template applying step for generating the mixdown sound data. In a music mixing apparatus that includes a template storage unit that stores a mixdown design template including design information related to mixdown processing, and that processes acoustic data including a plurality of tracks to generate mixdown acoustic data with a smaller number of tracks. A method for extracting the mixdown design template,
(a) In a mixdown process based on a user operation, a step of generating mixdown process process information for case acoustic data composed of a plurality of target tracks;
(b) extracting a music group in the case acoustic data based on the case acoustic data and the mixdown process information;
(c) Extracting at least the design information related to the mixdown process in each track and each music group from the mixdown process information, and associating the mixdown design template with each track and each music group. Generating step, and
(d) A template extraction method comprising a step of storing the mixdown design template generated in the step (c) in the template storage means. In a music mixing apparatus comprising template storage means for storing a mixdown design template including design information relating to mixdown processing, processing acoustic data consisting of a plurality of tracks, and generating mixdown acoustic data having a smaller number of tracks. A program for extracting a mixdown design template,
In the computer of the music mixing device,
In a mixdown process based on a user operation, a process information generation step for generating mixdown process information for case acoustic data composed of a plurality of tracks as targets,
A music group extraction step for extracting a music group in the case acoustic data based on the case acoustic data and the mixdown process information.
A template for extracting design information on mixdown processing in each of each track and each music group from the mixdown process information and generating the mixdown design template in association with each track and each music group A template extraction program for executing a generation step and a storage step of storing the mixdown design template generated in the template generation step in the template storage means.

Images