WO2021106694A1 - Dispositif de traitement d'informations, procédé de traitement d'informations et programme de traitement d'informations - Google Patents

Dispositif de traitement d'informations, procédé de traitement d'informations et programme de traitement d'informations Download PDF

Info

Publication number
WO2021106694A1
WO2021106694A1 PCT/JP2020/042873 JP2020042873W WO2021106694A1 WO 2021106694 A1 WO2021106694 A1 WO 2021106694A1 JP 2020042873 W JP2020042873 W JP 2020042873W WO 2021106694 A1 WO2021106694 A1 WO 2021106694A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
music
unit
information processing
feature
Prior art date
Application number
PCT/JP2020/042873
Other languages
English (en)
Japanese (ja)
Inventor
治彦 岸
Original Assignee
ソニーグループ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニーグループ株式会社 filed Critical ソニーグループ株式会社
Priority to US17/756,108 priority Critical patent/US20220406283A1/en
Priority to JP2021561336A priority patent/JPWO2021106694A1/ja
Priority to CN202080079390.XA priority patent/CN114730551A/zh
Priority to EP20894684.8A priority patent/EP4068272A4/fr
Publication of WO2021106694A1 publication Critical patent/WO2021106694A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/36—Accompaniment arrangements
    • G10H1/38—Chord
    • G10H1/383—Chord detection and/or recognition, e.g. for correction, or automatic bass generation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0008—Associated control or indicating means
    • G10H1/0025—Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/571—Chords; Chord sequences
    • G10H2210/576—Chord progression
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2220/00—Input/output interfacing specifically adapted for electrophonic musical tools or instruments
    • G10H2220/005—Non-interactive screen display of musical or status data
    • G10H2220/011—Lyrics displays, e.g. for karaoke applications
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/121—Musical libraries, i.e. musical databases indexed by musical parameters, wavetables, indexing schemes using musical parameters, musical rule bases or knowledge bases, e.g. for automatic composing methods
    • G10H2240/131—Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/121—Musical libraries, i.e. musical databases indexed by musical parameters, wavetables, indexing schemes using musical parameters, musical rule bases or knowledge bases, e.g. for automatic composing methods
    • G10H2240/131—Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
    • G10H2240/141—Library retrieval matching, i.e. any of the steps of matching an inputted segment or phrase with musical database contents, e.g. query by humming, singing or playing; the steps may include, e.g. musical analysis of the input, musical feature extraction, query formulation, or details of the retrieval process
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/171—Transmission of musical instrument data, control or status information; Transmission, remote access or control of music data for electrophonic musical instruments
    • G10H2240/181—Billing, i.e. purchasing of data contents for use with electrophonic musical instruments; Protocols therefor; Management of transmission or connection time therefor
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
    • G10H2250/311—Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation

Definitions

  • This disclosure relates to an information processing device, an information processing method, and an information processing program.
  • the music information proposed (generated) by AI can be used in the composition work, the user can compose based on a wider variety of viewpoints.
  • the automatic composition function by AI is set for general users, and general users can receive automatically created music information simply by setting an image such as bright or dark.
  • the creator of a song often sets the characteristics of the song, such as chord progression and bass progression, in the process of creating the song. There was a request to receive music information tailored to the characteristics.
  • the information processing apparatus of one form according to the present disclosure includes an acquisition unit that acquires music information and an extraction unit that extracts a plurality of types of feature quantities from the music information acquired by the acquisition unit. And a generation unit that generates information in which a plurality of types of feature quantities extracted by the extraction unit are associated with predetermined identification information as music feature information used as learning data in a composition process using machine learning. Has.
  • the configuration of the copyrighted work management device will be described. It is a figure which shows an example of the work music information storage part which concerns on embodiment. It is a figure which shows an example of the music storage part which concerns on embodiment. It is a figure which shows an example of the feature information storage part which concerns on embodiment. It is a figure explaining the structure of the whole information which concerns on embodiment. It is a figure which shows an example of the time-series information which comprises music feature information which concerns on embodiment. It is a figure which shows the structural example of the information processing apparatus which concerns on embodiment. It is a figure which shows an example of the user information storage part which concerns on embodiment. It is a figure which shows an example of the style information storage part which concerns on embodiment.
  • Embodiment 1-1 An example of information processing according to an embodiment 1-2.
  • Configuration of Information Processing System According to Embodiment 1-3.
  • Configuration of copyrighted work management device According to the embodiment 1-4.
  • Configuration of Information Processing Device According to Embodiment 1-5.
  • Information processing procedure according to the embodiment 1-6-1 Generation processing of style information of written music information 1-6-2.
  • Style information update process 1-7 Conceptual diagram of the configuration of the information processing system 1-7-1.
  • About the written music management server device 1-7-3.
  • About the music creator application section 2. Effect of this embodiment 3.
  • FIG. 1 is a conceptual diagram showing a flow of information processing according to an embodiment.
  • the information processing according to the embodiment is executed by the copyrighted work management device 100, the information processing device 200, and the user terminal 300.
  • the information processing device 200 is an information processing device that provides a service (also simply referred to as “service”) related to the creation of content (information) as a literary work.
  • a service also simply referred to as “service”
  • music music content
  • the content is not limited to music, and may be various content such as video content such as movies and text content such as books (novels, etc.).
  • Good the music referred to here is not limited to one completed song (whole), but various music information such as some sound sources constituting one song (music) and short sounds used for sampling are included. It is a concept that is included.
  • the information processing device 200 communicates with the copyrighted work management device 100 that manages the copyrighted music information using the private network N2 (see FIG. 9).
  • the private network N2 is, for example, a secure VPN (Virtual Private Network) or Internet communication with ID address authentication.
  • the information processing device 200 communicates with the user terminal 300 of the user who uses the service provided by the information processing device 200 using a network N1 (see FIG. 9) such as the Internet.
  • the number of user terminals 300 is not limited to that shown in FIG.
  • the copyrighted work management device 100 is a server device that registers and manages copyrighted music information.
  • the copyrighted work management device 100 periodically registers the copyrighted music information.
  • the copyrighted work management device 100 extracts a plurality of types of feature amounts from the registered music piece information, and transmits the extracted feature amounts to the information processing device 200 via the private network N2.
  • the user terminal 300 is an information processing terminal such as a PC (personal computer) or a tablet terminal.
  • Various program applications are installed on the user terminal 300.
  • An application related to music creation is installed in the user terminal 300.
  • the user terminal 300 has an automatic composition function by AI added by a plug-in (extended application) to an application such as a DAW that realizes a comprehensive music production environment.
  • the plug-in can take the form of VST (Steinberg's Virtual Studio Technology) (registered trademark), Audio Units, AAX (Avid Audio eXtension), and the like.
  • the user terminal 300 is not limited to the DAW, and for example, a mobile application such as iOS may be used.
  • the user terminal 300 activates and executes the automatic composition function by DAW and AI, and communicates with the information processing device 200 to receive the music information composed by the information processing device 200.
  • the user of the user terminal 300 receives music information through an administrator who operates and manages the entire system, a composer who creates music, an arranger, a creator such as a studio engineer, and an automatic composition function. One of the users. In this embodiment, it is assumed that the user terminal 300 is used by the creator Uc.
  • the information processing device 200 is a server device that executes information processing related to the automatic composition function by the AI of the user terminal 300.
  • the information processing device 200 is a so-called cloud server, executes automatic composition by AI according to instruction information by the user terminal 300, and provides the generated music information to the user terminal 300.
  • the information processing device 200 generates a composition model for music generation by performing machine learning.
  • the information processing device 200 provides the user terminal 300 with music information automatically composed by using a Markov model or the like.
  • the information processing device 200 uses style information (musical piece feature information) as learning data for the composition model.
  • style information is information obtained by associating a plurality of types of features such as chord progression, melody, and bass progression with predetermined identification information extracted from music information as feature quantities of a plurality of types, and is machine learning. It is used in the composition process using.
  • the information processing device 200 obtains a plurality of types of feature amounts from the written music information or the music information created by the creator, and collects each feature amount for each music information into a style information ID (predetermined identification information). By adding, multiple style information is generated and created in a database.
  • FIG. 2 is a diagram showing an example of a data structure of style information according to an embodiment.
  • the style information includes style information ID 710, which is identification information of style information, style palette sequence information 720 (music order information), style palette information 730 (music format information), score information 740, and lyrics information 750.
  • the score information 740 includes a plurality of types of features extracted from the music.
  • the score information 740 has a score ID, a melody information, a chord progression information, a base information, and a drum information.
  • the score ID is identification information of the score information.
  • Melody information is a melody in a bar of a specified length.
  • the chord progression information is information indicating the chord progression in a bar of a specified length.
  • Bass information is information indicating the progress of a bass note in a bar of a specified length.
  • Drum information is information indicating the progress (drum pattern or tempo) of a drum sound in a bar of a specified length.
  • the lyrics information 750 includes the lyrics ID and the lyrics information.
  • the lyrics ID is identification information of the lyrics information.
  • Lyrics information is information indicating lyrics in a bar of a specified length.
  • the lyrics information is, for example, a phrase or a character keyword that is the basis of the lyrics.
  • the information processing device 200 can also use self-action words by using a plurality of lyrics information 750 of the style information 700.
  • the style palette information 730 is information registered by associating the score ID of the score information 740 and the lyrics ID of the lyrics information 750 for the same measure with the style palette ID which is the identification information of the style palette information.
  • the style palette sequence information 720 is information indicating the order of the style palette information 730.
  • the style palette sequence information 720 includes a style palette ID uniquely indicating the style palette information 730 and a bar index as one set, and by having a plurality of these sets, information for managing the order of the style palette information 730 in the music. Become. For example, in the case of the example shown in FIG. 2, it is defined that measures 1 to 4 of the music correspond to the style palette ID731a, measures 5 to 8 correspond to the style palette ID731b, and measures xy to y correspond to the style palette ID731z. Will be done.
  • the information processing device 200 performs machine learning using this style information 700 as learning data, and performs composition processing. Therefore, the information processing device 200 does not learn the music information itself, but learns the style information composed of a plurality of types of feature quantities such as chord progression, melody, and bass progression extracted from the music information. That is, since the information processing device 200 learns a plurality of feature quantities extracted in advance from the music information, the load of information processing is smaller than that of learning the music information itself, and the music information to the user is also efficient. Can be provided to.
  • 3 to 7 are views showing an example of a display screen of the user terminal 300 according to the embodiment.
  • the window 370 shown in FIG. 3 is displayed on the user terminal 300.
  • the window 370 has a composition parameter setting unit 371, a style information display unit 372, a composition control unit 373, and a production music display editing unit 374.
  • the composition parameter setting unit 371 is an area in which parameters such as note length and complexity can be set.
  • the style information display unit 372 is an area in which style information used for composition can be selected by keyword input or pull-down selection.
  • the composition control unit 373 is an area where composition instructions can be given by selecting a composition execution instruction button.
  • the production music display editorial unit 374 is an area in which a plurality of piano rolls on which a melody or lyrics are displayed are displayed.
  • Chord progression candidates may be displayed in any order, such as alphabetical order, order of frequency of use by creators, order of frequency of use by all users, order of generation of style information, and so on.
  • chord progression all of the style information included in the information processing apparatus 200 may be displayed, or only a part of the style information may be displayed. If there are many chord progression candidates, the display area can be selected on the pager.
  • the information processing device 200 extracts the style information including the chord progression, and selects the chord progression information of each extracted style information in the style palette. The list may be displayed in the pull-down 372a.
  • the creator selects a desired chord progression from the chord progressions presented in the style palette selection pull-down 372a, and selects a composition execution instruction button.
  • the information processing apparatus 200 extracts the style information having the selected chord progression, performs machine learning using the extracted style information 700 as learning data, and performs composition processing.
  • the information processing device 200 provides music information to the user terminal 300.
  • the creator can receive the music information generated according to the chord progression only by selecting the desired chord progression from the chord progressions presented in the style palette selection pull-down 372a.
  • the style information 700 includes lyrics information as a feature amount
  • the creator can receive the presentation of style information matching the lyrics simply by inputting the desired lyrics. Specifically, as shown in FIG. 5, when the creator inputs the desired lyrics in the search keyword input field 372b, the information processing apparatus 200 extracts the lyrics or style information having lyrics similar to the lyrics. Then, the lyrics information of each of the extracted style information may be displayed in a list in the style palette selection pull-down 372a.
  • the creator selects the desired lyrics information from the lyrics information presented in the style palette selection pull-down 372a, and selects the composition execution instruction button.
  • the information processing device 200 extracts the style information having the selected lyrics information, performs machine learning using the extracted style information 700 as learning data, and performs composition processing to the user terminal 300. Provide music information.
  • the creator can receive the music information generated according to the lyrics only by selecting the desired lyrics from the lyrics presented in the style palette selection pull-down 372a.
  • the information processing device 200 may automatically generate lyrics according to the generated music and provide the user terminal 300 with music information in which the melody and the lyrics are associated with each other.
  • the melody and the lyrics corresponding to the melody are displayed on the melody display piano roll 374a of FIG.
  • the user terminal 300 displays a list of chord progression candidates of the style information presented by the information processing device 200, and creates the music of the creator. May be assisted.
  • the information processing device 200 generates style information having a plurality of types of feature quantities of the music information as a data set for learning the composition model, and trains the composition model.
  • the information processing device 200 provides the producer with music information composed according to the characteristics of the music.
  • the copyrighted work management device 100 extracts a plurality of types of feature quantities from the newly registered music music information (step S11). Step S12). At this time, the copyrighted work management device 100 extracts the feature amount by using, for example, a 12-tone analysis technique. Then, the information processing device 200 acquires a plurality of types of feature amounts from the copyrighted work management device 100 (step S13), summarizes each feature amount for each piece of music information, and assigns a style information ID to each of the feature amounts. The style information corresponding to the music information is generated (step S14).
  • the information processing device 200 when a new musical piece is created by inputting a feature amount such as a chord progression, a melody, or a bass progression by an operation by the creator Uc of the user terminal 300, the information processing device 200 includes each feature amount. Acquire music information (step S21). Then, the information processing device 200 extracts features such as chord progression, melody, and bass progression from the acquired music information (step S22). Then, the information processing device 200 collects each feature amount for each music information and assigns a style information ID to generate style information corresponding to each music information (step S23). The information processing device 200 generates a plurality of style information 700 and creates a database by performing the process shown in FIG.
  • FIG. 8 is a conceptual diagram showing the flow of information processing according to the embodiment.
  • the information processing apparatus 200 when the style information is selected by the creator Uc during the music creation by the creator Uc (step S31), the information processing apparatus 200 performs the composition processing by learning the style information.
  • the music information is output to the user terminal 300 (step S32).
  • the user Uc further performs based on the provided music information.
  • the information processing device 200 receives the performance information (step S33)
  • the information processing device 200 adds the performance information to the style information used for composition and updates the style information (step S34).
  • the style information used is updated according to the music creation of the creator Uc.
  • the information processing apparatus 200 can bring this style information closer to the style of music creation by the creator Uc, and can compose and provide the music information that matches the style of the creator Uc.
  • FIG. 9 is a diagram showing an example of the information processing system 1 according to the embodiment.
  • the information processing system 1 includes user terminals 300-1 to 300-3, an information processing device 200, and a copyrighted work management device 100.
  • the information processing system 1 functions as a copyrighted work management system and a learning model information management system.
  • three user terminals 300-1 to 300-3 are illustrated, but when the description is made without particular distinction, the user terminal 300 is described.
  • the information processing device 200 and the user terminal 300 are connected to each other via a network N1 so as to be able to communicate by wire or wirelessly. Further, the information processing device 200 and the copyrighted work management device 100 are connected to each other via a private network N2 so as to be communicable by wire or wirelessly.
  • the copyrighted work management device 100 manages the copyrighted music information.
  • the copyrighted music management device 100 periodically registers the copyrighted music information.
  • the copyrighted work management device 100 extracts a plurality of types of feature amounts from the registered music piece information, and transmits the extracted feature amounts to the information processing device 200.
  • the user terminal 300 transmits the music information created by the creator to the information processing device 200, and receives the music information composed by the information processing device 200 when the self-operated music function is activated.
  • the information processing device 200 generates a composition model by generating style information, which is learning data, from the written music information or the music information created by the creator, and performing machine learning.
  • the information processing device 200 provides the user terminal 300 with music information that has been automatically composed using a generative model.
  • FIG. 10 is a diagram showing a configuration example of the copyrighted work management device 100 according to the embodiment.
  • the copyrighted work management device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.
  • the communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like.
  • the communication unit 110 is connected to the private network N2 by wire or wirelessly, and transmits / receives information to / from the information processing device 200 via the private network N2.
  • the storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.
  • the storage unit 120 stores various data used for information processing.
  • the storage unit 120 includes a copyrighted music information storage unit 121, a music storage unit 122, and a feature information storage unit 123.
  • the copyrighted music information storage unit 121 stores information about the copyrighted music that is a copyrighted work written in the past.
  • FIG. 11 is a diagram showing an example of the written music information storage unit 121 according to the embodiment.
  • the copyrighted music information storage unit 121 stores the copyrighted music ID, the copyrighted music meta information, and the MP3 file name.
  • the copyrighted music ID is identification information for uniquely identifying the copyrighted music of, for example, the Beatles, Michael Jackson, and the like.
  • the copyrighted music meta information is additional information about the copyrighted music, such as the title, artist name, album name, release date, genre, mood, and lyrics of the copyrighted music.
  • the MP3 file name is a file name that identifies the MP3 file of the written music.
  • the music storage unit 122 stores the music information of the copyrighted music.
  • FIG. 12 is a diagram showing an example of the music storage unit 122 according to the embodiment. As shown in FIG. 12, the music storage unit 122 stores MP3 files of each copyrighted music. This MP3 file can be specified by the MP3 file name stored in the work music information storage unit 121.
  • the feature information storage unit 123 stores the feature amounts of a plurality of types of the written music.
  • FIG. 13 is a diagram showing an example of the feature information storage unit 123 according to the embodiment. As shown in FIG. 13, the feature information storage unit 123 stores the copyrighted music feature amount in association with the copyrighted music ID.
  • the copyrighted music feature information includes overall information indicating the characteristics of the entire copyrighted music and time-series information indicating the characteristics that change in time series.
  • FIG. 14 is a diagram showing an example of overall information according to the embodiment.
  • the overall information includes music type information such as tempo information, time signature information, genre and mood, for example.
  • the music type information is, for example, information such as an entrusted channel of the 12-tone analysis technology.
  • FIG. 15 is a diagram showing an example of time-series information constituting the written music feature information according to the embodiment.
  • the time-series information includes, for example, beat information, chord progression information, music structure information, and melody information.
  • the beat information includes time position information of each beat and time position information of each bar, such as a metronome.
  • the chord progression information is information on the time-series progression of accompaniment chords according to the flow of a musical piece, for example, information indicating the progression of a chord such as "CF-Am-G7".
  • the music structure information is information such as "A melody-B melody-sabi-A melody-sabi" which is the structure of the entire music.
  • the melody information includes, for example, melody line information inferred from the audio information of the music.
  • control unit 130 for example, a program stored inside the user terminal 300 is executed by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like using a RAM (Random Access Memory) or the like as a work area. It will be realized. Further, the control unit 130 is a controller, and may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).
  • ASIC Application Specific Integrated Circuit
  • FPGA Field Programmable Gate Array
  • the control unit 130 has a management unit 131, an extraction unit 132, and a transmission unit 133, and realizes or executes the functions and operations of information processing described below.
  • the management unit 131 manages various information related to the copyrighted work management device 100. For example, the management unit 131 stores various information in the storage unit 120, and updates the stored information as appropriate. Specifically, the management unit 131 stores the new work music in the work music information storage unit 121, and updates the information about the new work music.
  • the extraction unit 132 extracts a plurality of types of feature quantities from the written music information.
  • the extraction unit 132 acquires the copyrighted music information that is periodically registered at a preset timing, and uses the chord progression information, the beat information, the melody information, and the drum information as feature quantities from the acquired copyrighted music information. Extract.
  • the extraction unit 132 extracts the music feature amount of the MP3 file of each work music for each newly registered work music, and obtains the feature information.
  • the extraction unit 132 extracts various feature quantities from the music information by using, for example, a 12-sound analysis technique, and registers them in the feature information storage unit 123 as the copyrighted music feature information.
  • the extraction unit 132 performs feature quantity extraction processing on the written music information by receiving an instruction to extract the written music from the administrator terminal (not shown) used by the system administrator via the information processing device 200. Alternatively, the extraction unit 132 performs a feature amount extraction process for the copyrighted music information by receiving an instruction to extract the copyrighted music from the information processing device 200.
  • the transmission unit 133 transmits the feature amounts of the plurality of types of copyrighted music information extracted by the extraction unit 132 to the information processing device 200 together with information related to the copyrighted music information such as music structure information and music music meta information. ..
  • FIG. 16 is a diagram showing a configuration example of the information processing device 200 according to the embodiment.
  • the information processing device 200 includes a communication unit 210, a storage unit 220, and a control unit 230.
  • the information processing device 200 includes an input unit (for example, a keyboard, a mouse, etc.) that receives various operations from an administrator or the like that manages the information processing device 200, and a display unit (for example, a liquid crystal display) for displaying various information. ) May have.
  • the communication unit 210 is realized by, for example, a NIC or the like.
  • the communication unit 210 is connected to the network N1 and the private network N2 by wire or wirelessly, and transmits / receives information to / from the user terminal 300, the copyright management device 100, etc. via the network N or the private network N2.
  • the storage unit 220 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.
  • the storage unit 220 stores various data used for information processing.
  • the storage unit 220 includes a user information storage unit 221, a style information storage unit 222, an possession information storage unit 223, a production information storage unit 224, and an operation history information storage unit 225.
  • the user information storage unit 221 stores various information (user information) related to the user.
  • FIG. 17 is a diagram showing an example of the user information storage unit 221 according to the embodiment.
  • User information including user ID, user meta information, and authority information is stored in the user information storage unit 221.
  • the user information storage unit 221 stores user meta information and authority information corresponding to each user ID in association with each user ID.
  • the user ID indicates identification information for uniquely identifying the user.
  • the user ID indicates identification information for uniquely identifying a user such as a creator, a general user, or a system administrator.
  • the user meta information is additional information of the user such as the name and address of the user.
  • the authority information for example, values for identifying the authority such as system administrator authority information, creator authority information, and general user authority information are stored.
  • the user information storage unit 221 is not limited to the above, and may store various information depending on the purpose.
  • Various information about the user may be stored in the user meta information. For example, when the user is a natural person, the user meta information may store demographic attribute information such as the gender and age of the user, psychographic attribute information, and the like.
  • the style information storage unit 222 stores information about the composition model.
  • FIG. 18 is a diagram showing an example of the style information storage unit 222 according to the embodiment.
  • the style information storage unit 222 stores learning model information including model information ID, creator ID, model information meta information, style information 700, copyrighted work ID, and shareability information.
  • the style information storage unit 222 stores the creator ID, model information meta information, style information, copyrighted work ID, and shareability information corresponding to each model information ID in association with each model information ID.
  • the model information ID indicates identification information for uniquely identifying the composition model information.
  • the creator ID indicates identification information for uniquely identifying the creator of the corresponding composition model information.
  • the creator ID indicates identification information for uniquely identifying a user such as a system administrator, a creator, or a general user.
  • Model information meta information is, for example, information representing the characteristics of a copyrighted work to be learned.
  • Learning model information Meta information is information such as the tempo of a song, the genre, the atmosphere such as light and darkness, the structure of a song such as verse B verse, chord progression, scale, and church mode.
  • the style information 700 is learning data of the composition model generated by the generation unit 233 (described later) included in the information processing device 200.
  • the style information is information extracted from music information in which a plurality of types of features such as chord progression, melody, and bass progression are associated with predetermined identification information.
  • the sharing availability information indicates, for example, the sharing availability of the corresponding learning model.
  • As the shareability information for example, a value for identifying whether or not the corresponding learning model can be shared is stored.
  • the style information storage unit 222 is not limited to the above, and may store various information depending on the purpose.
  • the composition model information meta information may store various additional information regarding the composition model, such as information regarding the date and time when the composition model was created.
  • the possession information storage unit 223 stores various information related to the style information selected by the creator of the music when the music is created.
  • FIG. 19 is a diagram showing an example of the possession information storage unit 223 according to the embodiment.
  • the user ID of the creator who creates the music and the style information ID selected by the creator are stored in association with each other.
  • the production information storage unit 224 stores various information related to the produced music.
  • FIG. 20 is a diagram showing an example of the production information storage unit 224 according to the embodiment. As shown in FIG. 20, the production information storage unit 224 stores the user ID of the creator who created the music and the score ID created by the creator in association with each other.
  • the operation history information storage unit 225 stores the operation history information by the creator for the user terminal 300.
  • the operation history information storage unit 225 stores the operation history corresponding to each user ID in association with each user ID.
  • the operation history information indicates the operation history of the creator.
  • the operation history information may include various information related to the operation of the creator, such as the content of the operation performed by the creator and the date and time when the operation was performed.
  • the operations include selection of style information presented by the information processing device 200, selection of a composition execution instruction button, reproduction and editing of music information received from the information processing device 200, and the like.
  • the control unit 230 is realized by, for example, a CPU, an MPU, or the like executing a program stored inside the information processing apparatus 200 (for example, an information processing program according to the present embodiment) with a RAM or the like as a work area. .. Further, the control unit 130 is a controller, and may be realized by an integrated circuit such as an ASIC or FPGA.
  • the control unit 230 includes an acquisition unit 231, an extraction unit 232, a generation unit 233, a reception unit 234, a selection unit 235, a transmission unit 236, a composition unit 237, and an update unit 238. To realize or execute.
  • the acquisition unit 231 acquires music information.
  • the acquisition unit 231 acquires the music information created by the creator by communicating with the user terminal 300 via the network N1.
  • This music information is created by the creator using an application related to music creation installed in the user terminal 300, that is, an automatic composition function, and music such as chord progression, melody, bass progression, drum sound progression, etc. Includes features related to.
  • the acquisition unit 231 communicates with the copyrighted work management device 100 via the private network N2 to obtain a feature amount of a plurality of types of copyrighted music information extracted by the extraction unit 132 of the copyrighted work management device 100. Receive with information about copyrighted music information. That is, the acquisition unit 231 receives the copyrighted music feature information for each of the newly registered copyrighted music from the copyrighted work management device 100 via the private network N2.
  • the extraction unit 232 extracts a plurality of types of features from the music information.
  • the extraction unit 132 extracts chord progression information, beat information, melody information, and drum information as feature quantities from the music information created by the creator.
  • the extraction unit 232 extracts various feature quantities from the music information by using, for example, a 12-tone analysis technique.
  • the generation unit 233 generates style information in which the feature quantities of a plurality of types extracted by the extraction unit 232 and the style information ID are associated with each other as learning data in the composition process.
  • the generation unit 233 registers the music information created by the creator in the production information storage unit 224 in association with the style information ID of the style information 700 and the user ID of the creator.
  • the generation unit 233 may associate the style ID of the style information 700 with the copyrighted music ID.
  • the reception unit 234 receives various information transmitted from the user terminal 300.
  • the reception unit 234 receives information on the creator who uses the automatic composition function on the user terminal 300 and information on the style information selected by the creator.
  • the reception unit 234 can also accept registration of music associated with the style information 700, editing of the style information, and the like.
  • the selection unit 235 selects all or part of the style information when the automatic composition function is activated on the user terminal 300.
  • the transmission unit 236 transmits the presentation information of the style information selected by the selection unit 235 to the user terminal 300.
  • the chord progression or lyrics information of each style information is listed as a candidate in the style palette selection pull-down 372a of the user terminal 300.
  • the selection unit 235 selects the selected style information from the style information storage unit 222.
  • the composition unit 237 composes music information using machine learning based on the style information selected by the selection unit 235, and transmits the composed music information to the user terminal 300.
  • the composition unit 237 may compose music using various existing music generation algorithms. For example, the composition unit 237 may use a music generation algorithm using a Markov chain, or may use a music generation algorithm using deep learning. Further, the composition unit 237 may generate a plurality of music information for the instruction information transmitted from the user terminal 300. As a result, the producer can receive a plurality of proposals from the composition unit 237, and thus can proceed with the composition work by using more diverse information.
  • the update unit 238 When the update unit 238 receives the performance information based on the music information composed by the composition unit 237 from the user terminal 300, the update unit 238 adds the performance information to the selected style information and updates the selected style information.
  • FIG. 21 is a diagram showing a configuration example of the user terminal 300 according to the embodiment.
  • the user terminal 300 includes a communication unit 310, an input unit 320, an output unit 330, a storage unit 340, a control unit 350, and a display unit 360.
  • the communication unit 310 is realized by, for example, a NIC or a communication circuit.
  • the communication unit 310 is connected to the network N1 by wire or wirelessly, and transmits / receives information to / from the information processing device 200 and other devices such as other terminal devices via the network N1.
  • the input unit 320 has a keyboard and a mouse connected to the user terminal 3000.
  • the input unit 320 accepts input by the user.
  • the input unit 320 accepts user input using a keyboard or mouse.
  • the input unit 320 may have a function of detecting voice.
  • the input unit 320 may include a microphone that detects voice.
  • the input unit 320 may have a touch panel capable of realizing a function equivalent to that of a keyboard or a mouse.
  • the input unit 12 receives various operations from the user via the display screen by the function of the touch panel realized by various sensors.
  • the detection method of the user's operation by the input unit 320 the capacitance method is mainly adopted in the tablet terminal, but other detection methods such as the resistance film method, the surface acoustic wave method, the infrared method, and the electromagnetic wave are used. Any method such as a guidance method may be adopted as long as the user's operation can be detected and the touch panel function can be realized.
  • the user terminal 300 may have an input unit that also accepts operations by buttons or the like.
  • the output unit 330 outputs various information.
  • the output unit 330 has a speaker that outputs sound.
  • the storage unit 340 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.
  • the storage unit 340 stores various information used for displaying the information.
  • the control unit 350 is realized by, for example, a CPU, an MPU, or the like executing a program stored inside the user terminal 300 with a RAM or the like as a work area. Further, the control unit 350 is a controller, and may be realized by an integrated circuit such as an ASIC or FPGA.
  • the control unit 350 includes a display control unit 351, a registration unit 352, a transmission / reception unit 353, a selection unit 354, a reproduction unit 355, and a performance reception unit 356.
  • the display control unit 351 controls various displays on the display unit 360.
  • the display control unit 351 controls the display of the display unit 360.
  • the display control unit 351 controls the display of the display unit 360 based on the information received from the information processing device 200.
  • the display control unit 351 controls the display of the display unit 360 based on the information generated by the processing by each component of the control unit 350.
  • the display control unit 351 may control the display of the display unit 360 by an application that displays an image.
  • the display control unit 351 displays the window 370 (see FIGS. 3 to 7) and the like on the display unit 360 by the application of the automatic composition function by DAW and AI.
  • the display control unit 351 receives the presentation information of the style information from the information processing device 200, the display control unit 351 displays the chord progression and lyrics of the presented style information in the style palette selection pull-down 372a (see FIG. 4) of the window 370. ..
  • the display control unit 351 receives the music information and the lyrics information generated by the information processing device 200, the display control unit 351 displays the melody and the lyrics corresponding to the melody on the melody display piano roll 374a (see FIG. 7) of the window 370. Let me.
  • Registration unit 352 accepts registration of various information.
  • the registration unit 352 accepts the registration of the drum pattern, chord progression, and melody set by the user when the DAW is activated.
  • the drum pattern, chord progression and melody are registered via an application that displays images IM11, IM21, IM31, IM41 (FIGS. 26 and 27).
  • the transmission / reception unit 353 communicates with the information processing device 200 to transmit and receive various information.
  • the transmission / reception unit 353 transmits music information including the drum pattern, chord progression, and melody received by the registration unit 352 to the information processing device 200. Further, when the automatic composition function is activated, the transmission / reception unit 353 receives the presentation information of the style information transmitted from the information processing device 200.
  • the transmission / reception unit 353 transmits instruction information instructing the selection of style information to the information processing device 200. Then, the transmission / reception unit 353 receives the music information and the lyrics information generated by the information processing device 200. Further, the transmission / reception unit 353 transmits the performance information regarding the performance received by the user terminal 300 to the information processing device 200.
  • the selection unit 354 selects any of the style information presented by the information processing device 200. For example, among the chord progressions displayed in the style palette selection pull-down 372a (see FIG. 4) of the window 370, one of the chord progressions is selected by the operation of the input unit 320 by the user. As a result, the selection unit 354 transmits the selection instruction information of the style information corresponding to the selected chord progression from the transmission / reception unit 353 to the information processing device 200.
  • the playback unit 355 reproduces the music information generated by the information processing device 200. Specifically, the reproduction unit 255 sets arbitrary musical instrument information for each of the melody, chord, and bass sound included in the music data, and reproduces each data. The reproduction unit 255 may reproduce each of the melody, the chord, and the bass sound in combination.
  • the performance reception unit 356 accepts the performance by the producer when the producer performs the performance together with the composition using the automatic composition function. For example, the performance reception unit 356 receives the performance information to be played in accordance with the reproduction of the music information generated by the information processing device 200 by the automatic composition function.
  • the display unit 360 displays various information.
  • the display unit 360 is realized by, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like.
  • the display unit 360 displays various information according to the control by the display control unit 351.
  • the display unit 360 can also display information such as an image provided by the information processing device 200.
  • FIG. 22 is a sequence diagram showing an information processing procedure according to the embodiment.
  • the copyrighted work management device 100 extracts a plurality of types of feature quantities from each of the newly registered copyrighted music information (step S101). Step S102). At this time, the copyrighted work management device 100 extracts the feature amount by using, for example, a 12-tone analysis technique. Then, the literary work management device 100 transmits the literary music feature information including the plurality of types of feature quantities extracted from the literary music information and the information related to the literary music information to the information processing device 200 via the private network N2. (Step S103).
  • the information processing device 200 automatically creates style information 700 based on the received music feature information (step S104).
  • the information processing device 200 can generate score information 740 from, for example, beat information, chord progression information, and melody information of written music feature information.
  • the information processing device 200 can generate the lyrics information 750 from, for example, the lyrics information of the written music meta information.
  • the information processing device 200 can generate style palette information 730 by bundling those a plurality of score information 740 and lyrics information 750 that have similar chord progressions. Similar chord progressions are, for example, exactly the same chord progression. Alternatively, similar chord progressions may be such that each chord is classified into Tonic (T), Sub-dominat (S), and Dominat (D), and the sequences of T, S, and D are the same. In the case of C major and A minor, T is C / Em / Am, S is F and Dm, and D is G and Dm7-5. Since the chord progressions CDGC and Em-Dm-Bm7-5-Am are both TSD, they can be considered to have the same chord progression. Further, similar chord progressions can be classified based on similar chord progressions by machine learning or deep learning, for example, instead of using music theory.
  • the information processing apparatus 200 may independently register the automatically generated style palette information 730 in the style palette sequence information 720.
  • the information processing apparatus 200 may generate and register the style palette sequence information 720 in which a plurality of style palette information 730s are arranged.
  • the information processing device 200 can arrange them with reference to the music structure information.
  • the information processing device 200 registers the generated style information in association with the identification information of the written music information (step S105), and stores the generated style information in the style information storage unit 222.
  • FIG. 23 is a sequence diagram showing an information processing procedure according to the embodiment.
  • step S111 when the creator operates the user terminal 300 to activate a composition function such as a DAW (step S111) and selects the creation of style information (step S112), the creator composes from the user terminal 300.
  • the composition start information indicating the start is transmitted (step S113).
  • the information processing device 200 receives the composition start information, it transmits the information necessary for composition (step S114).
  • the user terminal 300 registers the drum pattern (for example, the tempo, the number of measures, the High Hat, the Bass Drumus, and the beat position for sounding the Snare) by the operation by the creator according to the UI instruction (step S115).
  • the user terminal 300 registers the chord progression by the operation of the creator (step S116).
  • the composition function application automatically plays the drum and the chord progression, so that the creator inputs the melody to the user terminal 300 a plurality of times in accordance with the automatic performance.
  • the user terminal 300 registers the input melody (step S117).
  • the user may additionally input the lyrics information by using the composition function.
  • the user terminal 300 continues to register the melody until the input of the melody by the user is completed.
  • the user terminal 300 transmits the music information by the creator to the information processing device 200 (step S119).
  • the music information includes features such as drum patterns, chord progressions, and melodies.
  • the information processing device 200 extracts each feature amount included in the music information to generate score information and lyrics information, and generates style palette information from the score information and lyrics information (step S120).
  • the information processing device 200 obtains each style palette sequence information in the music information, and generates the style information 700 by associating the style information ID with the score information, the lyrics information, the style palette information, and the style palette sequence information ( Step S121).
  • the information processing device 200 registers the style information 700 in the style information storage unit 222 (step S122). At the same time, the information processing apparatus 200 registers the data in which the style information ID and the user ID of the creator are associated with each other in the possession information storage unit 223. Then, the information processing device 200 registers the data in which the score ID and the user ID of the creator are associated with the production information storage unit 224, and ends the style information generation process.
  • the style palette sequence information may be generated by the creator arranging a plurality of style palette information using the composition function.
  • a plurality of style information 700s are accumulated in the information processing device 200. Users such as creators can use automatic composition by selecting their favorite style information from these style information.
  • FIG. 24 is a sequence diagram showing an information processing procedure according to the embodiment.
  • the information processing device 200 When the information processing device 200 receives the composition start information (step S132) when the automatic composition function is activated on the user terminal 300 by the creator, the style information is selected (step S133), and the style information is selected. Is transmitted to the user terminal 300 (step S134).
  • the information processing device 200 selects, for example, all style information, style information that the creator uses more than a predetermined number of times, or style information that all users use more than a predetermined number of times from the style information storage unit 222. And send the presentation information of the selected style information.
  • the user terminal 300 displays a list of style information based on the presented information (step S135).
  • the user terminal 300 displays, for example, a list of chord progressions of style information as candidates.
  • the user terminal 300 transmits the selection information indicating the selected style information to the information processing device 200 (step S137).
  • the information processing device 200 extracts selected style information, performs machine learning using the extracted style information as learning data, performs composition processing (step S138), and provides music information to the user terminal 300 (step S138). Step S139).
  • the information processing device 200 extracts the feature amounts of the composed music information by the extraction unit 232, stores new score information including these feature amounts in the storage unit 220, and registers the composition information in the possession information storage unit 223. To do.
  • step S140 When the user terminal 300 plays the provided music (step S140), the user terminal 300 accepts the performance process (step S141).
  • the producer plays, for example, by MIDI keyboard, the performance information becomes MIDI information.
  • the information processing device 200 receives the transmission of the performance information (step S142), the information processing device 200 extracts the feature amount from the performance information (step S143).
  • the information processing device 200 updates the style information by adding the feature amount extracted from the performance information as the score information to the style information selected by the creator (step S144). In this way, since the actual performance by the creator is added to the style information selected by the creator, the music information automatically composed using this style information approaches the style of the creator. In other words, the style information can be brought closer to the style of the creator who played the music.
  • the processes of steps S140 to S142 are repeated until the producer finishes the performance. Then, the score information generated in the repeating process of steps S140 to S142 repeated until the music is completed is stored in the storage unit 220.
  • the music information automatically composed using this style information approaches the style of the creator. .. That is, in the information processing system 1, the style information can be brought closer to the style of the creator who played the music.
  • step S145 when the performance by the producer is completed (step S145: Yes), the producer may operate the user terminal 300 to perform, for example, an arrangement process (step S146), a mix or a mastering process (step S147). ..
  • FIG. 25 is a diagram showing an example of a conceptual diagram of the configuration of the information processing system.
  • FIG. 25 is a schematic diagram showing a functional outline of a system which is an example of application of the information processing system 1.
  • the copyrighted music management server device shown in FIG. 25 corresponds to the copyrighted work management device 100 in the information processing system 1.
  • the server device shown in FIG. 25 corresponds to the information processing device 200 in the information processing system 1.
  • the system administrator application unit shown in FIG. 25 corresponds to an application installed on a terminal used by the system administrator.
  • the creator application unit shown in FIG. 25 corresponds to the application installed on the user terminal 300 in the information processing system 1.
  • the application unit for general users shown in FIG. 25 corresponds to an application installed on a terminal used by a general user.
  • the system administrator application unit, the music creator application unit, and the general user application unit are illustrated one by one, but a plurality of these are included according to the number of corresponding terminals. May be good.
  • the control unit of the copyrighted music management server device shown in FIG. 25 corresponds to the control unit 130 of the copyrighted work management device 100.
  • the copyrighted music management unit of the copyrighted music management server device corresponds to the management unit 131 of the copyrighted music management device 100.
  • the copyrighted music feature information analysis function of the copyrighted music management server device corresponds to the extraction unit 132 of the copyrighted music management device 100.
  • the database unit in the copyrighted music management server of the copyrighted music management server device corresponds to the storage unit 120 of the copyrighted music management device 100.
  • the learning processing unit and control unit of the server device shown in FIG. 25 correspond to the control unit 230 of the information processing device 200.
  • the learning processing unit of the server device corresponds to the composition unit 237 of the information processing device 200.
  • the in-server database unit of the server device corresponds to the storage unit 220 of the information processing device 200.
  • the display operation unit and control unit of the music maker application unit shown in FIG. 25 correspond to the control unit 350 of the user terminal 300.
  • the display operation unit of the music maker application unit corresponds to the display control unit 351 of the user terminal 300.
  • the display operation unit and control unit of the system administrator application unit and the general user application unit correspond to the control unit of the terminal device used by each user.
  • the server device is connected to the system administrator application unit, the music creator application unit, and the general user application unit via a network N1 such as the Internet. Further, the server device is connected to the work music management server device via the private network N2.
  • the copyrighted music management server device has a control unit and a database unit in the copyrighted music management server.
  • the control unit of the music composition management server device has a copyrighted music management function and a copyrighted music feature information analysis function.
  • the server device has a control unit, a learning processing unit, and a database unit in the server.
  • the control unit of the server device has a production music information management function, a style information management function, a user operation history information management function, and a copyrighted music analysis function.
  • the learning processing unit of the server device has a machine learning processing function and a deep learning processing function.
  • the music creator application unit has a display operation unit and a control unit.
  • the display operation unit of the music creator application unit has a production music information display function and a style information display / editing function.
  • the music creator application unit has a style information sharing function and a user operation history information transmission function.
  • the music maker application unit is, for example, music editing software (DAW, etc.), and can display, for example, music information with the produced music information display function. If the DAW has, for example, an AI-assisted music production function, new music information can be produced while using the learning model information display / editing function.
  • the system administrator application unit and the general user application unit have the same configuration, and the user's authority over the system is different.
  • FIG. 26 shows an example of the user interface when the music creation application is displayed on the screen of the user terminal 300.
  • the user interface IF11 displays the music data received by the music creation application.
  • the music data in the music creation application is composed of three different types of data: melody, chord, and bass sound.
  • the user interface IF11 shown in FIG. 26 displays data related to a melody among three types of different data.
  • Setting information ST11 displays information related to the style palette, which is an example of setting information in the automatic composition function.
  • the style palette is designated information for designating style information that is learning data for machine learning.
  • Setting information ST12 displays information related to harmony, which is an example of setting information in the automatic composition function.
  • the information about the harmony is, for example, information for determining the probability that the constituent sounds included in the chord appear in the melody in the music data composed by the information processing apparatus 200. For example, if the user sets the information about harmony to "strict”, the probability that the constituent notes included in the chord will appear in the melody is high in the automatic composition data. On the other hand, when the user sets the information about harmony to "loose", the probability that the constituent notes included in the chord will appear in the melody in the automatic composition data is reduced. In the example of FIG. 26, it is shown that the user has applied the information about harmony more than "strict”.
  • Setting information ST13 displays note length information, which is an example of setting information in the automatic composition function.
  • the note length information is, for example, information for determining the note length in the music data composed by the information processing apparatus 200. For example, when the user sets the note length information to "long", in the music data of the self-operated song, the note with a relatively long sound length (for example, a whole note, a half note, etc.) ) Will appear more likely. On the other hand, when the user sets the note length information to "short”, in the music data of the self-operated song, the note with a relatively short note length (for example, an eighth note or a sixteenth note) Etc.) will appear more likely.
  • Setting information ST14 displays information for determining the type and amount of material music other than the material music included in the designated information (style palette specified by the user), which is an example of the setting information in the automatic composition function.
  • Such information is, for example, information for determining whether or not to strictly perform learning based on the music included in the style palette specified by the user in the music data composed by the information processing device 200. For example, when the user sets such information to "never", there is a low tendency for music other than the music included in the style palette to be used in automatic composition learning. On the other hand, when the user sets such information to "only”, there is a high tendency that songs other than the songs included in the style palette are used in automatic composition learning.
  • the music data MDT1 displays specific music data transmitted from the information processing device 200.
  • the music data MDT1 includes information indicating a chord progression such as Cm, information indicating a pitch and a note length in a bar, a transition of a note pitch (in other words, a melody), and the like.
  • the music data MDT1 may include, for example, four types of different contents. That is, the information processing device 200 may transmit a plurality of music data instead of transmitting only one type of automatic music data. As a result, the user can select his / her favorite music data from the generated candidates for the plurality of music data, or combine the plurality of music data to compose the favorite music.
  • the user interface IF11 shown in FIG. 26 displays data related to the melody among the three different types of data included in the music data, that is, the melody, the chord, and the bass sound. Displayed on the interface. This point will be described with reference to FIG. 27.
  • the user terminal 300 has a user interface IF 12 that displays data related to chords and a user interface IF 13 that displays data related to bass sounds on the screen. It may be displayed in.
  • note information different from the music data MDT1 in the user interface IF11 is displayed on the user interface IF12 and the user interface IF13.
  • the user interface IF12 displays note information (for example, constituent sounds of chord Cm) related to chords corresponding to the melody of the music data.
  • the user interface IF13 displays note information (for example, "C" sound in the case of chord Cm) related to the bass sound corresponding to the melody or chord of the music data.
  • the user can select the information to be copied from the displayed user interface IF11, user interface IF12, and user interface IF13, or edit a part of the bass sound, for example.
  • the information processing apparatus includes an acquisition unit (acquisition unit 231 in the embodiment), an extraction unit (extraction unit 232 in the embodiment), and a generation unit. (In the embodiment, it has a generation unit 233).
  • the acquisition unit acquires music information.
  • the extraction unit extracts a plurality of types of feature quantities from the music information acquired by the acquisition unit.
  • the generation unit uses information that associates a plurality of types of feature quantities extracted by the extraction unit with predetermined identification information as music feature information (in the embodiment) that is used as learning data in a composition process using machine learning. Generate as style information).
  • style information having a plurality of types of feature quantities of music information can be generated as a data set for learning a composition model.
  • the information processing device by learning this style information in the composition model, it is possible to provide each user including the creator with the music information composed according to the characteristics of the music. .. Therefore, according to the information processing device according to the present embodiment, it is possible to improve the convenience of the music creation function by the user.
  • the acquisition unit receives music information including a feature amount related to the music created by the creator using the application related to the creation of the music installed in the terminal device from the terminal device (user terminal 300 in the embodiment). Acquire music information by.
  • the extraction unit extracts a plurality of types of features included in the music information.
  • the generation unit associates the music feature information with the creator's identification information. As a result, the information processing device can generate music feature information related to the music information created by the creator.
  • the features of the music created by the creator are chord progression information indicating chord progression, melody information indicating melody, and base signal indicating bass progression in a bar of a specified length. Therefore, the information processing device can generate the music feature information using the feature amount of the music created by the creator for the music information created by the creator, so that the music feature information can be generated quickly. Can be done.
  • the feature amount of the music created by the creator is the drum progress information indicating the progress of the drum in the bar of the specified length. Therefore, the information processing device can generate music feature information including drum progress information.
  • the acquisition unit acquires the written music information that is regularly registered at a preset timing.
  • the extraction unit extracts a plurality of types of features from the written music information.
  • the generation unit associates the style information with the identification information of the written music information. As a result, the information processing device 200 can automatically generate music feature information related to the copyrighted music information that is regularly registered.
  • the information processing device has a transmission unit (transmission unit 236 in the embodiment) that transmits presentation information of music feature information according to instruction information received from a terminal device in which an application related to music creation is installed.
  • the music composition unit (composing unit) that composes the music information using machine learning based on the selected music information and transmits the composed music information to the terminal device.
  • the information processing device has a composition unit 237).
  • the information processing device presents the music feature information according to the instruction information to the terminal device, so that the creator can select the desired music feature information from the music feature information. Then, the information processing device can provide music information composed based on the music feature information desired by the producer.
  • the information processing device when the information processing device receives the performance information based on the music information transmitted by the composition unit from the terminal device, the information processing device adds the performance information to the selected music feature information and updates the selected music feature information. It further has a unit (update unit 238 in the embodiment). In this way, the information processing device can bring the music feature information closer to the style of the music creator who played the music by adding the performance information by the creator to the selected music feature information.
  • the extraction unit extracts chord progression information indicating chord progression, melody information indicating melody, and base information indicating bass progression in a bar of a specified length as feature quantities from the music information.
  • the generation unit generates score information having chord progression information indicating chord progression, melody information indicating melody, and bass information indicating bass sound progression in a bar of a specified length, and is a component of music feature information.
  • the information processing device can generate music feature information having chord progression information, melody information, and base information. Then, at the time of composing, the information processing device learns not the music information itself but the feature quantities such as chord progression information, melody information, and base information, so that the music information can be efficiently provided to the user.
  • the extraction unit extracts drum information indicating the progress of the drum sound in a bar of a specified length as a feature amount from the music information.
  • the generation unit further adds drum progress information to the score information.
  • the information processing device can generate music feature information having chord progression information, melody information, base information, and drum information.
  • the generation unit generates lyrics information indicating the lyrics in a bar of a specified length from the lyrics information added to the music information, and uses it as a component of the music feature information.
  • the information processing device can extract the lyrics or the music feature information including the lyrics similar to the lyrics and present them to the terminal device, so that the user can create the music.
  • the convenience of the function can be improved.
  • the information processing device can also automatically generate lyrics.
  • the generation unit generates music format information for registering the identification information of the score information for the same measure and the identification information of the lyrics information in association with each other, and uses it as a component of the music feature information.
  • the information processing device can further provide music information desired by the user by learning the music feature information.
  • the generation unit adds the identification information of the score information having the chord progression information similar to the chord progression information of the score information registered in the music format information to the music format information and registers it.
  • the information processing device can compose music information according to the structure of the music.
  • the generation unit generates music order information indicating the order of music format information, and uses it as a component of music feature information. Since the information processing device can also learn the order of the music format information, the learning accuracy can be further improved.
  • Each of the above configurations is an example, and the information processing system 1 may have any system configuration as long as the above-mentioned information processing can be realized.
  • the copyrighted work management device 100 and the information processing device 200 may be integrated.
  • each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of the device is functionally or physically distributed / physically in arbitrary units according to various loads and usage conditions. Can be integrated and configured.
  • FIG. 28 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the information processing device 200, the copyrighted work management device 100, and the user terminal 300.
  • the computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each part of the computer 1000 is connected by a bus 1050.
  • the CPU 1100 operates based on the program stored in the ROM 1300 or the HDD 1400, and controls each part. For example, the CPU 1100 expands the program stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.
  • the ROM 1300 stores a boot program such as a BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started, a program that depends on the hardware of the computer 1000, and the like.
  • BIOS Basic Input Output System
  • the HDD 1400 is a computer-readable recording medium that non-temporarily records a program executed by the CPU 1100 and data used by the program.
  • the HDD 1400 is a recording medium for recording an information processing program according to the present disclosure, which is an example of program data 1450.
  • the communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (for example, the Internet).
  • the CPU 1100 receives data from another device or transmits data generated by the CPU 1100 to another device via the communication interface 1500.
  • the input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000.
  • the CPU 1100 receives data from an input device such as a keyboard or mouse via the input / output interface 1600. Further, the CPU 1100 transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. Further, the input / output interface 1600 may function as a media interface for reading a program or the like recorded on a predetermined recording medium (media).
  • the media is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.
  • an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk)
  • a magneto-optical recording medium such as an MO (Magneto-Optical disk)
  • a tape medium such as a magnetic tape
  • magnetic recording medium such as a magnetic tape
  • semiconductor memory for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.
  • the CPU 1100 of the computer 1000 realizes the functions of the control unit 130 and the like by executing the information processing program loaded on the RAM 1200.
  • the information processing program according to the present disclosure and the data in the storage unit 120 are stored in the HDD 1400.
  • the CPU 1100 reads the program data 1450 from the HDD 1400 and executes the program, but as another example, these programs may be acquired from another device via the external network 1550.
  • the present technology can also have the following configurations.
  • the acquisition department that acquires music information
  • An extraction unit that extracts a plurality of types of feature quantities from the music information acquired by the acquisition unit, and an extraction unit.
  • a generator that generates information that associates the plurality of types of feature quantities extracted by the extraction unit with predetermined identification information as music feature information that is used as learning data in composition processing using machine learning.
  • An information processing device characterized by having.
  • the acquisition unit acquires music information by receiving music information including features related to music created by the creator using an application related to music creation installed in the terminal device from the terminal device.
  • the extraction unit extracts a plurality of types of features included in the music information, and obtains a plurality of types of features.
  • the information processing device wherein the generation unit associates the identification information of the creator with the music feature information.
  • the feature amount of the music created by the creator is characterized by being chord progression information indicating chord progression, melody information indicating melody, and base signal indicating bass progression in a bar of a specified length.
  • the information processing device according to (2).
  • the information processing apparatus according to (3), wherein the feature amount related to the music created by the creator is drum progress information indicating the progress of the drum in a measure of a specified length.
  • the acquisition unit acquires the written music information that is regularly registered at a preset timing.
  • the extraction unit extracts the feature quantities of the plurality of types from the written music information, and then extracts the feature amounts of the plurality of types.
  • the information processing device wherein the generation unit associates the identification information of the copyrighted music information with the music feature information.
  • a transmitter that transmits the presentation information of the music feature information according to the instruction information received from the terminal device in which the application related to music creation is installed.
  • a composition unit that composes the music information using machine learning based on the selected music feature information and transmits the composed music information to the terminal device.
  • the information processing apparatus according to (1).
  • the update unit that adds the performance information to the selected music feature information and updates the selected music feature information.
  • the information processing apparatus according to (6).
  • the extraction unit extracts chord progression information indicating chord progression, melody information indicating melody, and base signal indicating bass progression in a bar of a specified length from the music information as feature quantities.
  • the generation unit generates score information having chord progression information indicating chord progression, melody information indicating melody, and base information indicating bass sound progression in the bar of the specified length, and the music feature information.
  • the information processing apparatus according to (1), characterized in that it is a component of the above.
  • the extraction unit extracts, as a feature amount, drum information indicating the progress of the drum sound in the bar of the specified length from the music information.
  • the information processing apparatus according to (8), wherein the generation unit further adds the drum information to the score information.
  • the generation unit is characterized in that, from the lyrics information added to the music information, the lyrics information indicating the lyrics in the measure of the specified length is generated and used as a component of the music feature information (8). ).
  • the generation unit is characterized in that it generates music format information in which the identification information of the score information and the identification information of the lyrics information for the same measure are registered in association with each other, and is used as a component of the music feature information ( The information processing apparatus according to 10).
  • (12) The generation unit is characterized in that identification information of score information having chord progression information similar to the chord progression information of the score information registered in the music format information is added to the music format information and registered (11). ).
  • the information processing device wherein the generation unit generates music sequence information indicating the order of the music format information and uses it as a component of the music feature information.
  • Information processing system 100 Document management device 110, 210, 310 Communication unit 120, 220, 340 Storage unit 130, 230, 350 Control unit 121 Music information storage unit 122 Music storage unit 123 Feature information storage unit 131 Management unit 132 Extraction Unit 133 Transmission unit 200 Information processing device 221 User information storage unit 222 Style information storage unit 223 Owned information storage unit 224 Production information storage unit 225 Operation history information storage unit 231 Acquisition unit 232 Extraction unit 233 Generation unit 234 Reception unit 235 Selection unit 236 Transmission unit 237 Composition unit 238 Update unit 300 User terminal 320 Input unit 330 Output unit 351 Display control unit 352 Registration unit 353 Transmission / reception unit 354 Selection unit 355 Playback unit 356 Performance reception unit 360 Display unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Electrophonic Musical Instruments (AREA)

Abstract

Un dispositif de traitement d'informations selon la présente divulgation comprend : une unité d'acquisition qui acquiert des informations de morceau de musique; une unité d'extraction qui extrait une quantité de caractéristiques pour chaque type d'une pluralité de types à partir des informations de morceau de musique acquises par l'unité d'acquisition; et une unité de génération qui génère, en tant qu'informations de caractéristique de morceau de musique à utiliser en tant que données d'apprentissage dans un processus de composition musicale utilisant un apprentissage automatique, des informations dans lesquelles des informations d'identification prescrites sont associées à la quantité de caractéristiques pour chaque type de la pluralité de types, qui est extrait par l'unité d'extraction.
PCT/JP2020/042873 2019-11-26 2020-11-17 Dispositif de traitement d'informations, procédé de traitement d'informations et programme de traitement d'informations WO2021106694A1 (fr)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US17/756,108 US20220406283A1 (en) 2019-11-26 2020-11-17 Information processing apparatus, information processing method, and information processing program
JP2021561336A JPWO2021106694A1 (fr) 2019-11-26 2020-11-17
CN202080079390.XA CN114730551A (zh) 2019-11-26 2020-11-17 信息处理设备、信息处理方法和信息处理程序
EP20894684.8A EP4068272A4 (fr) 2019-11-26 2020-11-17 Dispositif de traitement d'informations, procédé de traitement d'informations et programme de traitement d'informations

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-212912 2019-11-26
JP2019212912 2019-11-26

Publications (1)

Publication Number Publication Date
WO2021106694A1 true WO2021106694A1 (fr) 2021-06-03

Family

ID=76129304

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/042873 WO2021106694A1 (fr) 2019-11-26 2020-11-17 Dispositif de traitement d'informations, procédé de traitement d'informations et programme de traitement d'informations

Country Status (5)

Country Link
US (1) US20220406283A1 (fr)
EP (1) EP4068272A4 (fr)
JP (1) JPWO2021106694A1 (fr)
CN (1) CN114730551A (fr)
WO (1) WO2021106694A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021117434A (ja) * 2020-01-29 2021-08-10 国立大学法人京都大学 音楽モデル生成システム、自動作曲システム、編曲システム、音楽モデル生成方法、編曲方法、コンピュータプログラム、および音楽モデル

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023235448A1 (fr) * 2022-06-01 2023-12-07 Library X Music Inc. Moteur de génération automatisée de piste d'origine

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09244648A (ja) * 1996-11-05 1997-09-19 Casio Comput Co Ltd 自動作曲機
JP2011175006A (ja) * 2010-02-23 2011-09-08 Sony Corp 情報処理装置、自動作曲方法、学習装置、学習方法、及びプログラム
US9110817B2 (en) 2011-03-24 2015-08-18 Sony Corporation Method for creating a markov process that generates sequences
JP2017182089A (ja) * 2017-06-09 2017-10-05 カシオ計算機株式会社 自動作曲装置、方法、およびプログラム

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20000058995A (ko) * 2000-07-08 2000-10-05 강승일 웹을 이용한 음악작곡 시스템 및 방법
US6835884B2 (en) * 2000-09-20 2004-12-28 Yamaha Corporation System, method, and storage media storing a computer program for assisting in composing music with musical template data
WO2007133760A2 (fr) * 2006-05-12 2007-11-22 Owl Multimedia, Inc. Procédé et système d'extraction d'informations muisicales
WO2008050718A1 (fr) * 2006-10-26 2008-05-02 Nec Corporation Dispositif d'extraction d'informations de droit, procédé d'extraction d'informations de droit et programme
US20150193199A1 (en) * 2014-01-07 2015-07-09 Qualcomm Incorporated Tracking music in audio stream
US20160196812A1 (en) * 2014-10-22 2016-07-07 Humtap Inc. Music information retrieval
US10296638B1 (en) * 2017-08-31 2019-05-21 Snap Inc. Generating a probability of music using machine learning technology

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09244648A (ja) * 1996-11-05 1997-09-19 Casio Comput Co Ltd 自動作曲機
JP2011175006A (ja) * 2010-02-23 2011-09-08 Sony Corp 情報処理装置、自動作曲方法、学習装置、学習方法、及びプログラム
US9110817B2 (en) 2011-03-24 2015-08-18 Sony Corporation Method for creating a markov process that generates sequences
JP2017182089A (ja) * 2017-06-09 2017-10-05 カシオ計算機株式会社 自動作曲装置、方法、およびプログラム

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
FUKUYAMA, SATORU ET AL.: "Melody composition model for automatic musical composition based on the prosody of lyrics", PROCEEDINGS OF 2009 SPRING RESEARCH CONFERENCE OF THE ACOUSTICAL SOCIETY OF JAPAN, ACOUSTICAL SOCIETY OF JAPAN, 10 March 2009 (2009-03-10), JP, pages 873 - 874, XP009529133 *
JEAN-PIERRE BRIOT; FRAN\C{C}OIS PACHET: "Music Generation by Deep Learning - Challenges and Directions", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 9 December 2017 (2017-12-09), 201 Olin Library Cornell University Ithaca, NY 14853, XP081073413, DOI: 10.1007/s00521-018-3813-6 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021117434A (ja) * 2020-01-29 2021-08-10 国立大学法人京都大学 音楽モデル生成システム、自動作曲システム、編曲システム、音楽モデル生成方法、編曲方法、コンピュータプログラム、および音楽モデル
JP2022088461A (ja) * 2020-01-29 2022-06-14 国立大学法人京都大学 編曲システム、編曲方法、およびコンピュータプログラム
JP7122010B2 (ja) 2020-01-29 2022-08-19 国立大学法人京都大学 音楽モデル生成システム、自動作曲システム、音楽モデル生成方法、およびコンピュータプログラム
JP7371962B2 (ja) 2020-01-29 2023-10-31 国立大学法人京都大学 編曲システム、編曲方法、およびコンピュータプログラム

Also Published As

Publication number Publication date
JPWO2021106694A1 (fr) 2021-06-03
CN114730551A (zh) 2022-07-08
US20220406283A1 (en) 2022-12-22
EP4068272A1 (fr) 2022-10-05
EP4068272A4 (fr) 2022-12-07

Similar Documents

Publication Publication Date Title
US10657934B1 (en) Enhancements for musical composition applications
US7518052B2 (en) Musical theme searching
US20070261535A1 (en) Metadata-based song creation and editing
CN106708894B (zh) 电子书配置背景音乐的方法和装置
WO2021106694A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme de traitement d'informations
US11183160B1 (en) Musical composition file generation and management system
Turchet et al. Cloud-smart musical instrument interactions: Querying a large music collection with a smart guitar
KR20180025084A (ko) 음원 제공 서버의 불특정 다수에 의한 음원 제작 방법, 음원 제공 서버 및 음원 제작 단말
JP2023129639A (ja) 情報処理装置、情報処理方法及び情報処理プログラム
Müller et al. Content-based audio retrieval
US20220406280A1 (en) Information processing apparatus, information processing method, and information processing program
JPH11184883A (ja) 楽曲検索システム、楽曲検索方法及び記録媒体
WO2021166745A1 (fr) Procédé de génération d'agencement, dispositif de génération d'agencement et programme de génération
WO2020218075A1 (fr) Dispositif et procédé de traitement d'informations, et programme
WO2022044646A1 (fr) Procédé de traitement d'informations, programme de traitement d'informations et dispositif de traitement d'informations
WO2021005958A1 (fr) Dispositif, procédé et programme de traitement d'informations
US20240153475A1 (en) Music management services
KR20140054810A (ko) 반주음악 제작 서비스 시스템 및 그 방법, 그리고 이에 적용되는 장치
JP6509291B2 (ja) 楽曲共作支援システム
WO2024075634A1 (fr) Procédé d'affichage relatif à une distribution caractéristique de forme d'onde sonore
WO2024075638A1 (fr) Procédé d'entraînement de modèle acoustique
RU2808611C2 (ru) Способ и система для генерирования выходного звукового файла или файла midi посредством гармонической карты аккордов
WO2024075422A1 (fr) Procédé et programme de création de composition musicale
Müller et al. Multimodal music processing (dagstuhl seminar 11041)
JP6611633B2 (ja) カラオケシステム用サーバ

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20894684

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021561336

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020894684

Country of ref document: EP

Effective date: 20220627