JP5847048B2 - Piano roll type score display apparatus, piano roll type score display program, and piano roll type score display method - Google Patents

Description

  The present invention relates to a piano roll type score display device that displays a piano roll type score representing the scale and output timing of a performance sound included in score information, and particularly suitable for a chord piano roll type score included in the score information. It relates to improvements to display.

  A technique for displaying a piano roll indicating the scale and output timing of a performance sound included in musical score information is known. As a typical mode, an image simulating a piano keyboard is displayed on one end of the screen, a display area is set so that each key extends from the image toward the other end, and each keyboard in the display area is set. That is, a band image indicating the output timing and the sound length is displayed at a position corresponding to (ie, scale). According to such a piano roll, the scale and output timing of the performance sound included in the score information can be displayed in an easily understandable manner.

  The piano roll is preferably used in, for example, sequence software in the field of DTM (Desk Top Music), etc., and performs the performance output of music data selected from a large number of music data and the lyrics of the music. It is also applied to karaoke devices that display character images. Therefore, a technique for creating the piano roll corresponding to music data used in a karaoke apparatus has been proposed. For example, it is a karaoke apparatus described in Patent Document 1. According to this technology, using the data of the guide melody track included in the music data for playing the karaoke music, the pitch of the singing melody is developed on the vertical axis in parallel with the performance of the karaoke music. By displaying on the monitor a piano roll that displays the notes in a strip with the length of the horizontal axis expanded, the melody to be sung can be easily displayed on the monitor without reading the score. Has been.

JP 2004-205817 A

  By the way, the music data used in the karaoke apparatus often includes the performance sound of a plucked string instrument such as an electric guitar or a bass guitar. However, the conventional technique preferably uses the performance sound of such a plucked string instrument. It could not be displayed as a piano roll. That is, for a musical instrument that is played by mixing a plurality of performance sounds corresponding to one performance operation, such as a plucked string instrument, the output start time is different for each performance (code) for one stroke. A plurality of performance sounds are defined in the music data, but when such performance sounds are displayed as a conventional piano roll, there is a risk that it may be difficult to understand without knowing at what timing the performance should be performed. . Such a problem has been newly found in the process in which the present inventor continues eager research with the intention of improving the piano roll type score display device.

  The present invention has been made against the background of the above circumstances, and the object of the present invention is to provide a piano roll type score display apparatus and a piano roll type for suitably displaying a chord piano roll type score included in the score information. A music score display program and a piano roll type music score display method are provided.

In order to achieve such an object, the gist of the first aspect of the present invention is to provide a piano roll type musical score display device for displaying a piano roll type musical score representing the scale and output timing of the performance sound included in the musical score information of the plucked string instrument. a is, the included in the music information of the plucked string instrument, a chord extracting means respectively output start point to extract respectively a stroke of chords different performance sound is output by superimposing, by its chord extracting means Output start time determining means for determining the output start time of the extracted chord based on the output start time of at least one performance sound among the output start times of each of the plurality of performance sounds output superimposed on the chord If, as a piano roll type score representing said chord, arranged a code corresponding to each of the plurality of chord included in the music information in one axis, and said first shaft A piano roll type score display control means for displaying the band images representing the chord as one time position corresponding to the output start time, which is determined by the output start time determining means for interlinking direction time position direction, include It is characterized by.

In order to achieve the above object, the gist of the second aspect of the present invention is to provide a piano roll type score display apparatus for displaying a piano roll type score indicating the scale and output timing of the performance sound included in the score information of the plucked string instrument . The control device extracts chords for one stroke each of which is output by superimposing a plurality of performance sounds each having a different output start time included in the musical score information of the plucked string instrument, and extracted by the chord extraction means An output start time determination means for determining the output start time of the chord based on the output start time of at least one performance sound among the output start times of each of the plurality of performance sounds output by superimposing on the chord; and, as a piano roll type score representing said chord, arranged a code corresponding to each of the plurality of chord included in the music information in one axis, the 1 Be made to function as a piano-roll score display control means for displaying the band images representing the chord as one time position corresponding to the output start time, which is determined by the output start time determining means for direction time position direction perpendicular to the Is a piano roll type music notation display program.

In order to achieve the above object, the gist of the third invention is a piano roll-type score display method for displaying a piano roll-type score indicating the scale and output timing of the performance sound included in the score information of the plucked string instrument. A chord extraction process for extracting chords for one stroke each of which is output by superimposing a plurality of performance sounds each having a different output start time included in the musical score information of the plucked string instrument, and extraction in the chord extraction process An output start time determination process for determining an output start time of a chord to be performed based on an output start time of at least one performance sound among the output start times of each of the plurality of performance sounds that are output superimposed on the chord; as a piano-roll score representing said chord, arranged a code corresponding to each of the plurality of chord included in the music information in one axis, and said first shaft A piano roll type score display control process for displaying the band images representing the chord as one time position corresponding to the output start time determined in the said output start time determining process of the direction orthogonal as the time position direction, include It is characterized by.

Thus, according to the first invention, chord extraction to extract the included in the music information of the plucked string instrument, 1 stroke chords each output start time is different performance sound is output by superimposing the respective And the output start time of the chord extracted by the chord extraction means based on the output start time of at least one performance sound among the output start times of each of the plurality of performance sounds output superimposed on the chord. Output start time determining means for determining the chord and a piano roll type score representing the chord, the chords corresponding to each of the plurality of chords included in the score information are arranged in one axis, and the direction orthogonal to the one axis piano for displaying a band image representing the chord as one end time corresponding positions in the output start time determined by the output start time determining means as a time position direction An Lumpur type score display control means, since it is intended to include, at a performance sound of plucked string instrument, a chord, each output start time is played by commingled plurality of different performance sound, visually understandable piano roll type score Can be displayed. That is, it is possible to provide a piano roll type musical score display device that suitably displays a chord piano roll type musical score included in the musical score information.

  Here, in the first invention, preferably, the chord extracting means extracts a chord corresponding to the performance for one stroke of the plucked string instrument included in the score information of the plucked string instrument. In this way, it is possible to display a chord of a plucked string musical instrument that is played by mixing a plurality of performance sounds having different output start points, as a visually easy-to-understand piano roll score.

  Preferably, the piano roll type score display control means arranges chords corresponding to each of a plurality of chords extracted by the chord extraction means in one axis direction, and sets a direction orthogonal to the one axis direction as time. As the position direction, a band image extending in the time position direction corresponding to the chord length is displayed with the time position corresponding to the output start time determined by the output start time determination means as one end. . In this way, it is possible to display a chord that is played by mixing a plurality of performance sounds having different output start points, as a piano roll score in a practically easy-to-understand manner.

Preferably, the piano roll type score display control means, together with the piano roll type score representing the chord, an image representing how to hold down the finger position and the string position in the plucked string instrument corresponding to the chord. Is displayed. If it does in this way, in the aspect which displays the said piano roll type | form score, for example at the time of practicing a plucked string instrument, the performance method of the said plucked string instrument can be displayed on the piano roll type | mold score clearly intelligibly.

  Preferably, the output start time determination means determines the bar line closest to the average value of the output start times of each of the plurality of performance sounds that are output superimposed on the chord as the output start time of the chord. Is. In this way, it is possible to determine in an appropriate and practical manner the output start time of a chord that is played by mixing a plurality of performance sounds each having a different output start time.

  Preferably, the plucked string instrument is an electric guitar. In this way, it is possible to display the chord of the electric guitar that is played by mixing a plurality of performance sounds having different output start times as a piano roll type score that is easy to understand visually.

  Also preferably, the plucked string instrument is a bass guitar. In this way, it is possible to display the chords of the bass guitar that are played by mixing a plurality of performance sounds having different output start points as a piano roll score that is easy to understand visually.

In addition, according to the second invention, the control device of the piano roll type score display device, wherein is included in the music information of the plucked string instrument, a plurality of performance sound output start time are different from each is output by superimposing 1 Chord extraction means for extracting each chord for the stroke , and at least one of the output start times of each of the plurality of performance sounds output by superimposing the output start time of the chord extracted by the chord extraction means. An output start time determination means for determining based on an output start time of two performance sounds, and a chord corresponding to each of the plurality of chords included in the score information as one axis as a piano roll type score representing the chord arranged as a part time position corresponding to the output start time, which is determined by the output start time determining means and a direction perpendicular to said first shaft as a time position direction Since this is a piano roll-type score display program that functions as a piano roll-type score display control means for displaying a band image representing a chord of a chord, a plurality of output start points, such as performance sounds of plucked string instruments, are different. Can be displayed as a piano roll score that is easy to understand visually. That is, it is possible to provide a piano roll-type score display program that suitably displays a chord piano roll-type score included in the score information.

Further, according to the third invention, the included in the music information of the plucked string instrument, a chord extracting step of extracting 1 stroke chord each having a plurality of performance sound output start time are different from each is output by superimposing And determining the output start time of the chord extracted in the chord extraction process based on the output start time of at least one performance sound among the output start times of each of the plurality of performance sounds output superimposed on the chord. A chord corresponding to each of the plurality of chords included in the score information is arranged on one axis as a piano roll type music representing the chord, and a direction orthogonal to the one axis is time Pianoro for displaying the band images representing the chord as one time position corresponding to the determined output start time at said output start time determination process as the position-direction As a piano roll type score, a chord that is produced by mixing multiple performance sounds with different output start points, such as the performance sound of a plucked string instrument, for example, is included. Can be displayed. In other words, it is possible to provide a piano roll type score display method for suitably displaying a chord piano roll type score included in the score information.

It is a figure explaining the structure of the communication karaoke system in which this invention is used suitably. It is a block diagram which illustrates the composition of the karaoke device which is one example of the piano roll type musical score display device of the present invention. It is a functional block diagram explaining the principal part of the control function with which CPU of the karaoke apparatus of FIG. 2 was equipped. It is a figure explaining the problem at the time of performing the display corresponding to the score information of a plucked instrument, such as an electric guitar, in the conventional piano roll. FIG. 3 is a diagram for describing control for determining a bar line closest to an average value of output start times of a plurality of performance sounds as the output start time of the chord, with respect to piano roll type score display control by the karaoke apparatus of FIG. 2. It is a figure which shows an example of the piano roll type score display screen displayed on a display by the piano roll type score display control by the karaoke apparatus of FIG. It is a figure which shows another example of the piano roll-type score display screen displayed on a display by the piano roll-type score display control by the karaoke apparatus of FIG. It is a flowchart explaining the principal part of an example of the piano roll type | formula display control and performance evaluation control by CPU of the karaoke apparatus of FIG.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of a communication karaoke system 10 in which the present invention is preferably used. In the communication karaoke system 10 shown in FIG. 1, a plurality of private rooms 14a, 14b, 14c,... In a store 12 such as a karaoke box, a snack, an inn, etc. (hereinafter simply referred to as a private room 14 unless otherwise distinguished). Karaoke apparatuses 16a, 16b, 16c, which are one embodiment of the piano roll-type score display apparatus of the present invention, each of which is one or more (one in FIG. 1). In this case, it is simply called karaoke device 16). The plurality of karaoke devices 16 are connected to the communication line 20 via the router 22, and similarly, information is exchanged between the server device 18 and the like connected to the communication line 20 via the communication line 20. Communication is possible.

  The online karaoke system 10 includes a plurality of electronic quick sample devices 28a, 28b, 28c,... (Hereinafter simply referred to as the electronic quick sample device 28 unless otherwise distinguished). When using 16, one to several electronic quick sample devices 28 are lent to each user (group), and as described later in each private room 14, an association process is performed with the karaoke device 16. By being performed, the karaoke device 16 is used as a remote control device. Further, as shown in FIG. 1, a LAN 24 for connecting the plurality of karaoke devices 16 to each other is laid in the store 12, and an input to the karaoke device 16 from the electronic quick sample device 28 is predetermined. The access point 26 and the LAN 24 are used for LAN communication.

  The communication line 20 is a wide area information communication network connected to the Internet such as a WWW (World Wide Web) configured by, for example, a public telephone line, an ADSL line, or an optical fiber line. The server device 18 is a server operated by, for example, an information distribution service provider that manages the communication karaoke system 10, and music data (karaoke data), background video information, and music intervals in the communication karaoke system 10. It functions as a center device that performs basic control of storage and input / output management of digital contents such as information. Also, an SNS (Social Network Service) that stores information related to karaoke performance using the karaoke device 16 for each user who uses the online karaoke system 10 in association with identification information (user ID) of the user. ) Functions as a database server that manages the database. In the present embodiment, a description will be given of a configuration including the single server device 18 having both the function as the center device of the online karaoke system 10 and the function as the database server. May be configured as an individual server device.

  As shown in FIG. 1, the communication karaoke system 10 can transmit and receive information to and from a plurality of systems or communication terminal devices via the communication line 20. That is, a personal computer 30 as a communication terminal device is connected to the communication line 20, and a mobile phone 32 and a tablet 34 as portable information communication terminals are connected to the communication line 20 via a relay base station 36. The personal computer 30, the mobile phone 32, the tablet 34, and the like and the server device 18 can exchange information with each other.

  FIG. 2 is a block diagram illustrating a configuration of a karaoke apparatus 16 that is an embodiment of the piano roll type musical score display apparatus of the present invention provided in the communication karaoke system 10. As shown in FIG. 2, the karaoke apparatus 16 of this embodiment includes a CPU 40 that is a central processing unit, a ROM 42 that is a read-only memory, a RAM 44 that is a write / read memory as needed, and a hard disk 46 that is a storage device. And the video processing unit 48 such as a graphics chip (graphic board), the sound processing unit 50 such as a sound chip (sound board), the operation panel 52, the display control unit 54, and the input control unit 56. A touch panel display 58 connected to the CPU 40, a communication interface 64, a LAN interface 66, a wireless communication unit 68, and a display 72 as a video display device connected to the video processing unit 48 via a video output terminal 70 , Audio connected to the audio processing unit 50 via the audio input terminal 74 A microphone 76 which is a power device, an amplifier 80 which is a sound amplifying device connected to the sound processing unit 50 via an audio output terminal 78, a speaker 82 which is a sound output device provided in the amplifier 80, and an audio A sound effector 88 connected to the sound processing unit 50 via an input terminal 86 is provided.

  The CPU 40 is a so-called microcomputer that processes and controls electronic information based on a predetermined program stored in advance in the ROM 42 using the temporary storage function of the RAM 44, and executes various controls in the karaoke device 16. . That is, when a predetermined music piece (karaoke performance music piece) is selected and input by the operation panel 52, the touch panel display 58, the electronic quick sample device 28, or the like, the selected music piece is reserved music provided in the RAM 44 or the like. Music selection reservation control to be registered in the table, music data read control for reading performance information and lyric information (music data) of the karaoke performance music selected from the hard disk 46 according to the performance order of the reserved music table, music performance reading progress In response to the performance output control for transmitting performance information from the RAM 44 to the sound processing unit 50, a lyric character image is generated based on the lyrics information developed in the RAM 44 during the performance output control, and sent to the video processing unit 48. The lyric character video output control to be transmitted, the video processing in the performance output control Control of 48 and background video output control for playing back a predetermined background video, inter-song information output for outputting inter-song information such as new music information, music selection ranking, store advertisement, etc. while karaoke performance is not performed, that is, between songs In addition to control and basic control such as information communication control with the server device 18 and the like via the communication line 20, various controls such as piano roll type score display control of this embodiment described later are executed. .

  The video processing unit 48 performs various controls related to drawing of a screen (video) displayed on the display 72. For example, based on the data supplied from the CPU 40, the data is written into the graphics memory, and the data is read out to control the display 72 to display a predetermined screen via the video output terminal 70. Specifically, at the time of musical performance output (karaoke performance) by the karaoke device 16, a text image (telop) such as a lyric character image generated by the CPU 40 is output, or an MPEG ( Based on the background video information such as Moving Picture Experts Group) data, a predetermined background video is reproduced (decoded), the lyrics character video is synthesized on the front side of the background video and displayed on the display 72, Various display controls are performed such as displaying lyric text images in a color-changed manner as the music progresses. Further, based on the data supplied from the CPU 40, piano roll type score display control of this embodiment described later is executed. In the present embodiment, a mode in which display control of the display 72 is performed by the video processing unit 48 and display control of the touch panel display 58 (display device 60) is performed by the display control unit 54 described later will be described. A mode in which display control of the touch panel display 58 is also performed by the video processing unit 48 is also conceivable. In this aspect, the display control unit 54 is not necessarily provided.

  The sound processing unit 50 includes various sound sources such as an FM sound source and a PCM sound source, and performs various controls related to sound output by the karaoke device 16. Preferably, a synthesizer that synthesizes sound by an electronic circuit and generates various timbres is provided. The synthesizer generates a music signal such as a musical instrument performance signal based on performance information of a karaoke performance song read from the hard disk 46 and sent. The synthesizer preferably includes a MIDI (Musical Instrument Digital Interface) terminal, and the performance information is, for example, data in the MIDI format. The music signal generated by the synthesizer based on the MIDI data is mixed with the singing voice of the user (performer) input from the microphone 76 via the audio input terminal 74, and via the audio output terminal 78. Is supplied to the amplifier 80, amplified by the amplifier 80, and output from the speaker 82. Sound information input from the electric guitar 90 and the like described later via the sound effector 88 is also supplied to the amplifier 80 via the audio output terminal 86, amplified by the amplifier 80, and output from the speaker 82. The

  The operation panel 52 allows the user of the karaoke apparatus 16 to select a song that the user wants to sing, adjust the pitch related to the performance output of the song, adjust the volume balance between the performance and the song, The input device includes operation buttons (switches) or knobs for performing various adjustments such as volume and tone. The touch panel display 58 is a device that displays an image (video) and performs an operation input to the karaoke device 16 according to a user's operation. The touch panel display 58 displays a predetermined image (video). And a touch panel 62 that performs input by touching the touch panel display 58 with a user's finger, an unillustrated pen, or the like. The display control unit 54 is a video processing unit that controls drawing of a screen (video) displayed on the display device 60 based on information supplied from the CPU 40. The input control unit 56 is an input processing unit that supplies operation input information input from the touch panel 62 to the CPU 40 and the like. With the above configuration, the touch panel display 58 functions as a second video display device separately from the display 72, and the user of the karaoke apparatus 16 selects a song that the user wants to sing, It functions as an input device for adjusting the pitch related to the performance output, adjusting the volume balance between the performance and the song, and performing various adjustments such as echo, volume and tone.

  The karaoke apparatus 16 is connected to the communication line 20 via the communication interface 64 and the network connection terminal 92, and similarly, the karaoke apparatus 16 communicates with the server apparatus 18 and the like connected to the communication line 20. Information can be communicated via the line 20. The karaoke device 16 is preferably connected to the server device 18 via the communication line 20, and digital data such as music data (karaoke data), background video information, and information between songs is transmitted from the server device 18. It accepts distribution of contents (Digital Contents). That is, the karaoke device 16 is preferably a communication karaoke device that is connected to a predetermined communication line and transmits / receives various information to / from the server, but is not connected to such a communication line. The present invention is also suitably applied to devices and the like.

  The LAN interface 66 is a connector for connecting the karaoke device 16 to the LAN 24 via a LAN connection terminal 94. By being connected to the LAN 24 as such, the karaoke device 16 is similarly connected to the LAN 24. Information can be transmitted / received to / from other devices such as the other karaoke device 16 and the electronic sample device 28 connected to the LAN 24. For example, the music selection input from the electronic quick sample device 28 received via the access point 26 is accepted and stored in a reserved music table provided in the RAM 44, or the karaoke device 16 via the access point 26. Thus, mutual exchange of information between the karaoke device 16 and the electronic quick sample device 28 is executed via radio waves, such as transmitting predetermined information from the electronic quick sample device 28 to the electronic quick sample device 28. Communication between the electronic quick sample device 28 and the server device 18 via the communication line 20 is also performed via the LAN 24 or the karaoke device 16 provided with the router 22.

  The wireless communication unit 68 performs wireless communication between the karaoke device 16 and an input device such as the electronic quick sample device 28. For example, it functions as a remote control receiving unit that receives a remote control signal transmitted from an input device such as the electronic sample device 28. The association (sticking) processing between the karaoke device 16 and the electronic quick sample device 28 is preferably performed via the wireless communication unit 68 by such a remote control signal (infrared signal). In other words, the electronic sample device 28 has an individual serial number, and in the association process, for example, a signal including the serial number (for example, the last four digits) and a predetermined connection code is notified of the connection. The electronic quick sample device 28 is associated with the karaoke device 16 that is transmitted to the karaoke device 16 and receives the connection signal by the wireless communication unit 68. The electronic sample device 28 thus associated with the karaoke device 16 functions as an input device (remote control device) for the karaoke device 16, and a signal transmitted from the electronic control device 28 is sent to the CPU 40. By being supplied, the user of the karaoke apparatus 16 selects a song that the user wants to sing, adjusts the pitch related to the performance output of the song, adjusts the volume balance between the performance and the song, An input for performing various adjustments such as volume and tone is accepted. After the association process is performed, communication between the electronic sample device 28 and the karaoke device 16 is performed by LAN communication via the LAN 24, the access point 26, and the like. Further, in the present embodiment, the following description will be given on the assumption that the input device such as the electronic quick sample device 28 subjected to the association processing with the karaoke device 16 also constitutes a part of the karaoke device 16.

  The hard disk 46 stores a music database 96 (see FIG. 3) for storing a large number (for example, tens of thousands of music) of music data (karaoke data) for performing the performance output (karaoke performance) of the music by the karaoke device 16. ) And other databases. The music data stored in the music database 96 is stored corresponding to each music output in the karaoke apparatus 16 using the music selection number as identification information as an index. For example, metadata, performance information MIDI data and telop data as lyrics information.

  The metadata includes, for example, information such as a song number (song selection number), a song name, a singer name, and a registration date and time as identification information of each song data. The telop data is information for performing lyrics character video display control of each song via the video processing unit 48. For example, each of the lyrics of a song (performance song) corresponding to the song data is divided into a plurality of pieces. The block data corresponding to the section is included. The MIDI data is information for performing performance output control of each musical piece through the audio processing unit 50, and includes a plurality of tracks (channels) respectively corresponding to a plurality of types of musical instruments, and the musical instruments corresponding to the respective tracks. The output timing of the performance sound is defined.

  In the MIDI data, a header (header chunk) is defined, and as the plurality of tracks (track chunks), a conductor track, a track 1, a track 2, a track 3,. 16) is defined. The header stores information such as data length, format, number of tracks, and time unit. This time unit indicates the length of a quarter note and corresponds to the resolution of the MIDI data. The conductor track has a tempo of the MIDI data. In each of the tracks 1 to 16, the musical score (performance sound) related to the performance output corresponding to each track, the scale of the performance sound to be output, the output timing (output start time and sound length), and the like are determined. Information is defined. For example, the type of musical instrument (performance sound) corresponding to each track is determined such that the track 2 is “piano”, the track 3 is “electric guitar”, the track 4 is “bass guitar”,. As the musical score information of the instrument, information such as output timing (delta time), scale, and volume is determined for each track. In addition, a predetermined track (for example, track 10) of the plurality of tracks corresponds to a rhythm track that defines musical score information of an instrument for taking a rhythm exclusively, and a percussion instrument that cannot set the scale of the performance sound. For example, corresponding to musical instruments (performance sounds) such as “bass drum”, “snare drum”, and “hi-hat”, the output timing as musical score information is determined in the rhythm track.

  The sound effector 88 is an instrument input device that is connected to a musical instrument that is an external device and supplies a signal input from the instrument to the sound processing unit 50 via the audio input terminal 86. Preferably, an external device such as an electric guitar 90 or a bass guitar (the electric guitar 90 is shown in FIG. 2) is connected via a shield and a signal (electric guitar 90) input from the electric guitar 90 or the like. A performance signal corresponding to the performance of the sound is supplied to the audio processing unit 50 via the audio input terminal 86. The signal from the sound effector 88 may be input directly to the amplifier 80. Preferably, a plurality of sound effects corresponding to the electric guitar 90 and the like are stored in advance in the sound effector 88 and can be set, and a signal input from the electric guitar 90 or the like. A plurality of preset sound effects are reflected to the audio processing unit 50. It is also possible to accept an input signal from the electric guitar 90 or the like during a karaoke performance based on the MIDI data. In this aspect, the music signal generated by the synthesizer based on the MIDI data is mixed with the performance sound of the electric guitar 90 input from the sound effector 88 via the audio input terminal 86, and the audio The signal is supplied to the amplifier 80 via the output terminal 78, amplified by the amplifier 80, and output from the speaker 82.

  FIG. 3 is a functional block diagram illustrating the main part of the control function provided in the CPU 40 of the karaoke apparatus 16. The performance control means 102 shown in FIG. 3 controls the output (playback) of music by the karaoke apparatus 16. Specifically, in the karaoke performance based on the music data stored in the music database 96, the reserved music stored in the reserved music table in order from the upper reserved music in the reserved music table such as the RAM 44 (in order of input). The music data corresponding to the music selection number of the music is read from the music database 96 of the hard disk 46, and the output of the performance music is controlled based on the performance information included in the music data. For example, on the basis of MIDI data as performance information, a synthesizer provided in the sound processing unit 50 outputs musical instrument performance sounds (music information) corresponding to each track as score information in the MIDI data, and the amplifier And output from the speaker 82 via 80 or the like. Further, the output of the lyric character image related to the performance music is controlled based on the lyric information (telop data) included in the music data. That is, a lyric character image is generated based on the lyric information, and the lyric character image is displayed on the display 72 via the image processing unit 48. Further, as the karaoke performance progresses, the lyric character image displayed on the display 72 is switched and displayed, and the lyric character image is sequentially color-changed and displayed based on the lyric color change information.

  The external input receiving means 106 receives sound information input from an external device connected to the karaoke device 16. Preferably, a sound from an external device corresponding to an electric guitar or a bass guitar (in this embodiment, electric guitar 90) is used as a musical instrument to be displayed by a piano roll type score display control means 116 described later. Accept input of information. For example, input of sound information (performance signal) from the electric guitar 90 as an external device connected to the sound processing unit 50 via the sound effector 88 and the audio input terminal 86 is input to the sound processing unit 50. . The sound information received by the external input receiving means 106 is supplied from the audio processing unit 50 to the amplifier 80 via the audio output terminal 78, and is output from the speaker 82 by the amplifier 80 as described above. . Here, the external input receiving means 106 may be provided in a control unit built in the sound effector 88. Further, the voice processing unit 50 may function as the external input receiving unit 106.

  The performance control means 102 includes minus one performance control means 104. The minus one performance control means 104, for example, when the external input acceptance means 106 accepts the input of sound information from the external device, the score corresponding to each of the plurality of types of musical instruments predetermined in the music data. Of the information, minus one (N-1) performance control is performed in which the performance sound is output based on the musical score information corresponding to the other musical instruments without outputting the performance sound based on the musical score information corresponding to the prescribed musical instrument. . Here, it is preferable that the external device that receives the input of sound information by the external input receiving unit 106 corresponds to the specified musical instrument that is excluded from the output target by the minus one performance control unit 104. In other words, the minus one performance control means 104, for example, when the external input reception means 106 accepts input of sound information from the electric guitar 90, plays musical instruments corresponding to a plurality of tracks in the music data. Of the sounds, the performance sound of the musical instrument corresponding to the remaining track is output without outputting the performance sound of the track (for example, track 3) corresponding to the electric guitar 90. In other words, the output of the music based on the music data is controlled assuming that there is no track corresponding to the electric guitar 90. At the time of performance control by the minus one performance control means 104, if the input of sound information from the external device is received by the external input receiving means 106, the sound information input from the external device is as if one part (external) As a part corresponding to the apparatus, it is possible to realize a karaoke performance as if a band performance was performed in accordance with the performance of the musical instrument corresponding to the remaining track. Further, the minus one performance control means 104 accepts selection of the type of musical instrument excluded from the performance output target in response to a user input operation via the operation panel 52, the touch panel display 58, etc. The minus one performance control may be executed in which the performance sound is output based on the score information corresponding to the other musical instruments without outputting the performance sound based on the score information corresponding to the selected type of musical instrument. .

  The performance evaluation control means 108 evaluates the performance corresponding to the input sound (input signal) from the external device received by the external input reception means 106. For example, sound information input to the sound processing unit 50 from the electric guitar 90 or the like, which is an external device, via the sound effector 88 and the audio input terminal 86 or the like, and music data read from the music database 96 The musical score information of the track corresponding to the electric guitar 90 in the performance information (MIDI data), that is, sample data, is compared, and the scale (pitch) and output timing (output start time, tone length, etc.) in the sample data, Evaluation is performed on the basis of the relative deviation from the scale of the sound information received by the external input receiving means 106 and the input timing. For example, the performance evaluation is performed so that the smaller the relative deviation, the higher the evaluation. Preferably, the performance evaluation control means 108 preferably accepts the external input with respect to the specified musical instrument that is excluded from performance output targets in the minus one performance control when the minus one performance control means 104 performs the minus one performance control. The performance corresponding to the input voice from the specified musical instrument received by the means 106 is evaluated. As an aspect of this evaluation, for example, scoring may be made numerically, such as how many points out of a thousand points, or it may be as simple as determining which of the five grades is applicable. There may be. Further, preferably, the evaluation is performed for each predetermined phrase (musical phrase) in the music a plurality of times at a predetermined timing during the output of the target music, that is, from the performance start to the performance end, The final evaluation is calculated at the end of the performance. More preferably, for each chord (chord) extracted by the chord extraction means 110 described later, the performance corresponding to the chord is evaluated.

  The chord extraction means 110 extracts a chord that is output by superimposing a plurality of performance sounds each having a different output start time included in the MIDI data as the musical score information included in the music data. That is, the output start time is different from each other and is superimposed (synchronized) for at least a predetermined time from the musical score information that determines the scale and output timing of the performance sound of the musical instrument corresponding to the track defined in the MIDI data. The musical score information of the portion corresponding to the consonant (or dissonant) of the plurality of performance sounds to be output (performance sounds corresponding to different scales) is extracted.

  The chord extraction means 110 is preferably included in the musical score information corresponding to the prescribed musical instrument that is excluded from performance output targets in the minus one performance control when the minus one performance control is performed by the minus one performance control means 104. A chord output by superimposing a plurality of performance sounds each having a different output start time is extracted. That is, a chord that is output by superimposing a plurality of performance sounds each having a different output start time included in the musical score information of a plucked string instrument such as the electric guitar 90 or the bass guitar is extracted. In the case of a plucked musical instrument that performs by continuously scratching a plurality of strings, such as the electric guitar 90, when a unit performance that stirs one stroke is performed, the performance sounds of a plurality of scales are mixed together. Played as a chord. That is, the performance of a plucked string instrument such as the electric guitar 90 is to continuously stir a plurality of strings, and thus when a unit performance that stirs one stroke is performed, each of the plurality of strings is performed. The timing at which is played is strictly different. Accordingly, in the musical score data corresponding to the plucked string instrument such as the electric guitar 90 in the MIDI data of the music data, the string (down stroke) which is squeezed relatively earlier is corresponded to the performance of one stroke of the plucked string instrument. The performance sound corresponding to the string on the left side of the performance in the performance and the performance sound corresponding to the string that is squeezed relatively later (the string on the right side of the front in the down stroke performance) A time lag is added. That is, the musical score information corresponding to the plucked instrument such as the electric guitar 90 in the MIDI data includes a chord that is output by superimposing a plurality of performance sounds each having a different output start time. The chord extraction means 110 preferably extracts a chord corresponding to the performance of one stroke of the plucked string instrument included in the score information of the plucked string instrument such as the electric guitar 90.

  The chord determination unit 112 determines a chord corresponding to the chord extracted by the chord extraction unit 110. Preferably, a code database 98 (see FIG. 3) is stored in the hard disk 46 of the karaoke apparatus 16 in order to perform the determination by the code determination means 112. In the code database 98, for example, various kinds of information relating to the chords (guitar chords) of the electric guitar 90 are stored. The chord is, for example, a performance sound (chord) in which the six strings provided in the electric guitar 90 are played when the electric guitar 90 is played, and the sounds that are played overlap each other. It is determined by how to hold the string (how to hold the string with the finger on the opposite side of the hand from scratching the string). That is, in the electric guitar 90, the performer presses the six strings (not necessarily pressing all six) with the fingers of one hand, while holding the six strings with the other hand in a predetermined direction. By choking continuously (at once), a chord corresponding to a predetermined chord is played as a performance sound for one stroke. In other words, the same performance sound (chord) can be played if the electric guitar 90 performs the same performance. The chord database 98 stores the relationship between at least each chord and the scales of a plurality of performance sounds constituting the chord. Preferably, the chord determination means 112 is based on the scale stored in the chord extracted by the chord extraction means 110 based on the scales of a plurality of performance sounds output from the relationship stored in the chord database 98. Determine the chord corresponding to the chord.

  FIG. 4 is a diagram for explaining a problem in performing display corresponding to the musical score information of a plucked instrument such as the electric guitar 90 on the conventional piano roll 120. FIG. 4 exemplifies a typical piano roll in the prior art, and an image 122 simulating a piano keyboard is displayed at one end of the screen (the left end in FIG. 4). At the same time, a scroll display area 124 is set so that each keyboard extends from the image 122 toward the other end. In the scroll display area 124, dark gray belt images 126a and 126b showing the output timing and the sound length of the performance sound of the scale included in the musical score information to be played at the position corresponding to each keyboard (ie, the scale). , 126c (hereinafter simply referred to as the band image 126 unless otherwise specified). In the performance output of the music corresponding to the musical score information, the scroll display area 124 is scrolled and displayed so as to be drawn into each keyboard in the video 122 toward the video 122 imitating the piano keyboard. Then, when the scroll display area 124 is scrolled, the band image 126 displayed in the scroll display area 124 comes into contact with the performance line 128 at the boundary between the scroll display area 124 and the image 122. The performance sound of the scale corresponding to the band image 126 is output, and while the band image 126 passes the performance line 128, the output of the performance sound is continued.

  In FIG. 4, as the chords corresponding to the performance of one stroke of the electric guitar 90, the chord band images 126a, 126b, 126c corresponding to the guitar chord “C” are illustrated. That is, the band image 126a corresponding to the performance sound of “C (C in C major)”, the band image 126b corresponding to the performance sound of “E (Mi in C major)”, and “G (So in C major)”. A band image 126c corresponding to the performance sound is displayed. Here, as described above, in the musical score information corresponding to the electric guitar 90 in the MIDI data of the music data, the performance corresponding to the string that is struck relatively earlier corresponding to the performance for one stroke. There is a time lag at the output start point between the sound and the performance sound corresponding to the string that is struck relatively later. In the example shown in FIG. 4, the output start times of the band images 126a, 126b, and 126c are dispersed (different from each other) across the bar line 130 representing the beat indicated by the vertical line in the scroll display area 124. ) And the output of the performance sound corresponding to the band video 126a, the band video 126b, and the band video 126c in this order is started. Therefore, especially for beginners of performance and the like, the timing for striking the electric guitar 90 is not known, and the piano roll originally displays the musical score information in an easy-to-understand manner, but in the example shown in FIG. It's hard to understand.

  The output start time determination means 114 shown in FIG. 3 includes at least the output start time of each of the plurality of performance sounds output by superimposing the output start time of the chord extracted by the chord extraction means 110 on the chord. The determination is made based on the output start time of one performance sound. Preferably, a bar line (for example, one corresponding to a bar in units of 48 notes) closest to the average value of the output start times of the plurality of performance sounds output superimposed on the chord is output as the chord output start time. (Quantize; automatic correction). For example, in the example shown in FIG. 4 described above, the output start points of the band images 126a, 126b, and 126c are distributed across the bar lines 130, and the bar lines 130 correspond to the band images 126a, 126b, and 126c. This corresponds to the bar line closest to the average value at the start of each output. Therefore, when the chord corresponding to the band images 126a, 126b, and 126c shown in FIG. 4 is extracted by the chord extracting unit 110, the output start time determination unit 114, as shown in FIG. The bar line 130 closest to the average value of the respective output start times of 126b and 126c is determined as the output start time of the band image 132 corresponding to the chord. In general, playing a guitar with a stroke timing that matches the beat (bar) is skillful (skilled), so the chord line output start point is aligned with the bar line as described above. The musically preferable output timing can be determined.

  The piano roll type musical score display control means 116 shown in FIG. 3 represents the chord as a piano roll type music representing the chord with the time position corresponding to the output start time determined by the output start time determination means 114 as one end. Display the band image. For example, as a piano roll type score representing a chord included in the musical score information of the electric guitar 90, a plurality of chords determined by the chord determining unit 112 corresponding to each of a plurality of chords extracted by the chord extracting unit 110. An output start determined by the output start time determination unit 114, with codes arranged in one axis direction (for example, the vertical axis direction) and a direction orthogonal to the one axis direction as a time position direction (for example, the horizontal axis direction) A band image 132 (see FIG. 5) is displayed, with the time position corresponding to the time point as one end and extending in the time position direction corresponding to the length of the chord. That is, the piano roll type musical score display control means 116 does not strictly display a piano roll as shown in FIG. 4 which is conventionally general, but as a piano roll type score similar to such a piano roll, Images such as those shown in FIGS. 6 and 7, which will be described later, are displayed on the display 72 and the like via the image processing unit 48 and the like.

  Preferably, the piano roll type score display control means 116 displays the piano roll type score representing the chord included in the score information of the plucked string instrument such as the electric guitar 90, together with the piano roll type score representing the chord, An image representing how to hold the string in the plucked string instrument corresponding to the chord is displayed. Preferably, in order to perform such display control, the hard disk 46 of the karaoke apparatus 16 is provided with a performance method database 100 (see FIG. 3). In this performance method database 100, for example, a plurality of chords (guitar chords) that can be played on the electric guitar 90 and a video (or such video) showing how to press a string on the electric guitar 90 corresponding to each chord. Information for display) is stored in association with each other. The piano roll type score display control means 116, for example, displays the chord based on the information stored in the performance method database 100 when displaying the piano roll type score representing the chords included in the score information of the electric guitar 90. A video representing how to hold down the strings in the electric guitar 90 corresponding to the above is displayed together with the piano roll type score. Preferably, as shown in FIG. 7, which will be described later, an image representing how to hold the strings on the electric guitar 90 corresponding to each chord, that is, a performance method image corresponding to each chord, is displayed on one end of the screen. A display area is set such that the range corresponding to each chord (performance video) extends toward the other end, and the chord output timing and the position corresponding to each chord (ie chord) in the display area are set. A band image showing the sound length is displayed.

  The piano roll type score display control means 116 preferably displays one piece of musical score information (for example, one musical piece) in the display of a piano roll type score representing a chord included in the score information of a plucked string instrument such as the electric guitar 90. All types of chords extracted by the chord extraction means 110 from the corresponding musical score information) are arranged in one axis direction (for example, the vertical axis direction), and the direction orthogonal to the one axis direction is set to the time position direction (for example, As the horizontal axis direction), a band image 132 having a time position corresponding to the output start time determined by the output start time determination unit 114 as one end and extending in the time position direction corresponding to the chord length is displayed. Let For example, if one piece of musical score information corresponding to the electric guitar 90 includes codes “Cm”, “C #”, “D”, “Dm”, “Am”, and “B”, those codes are used. Display control of the piano roll type music is arranged in one axis direction. Since there are a relatively large number of chords that can be played by the electric guitar 90 (for example, 168 types), it is difficult to display an image showing all the playing methods in a piano roll type score. By arranging all types of chords extracted from the musical score information by the chord extracting means 110 in one axis direction, a video showing a performance method corresponding to necessary and sufficient chords in a piano roll type score, and corresponding to each chord A band image representing the scale and output timing of the performance sound to be played can be displayed, and a piano roll type score similar to a conventional piano roll can be realized.

  FIG. 6 is a view showing an example of a piano roll type score display screen displayed on the display 72 via the video processing unit 48 by the piano roll type score display control means 116. The piano roll score display screen 140 shown in FIG. 6 is displayed for each of a plurality of chords extracted from the musical score information of the performance information (MIDI data) of the music piece, for example, when the performance control unit 102 controls the performance of the predetermined music piece. The band image representing the output timing of each chord is displayed. Specifically, when the minus one performance control is performed by the minus one performance control means 104, a plurality of chords (guitar chords) extracted from the musical score information of the electric guitar 90 excluded from the performance output target in the minus one performance control. ), A band image representing the output timing of each chord is displayed, and the display is sequentially changed in accordance with the progress of the music performance by the minus one performance control means 104 as described in detail below. It is.

  As shown in FIG. 6, in the piano roll type score display screen 140, a used chord list display area 142, a piano roll type score display area 144, a performance method display area 146, and a telop display area 148 are defined. ing. In the used chord list display area 142, chords included in the musical score information (music data) to be displayed on the piano roll type music are listed. That is, chords corresponding to all types of chords extracted from the score information are displayed in the used chord list display area 142. In the piano roll type musical score display area 144, band images 152a, 152b, 152c, and 152d (hereinafter, particularly distinguished) indicating the output timing of each chord for each of the plurality of chords listed in the used chord list display area 142. If not, a piano roll type score 150 for displaying a band image 152 is displayed. In the performance method display area 146, corresponding band images 152 correspond to the chords displayed in the piano roll type score display area 144, and performance method images 160a and 160b showing how to hold the strings in the electric guitar 90 are displayed. , 160c, 160d (hereinafter simply referred to as performance method image 160 unless otherwise distinguished). In the telop display area 148, based on the telop data included in the music data to be displayed on the piano roll type music, the lyrics character video (lyric telop) 162 of the music is displayed.

  Preferably, the piano roll score display screen 140 displays the piano roll score 150 and the like corresponding to the score information of the electric guitar 90 that is excluded from the performance output target in the minus one performance control. It is. That is, at the start of music performance in the minus one performance control, all types of chords included in the musical score information of the electric guitar 90 in the MIDI data are extracted from the MIDI data to be played, and the used code list display area 142 is listed. Further, the piano roll type score 150 is displayed in the piano roll type score display area 144 corresponding to the extracted type of chord. In this piano roll type score 150, all the extracted types of chords are arranged as items at one end of the piano roll type score display area 144 (the left end in FIG. 6). In addition, a scroll display area 154 is set such that items corresponding to the respective codes extend toward the other end. In this scroll display area 154, a dark gray band image 152 indicating the output timing and tone length of the chord included in the musical score information to be played is displayed at a position corresponding to each chord (ie chord). Here, as described above with reference to FIG. 5, the bar line 158 closest to the average value of the output start times of each of the plurality of performance sounds that are preferably output superimposed on each chord is the output start time of the chord. Determination is made, and the band image 152 is displayed. In the performance output of the music corresponding to the musical score information, the scroll display area 154 is scroll-displayed so as to be drawn into items representing each chord in the video toward the video at one end representing the chord. When the scroll display area 154 is scrolled, the band image 152 displayed in the scroll display area 154 comes into contact with the performance line 156 at the boundary between the scroll display area 154 and the video representing the chord. The chord performance sound corresponding to the band image 152 is output, and the chord output is continued while the band image 152 passes the performance line 156.

  On the piano roll type score display screen 140, the display control of the performance method image 160 in the performance method display area 146 is synchronized with the scroll display control of the piano roll type score 150 in the piano roll type score display area 144. Is done. For example, the performance method image 160 displayed in the performance method display area 146 in accordance with the transition of the scroll display region 154 (the band image 152 displayed in the scroll display region 154) in the piano roll type music 150. It is updated sequentially. That is, in the piano roll score 150 of FIG. 6, the band image 152a corresponding to the chord “Am” is in contact with the performance line 158, and the chord of the chord “Am” corresponding to the band image 152a is output. It is shown that it is the timing. In addition, the scroll display area 154 is scrolled to the left toward the paper surface, so that the band image 152b corresponding to the code “Dm” and the band image 152c corresponding to the code “C #” follow the band image 152a. The performance line 156 is touched in the order of the band image 152d corresponding to the chord “Am”, and the performance output corresponding to each band image 152 is performed. Correspondingly, in the performance method display area 146, the performance method image 160a (corresponding to the belt image 152a) showing how to press the string corresponding to the chord “Am”, and the string press corresponding to the chord “Dm” Performance method image 160b (corresponding to the band image 152b) representing the way, performance method image 160c (corresponding to the band image 152c) representing how to press the string corresponding to the chord “C #”, and the string corresponding to the chord “Am” A performance method image 160d (corresponding to the belt image 152d) indicating how to press is displayed with arrows in order from the top toward the page. That is, in response to the display on the piano roll score 150, a performance method image 160 showing how to hold down the strings for playing the chord corresponding to each band image 152 included in the piano roll score 150 is the performance. In addition to being displayed in the method display area 146, the display in the performance method display area 146 is updated as the piano roll score 150 is scrolled and displayed as the music progresses. That is, the performance method image 160 corresponding to the chord that has been played is deleted from the performance method display area 146, and the performance method image 160 corresponding to the chord as a new performance candidate is newly displayed in the performance method display area 146. It will be displayed.

  In addition, on the piano roll type score display screen 140, the lyrics character image (lyric telop) in the telop display area 148 is synchronized with the scroll display control of the piano roll type score 150 in the piano roll type score display area 144. Display control 162 is performed. For example, in the performance output based on the music data, a lyric character image 162 is generated based on the telop data included in the music data, and the lyric character image 162 is transmitted to the telop display area 148 via the video processing unit 48. Is displayed. Further, as the performance of the music progresses, the lyric character image 162 displayed in the telop display area 148 is sequentially switched and displayed, and the lyric character image 162 is changed according to the transition of the singing part (output of the guide melody). Are sequentially displayed in different colors.

  When the performance evaluation unit 108 evaluates the performance corresponding to the input voice from the electric guitar 90 in response to the music performance by the minus one performance control unit 104, the piano roll type score display screen 140 is displayed. For example, in the performance method display area 146, an evaluation result video 164 showing the evaluation result by the performance evaluation means 108 is displayed. For example, an evaluation result video 164 indicating an evaluation result by the performance evaluation means 108 relating to the performance of a chord (chord) corresponding to the band video 152a is attached to the performance method video 160a corresponding to the band video 152a. It is displayed in the method display area 146. In addition, when the evaluation result by the performance evaluation means 108 exceeds a predetermined threshold value continuously for a predetermined number of times, the evaluation result video 164 is made gorgeous, or the display mode of the piano roll score 150 is changed. For example, various display controls may be performed based on the evaluation result by the performance evaluation unit 108.

  FIG. 7 is a view showing another example of a piano roll type score display screen displayed on the display 72 via the video processing unit 48 by the piano roll type score display control means 116. In the piano roll type score display screen 170 shown in FIG. 7, the same reference numerals are given to portions common to the piano roll type score display screen 140 described above with reference to FIG. The piano roll score 172 displayed on the piano roll score display screen 170 shown in FIG. 7 corresponds to all types of chords (chords) extracted by the chord extraction means 110 from the musical score information to be displayed. Then, chord definition images 174a to 174f including performance method images showing how to hold the strings in the electric guitar 90 corresponding to each chord at one end of the screen (hereinafter simply referred to as chord definition images 174 unless otherwise distinguished). ) Are listed in the vertical direction on the page. Then, the scroll display area 154 is set so that the range corresponding to each code from the plurality of code definition images 174 extends toward the other end (rightward in the drawing). In such a piano roll type score 172, a piano roll type score having a form very similar to the piano roll 120 as shown in FIG. 4 is superimposed on a plurality of performance sounds each having a different output start time. This can be realized with respect to musical score information including output chords.

  FIG. 8 is a flowchart for explaining a main part of an example of piano roll type score display control and performance evaluation control by the CPU 40 of the karaoke apparatus 16, and is repeatedly executed at a predetermined cycle.

  First, in step (hereinafter, step is omitted) S1, whether or not the guitar practice mode is established, that is, among the performance sounds of the musical instruments corresponding to a plurality of tracks in the music data, corresponds to “electric guitar”. It is determined whether or not the minus one performance mode is established in which the performance sound of the musical instrument corresponding to the remaining track is output without outputting the performance sound of the track to be played. If the determination in S1 is negative, in S2, normal karaoke performance processing, that is, performance control for outputting performance sounds of all tracks defined in the music data, is performed, and then this routine is terminated. However, if the determination in S1 is affirmative, in S3, whether or not the predetermined music is in the performance order related to the karaoke performance, that is, the performance order of the music stored at the top of the reserved music table and Thus, it is determined whether or not the karaoke performance of the predetermined music is started. If the determination in S3 is negative, the routine is terminated accordingly, but if the determination in S3 is affirmative, the processing in S4 and subsequent steps is executed.

  In S4, the music data corresponding to the music related to the start of performance is read from the music database 96 and expanded in the RAM 44, and the music performance output based on the expanded music data (corresponding to “electric guitar”). Except for the track to be played). Next, in S5, a plurality of performance sounds with different output start times included in the score information of the track corresponding to “electric guitar” are superimposed from the music data (MIDI data) corresponding to the music related to the performance start. All kinds of chords output are extracted. Next, in S6, based on the relationship stored in the chord database 98, the chord corresponding to the chord is determined based on the scales of a plurality of performance sounds that are superimposed on the chord extracted in S5. The Next, in S7, the output start time of the chord extracted in S5 (the chord determined in S6) is at least the output start time of each of the plurality of performance sounds that are superimposed on the chord. The determination is made based on the output start time of one performance sound.

  Next, in S8, a piano roll score corresponding to the chord extracted in S5 (the chord determined in S6) is displayed on the display 72 via the video processing unit 48 and the like. That is, as the piano roll type music representing the chord determined in S6, the band image 152 representing each chord is displayed with the time position corresponding to the output start time determined in S7 as one end, and the progress of the music The display is sequentially updated accordingly. Here, when the musical score information includes a single performance sound such as a solo performance portion in an electric guitar, a piano roll corresponding to the single sound may be displayed in parallel with the display of the band image 152. Good. Next, in S9, it is determined whether or not a voice input from the electric guitar 90 is accepted via the sound effector 88 or the like. If the determination in S9 is negative, the processing from S12 onward is executed. If the determination in S9 is positive, in S10, the sound information input from the electric guitar 90 and the performance output are output. The performance information by the electric guitar 90 is evaluated by comparing the musical score information of the track corresponding to the “electric guitar” in the performance information (MIDI data) of the music data (i.e., MIDI data), that is, the sample data. Next, in S11, the evaluation result of S10 is displayed together with the piano roll type score displayed in S8. Next, in S12, it is determined whether or not the performance is ended, for example, by playing the music to the end. If the determination at S12 is affirmative, the routine is terminated accordingly. If the determination at S12 is negative, the processing from S4 onward is executed.

  In the above control, S2 to S4 and S12 are operations of the performance control means 102 (minus one performance control means 104), S9 is an operation of the external input receiving means 106, and S10 is an operation of the performance evaluation means 108. S5 is the operation of the chord extraction means 110 or chord extraction process, S6 is the operation of the chord determination means 112, S7 is the operation of the output start time determination means 114 or the output start time determination process, and S8 is the This corresponds to the operation of the piano roll type score display control means 116 or the piano roll type score display control process.

  Thus, according to the present embodiment, the chord extraction means 110 (S5) for extracting a chord output by superimposing a plurality of performance sounds each having a different output start time included in the score information, and the chord. An output for determining the output start time of the chord extracted by the extraction means 110 based on the output start time of at least one performance sound among the output start times of each of the plurality of performance sounds output superimposed on the chord. A band representing the chord with the time position corresponding to the output start time determined by the output start time determination means 114 as one end, as the start time determination means 114 (S7) and the piano roll music notation 150, 172 representing the chord. Since it includes piano roll score display control means 116 (S8) for displaying video 152, for example, the performance sound of a plucked string instrument, etc. Is a chord of the output start time is played by commingled plurality of different performance sound can be displayed as a visually understandable piano roll type score 150,172. That is, it is possible to provide the karaoke device 16 as a piano roll type musical score display device that suitably displays the chord piano roll type musical scores 150 and 172 included in the musical score information.

  Further, the chord extraction means 110 extracts chords corresponding to the performance of one stroke of the plucked string instrument included in the score information of the electric guitar 90 that is a plucked string instrument. It is possible to display chords of a plucked string musical instrument that is played by mixing a plurality of performance sounds having different numbers as piano roll type notations 150 and 172 that are easy to understand visually.

  The piano roll type score display control means 116 arranges chords corresponding to each of the plurality of chords extracted by the chord extraction means 110 in one axis direction, and sets the direction orthogonal to the one axis direction to the time position direction. As described above, a band image 152 having a time position corresponding to the output start time determined by the output start time determination means 114 as one end and extending in the time position direction corresponding to the chord length is displayed. Therefore, it is possible to display chords produced by mixing a plurality of performance sounds having different output start points as piano roll staves 150 and 172 that are visually easy to understand and practical.

  The piano roll type score display control means 116 displays a piano roll type score 150, 172 representing the chord and a performance method image 160 representing how to press a string in the plucked string instrument corresponding to the chord. Therefore, for example, in a mode in which the piano roll type music 150, 172 is displayed when practicing a plucked string instrument, the practicing method of the plucked string instrument is displayed in an easily understandable manner in association with the piano roll type music 150, 172. it can.

  The output start time determination means 114 determines the bar line 158 closest to the average value of the output start times of each of the plurality of performance sounds output superimposed on the chord as the output start time of the chord. Therefore, it is possible to determine in an appropriate and practical manner the output start point of a chord that is played by mixing a plurality of performance sounds each having a different output start point.

  In addition, since the plucked string instrument is the electric guitar 90, the chords of the electric guitar 90 that are played by mixing a plurality of performance sounds with different output start points are displayed as piano roll charts 150 and 172 that are visually easy to understand. Can be made.

  Further, according to the present embodiment, the CPU 40 in the control device of the karaoke apparatus 16 extracts the chord that is output by superimposing a plurality of performance sounds that are included in the score information and that have different output start times. The output start time point of the chord extracted by the means 110 and the chord extraction means 110 is set as the output start time point of at least one performance sound among the output start time points of each of the plurality of performance sounds output superimposed on the chord. As the output start time determination means 114 and the piano roll music notation 150 and 172 representing the chord, the chord with the time position corresponding to the output start time determined by the output start time determination means 114 as one end is determined. Piano roll type score display functioning as piano roll type score display control means 116 for displaying a band image 152 representing Since it is a program, for example, chords that are played by mixing a plurality of performance sounds with different output start points, such as the performance sound of a plucked string instrument, can be displayed as easy-to-understand piano roll charts 150 and 172. it can. That is, it is possible to provide a piano roll-type score display program that suitably displays the chord piano roll-type scores 150 and 172 included in the score information.

  Further, according to the present embodiment, in the chord extraction process (S5) for extracting a chord that is output by superimposing a plurality of performance sounds each having a different output start time included in the score information, and in the chord extraction process An output start time determination process for determining the output start time of the extracted chord based on the output start time of at least one performance sound among the output start times of each of the plurality of performance sounds that are output superimposed on the chord (S7) and a piano roll type score 150, 172 representing the chord, and a band image 152 representing the chord with the time position corresponding to the output start time determined in the output start time determination process as one end. Since it includes the roll type score display control process (S8), for example, a plurality of performance sounds having different output start times such as a performance sound of a plucked string instrument are mixed. The is played match chords, it can be displayed as a visually understandable piano roll type score. In other words, it is possible to provide a piano roll type score display method for suitably displaying a chord piano roll type score included in the score information.

  The preferred embodiments of the present invention have been described in detail with reference to the drawings. However, the present invention is not limited to these embodiments, and may be implemented in other modes.

  For example, in the above-described embodiment, as an example of the piano roll type music, the piano roll type music representing chords (chords) included in the musical score data (MIDI data) corresponding to the electric guitar 90 which is a plucked string instrument. Although the control for displaying 150 and 172 has been described, the present invention is not limited to this. For example, a piano that represents a chord included in the score information corresponding to the score information of a bass guitar that is a plucked string instrument. A roll type score may be displayed. In this way, it is possible to display the chords of the bass guitar that are played by mixing a plurality of performance sounds having different output start points as a piano roll score that is easy to understand visually. The present invention also includes a plucked string instrument that plays a performance sound by continuously scratching a plurality of strings, such as a ukulele, mandolin, balalaika, shamisen, cocoon, and sanshin. The present invention can be widely applied to the display of piano roll-type music representing chords included in musical score information corresponding to various musical instruments that can perform performances corresponding to the chords.

  Further, in the above-described embodiment, the piano roll type score display control means 116 displays the piano roll type scores 150 and 172 on the display 72 via the video processing unit 48. The touch panel display 58 (display device 60) may display the piano roll type music 150 and 172. In this way, the display 72 displays a performance screen related to normal karaoke performance (with the lyric character image displayed on the front side of the background image) and the touch panel as the second image display device. The piano roll type music 150 and 172 can be displayed on the display 58. In addition, information for displaying the piano roll charts 150 and 172 is transmitted to the electronic quick sample device 28 associated with the karaoke device 16 through the LAN 24, and the touch panel display of the electronic quick sample device 28 is transmitted. It is also conceivable to display the piano roll model 150, 172 on the screen.

  Further, in the above-described embodiment, as an example of a piano roll type musical score display device for displaying a piano roll type musical score indicating the scale and output timing of the performance sound included in the musical score information, it is selected from a large number of music data. The example in which the present invention is applied to the karaoke device 16 that performs performance output of music data and displays the lyric text image of the music on the video display device has been described, but the present invention is not limited to this, Naturally, another form of piano roll score display device is also conceivable. For example, the piano roll type music score display program of the present invention is installed in the control device such as the personal computer 30, the mobile phone 32, or the tablet 34, and the control device such as the personal computer 30, the mobile phone 32, or the tablet 34 The chord extraction unit 110, the chord determination unit 112, the output start time determination unit 114, the piano roll type score display control unit 116, and the like may be used. Alternatively, consumer software for executing a game such as a so-called sound game, sequence software in the field of DTM (Desk Top Music), a musical instrument performance training apparatus used exclusively to practice the performance of musical instruments such as the electric guitar 90 The present invention is preferably applied to a machine or the like. In addition, the present invention is applied to a server device that distributes moving image information and the like via a communication line, and the piano roll type score created in the server device is distributed to a plurality of communication terminal devices via the communication line. It may be done.

  In addition, although not illustrated one by one, the present invention is implemented with various modifications within a range not departing from the gist thereof.

  16: Karaoke device (piano roll-type score display device), 90: Electric guitar (external device, plucked string instrument), 110: Chord extraction means, 114: Output start time determination means, 116: Piano roll-type score display control means, 150 172: Piano roll type score, 132, 152: Band image, 158: Bar line, 160: Performance method image

Claims (5)

A piano roll type score display device for displaying a piano roll type score representing the scale and output timing of a performance sound included in the score information of a plucked string instrument ,
Chord extraction means for extracting chords for one stroke each of which is output by superimposing a plurality of performance sounds each having a different output start time included in the score information of the plucked string instrument ;
The output start time of the chord extracted by the chord extraction means is determined based on the output start time of at least one performance sound among the output start times of each of the plurality of performance sounds output superimposed on the chord. An output start time determination means;
As the piano roll score representing the chord, chords corresponding to each of the plurality of chords included in the score information are arranged on one axis, and the direction perpendicular to the one axis is set as the time position direction, and the output start time determination Piano roll-type score display control means, comprising: a piano roll-type score display control means for displaying a band image representing the chord with the time position corresponding to the output start time determined by the means as one end. The piano roll type score display control means displays a video showing how to hold down the strings indicating the position of the finger and the position of the string in the plucked string instrument corresponding to the chord, together with the piano roll type music representing the chord. The piano roll type score display apparatus according to claim 1 , wherein 2. The output start time determination means determines a bar line closest to an average value of output start times of a plurality of performance sounds that are output superimposed on the chord as an output start time of the chord. Or the piano roll type | formula score display apparatus of 2. A control device for a piano roll type score display device for displaying a piano roll type score representing the scale and output timing of the performance sound included in the score information of the plucked string instrument
Chord extracting means for extracting said included in music information of plucked string instrument, 1 stroke chords each output start time is different performance sound is output by superimposing the respective
The output start time of the chord extracted by the chord extraction means is determined based on the output start time of at least one performance sound among the output start times of each of the plurality of performance sounds output superimposed on the chord. Output start time determination means,
And as a piano roll type score representing the chord, chords corresponding to each of the plurality of chords included in the score information are arranged on one axis, and the output is started with the direction orthogonal to the one axis as the time position direction A piano roll type score display program which functions as piano roll type score display control means for displaying a band image representing the chord with the time position corresponding to the output start time determined by the time point determination means as one end. A piano roll type score display method for displaying a piano roll type score indicating the scale and output timing of a performance sound included in the score information of a plucked string instrument ,
A chord extraction process for extracting chords for one stroke each of which is output by superimposing a plurality of performance sounds each having a different output start time included in the musical score information of the plucked string instrument ;
The output start time of the chord extracted in the chord extraction process is determined based on the output start time of at least one performance sound among the output start times of each of the plurality of performance sounds output superimposed on the chord. An output start time determination process;
As the piano roll score representing the chord, chords corresponding to each of the plurality of chords included in the score information are arranged on one axis, and the direction perpendicular to the one axis is set as the time position direction, and the output start time determination A piano roll type score display method, comprising: a piano roll type score display control step of displaying a band image representing the chord with the time position corresponding to the output start time determined in the process as one end.