JP6065871B2 - Performance information display device and performance information display program - Google Patents

Description

  The present invention relates to a performance information display device.

  2. Description of the Related Art Conventionally, there is a fingering guide display technique for instructing the movement of a finger corresponding to a performance operator of a musical instrument in accordance with the progress of the performance of a musical piece for musical instrument performance practice. For example, there is a performance information display device described in Patent Document 1.

  The performance information display device described in Patent Document 1 includes a video signal (hereinafter referred to as an image) that captures a hand where a performer operates a performance operator of a keyboard instrument, and a note bar indicating a keyboard to be pressed and an operation timing. Display synchronously. Further, note bar display is started prior to the operation timing, and scroll display is performed at a position corresponding to the performance operator.

Japanese Patent Laid-Open No. 2005-196072

  When practicing a musical instrument, in general, a user often practices fingering according to music and a method of keying or plucking. In addition, the user may desire to perform like a performer using the performer's performance as a model. However, even if the performance of the musical instrument is the same chord, the timbre may differ depending on the fingering of the performer and the keying or plucking method.

  For example, when the technique described in Patent Literature 1 is applied to guitar practice, a note bar corresponding to fingering is displayed in advance, so that the user can know fingering. However, in order to perform like a model player, the picking method that changes the timbre or the method of plucking with a finger, the position of the finger that presses the string, and the amount of arm movement that changes the volume of the sound It is necessary to understand performance styles such as. However, since the video of the player's hand showing the angle of the finger holding the string, the position of the finger, and the magnitude of the arm movement is displayed only in accordance with the operation timing, the user can change the performance technique of the player. You can know it at the same time as the plucking timing. For this reason, the user cannot prepare for performance. It is also difficult to understand the performer's performance style before the timing of the guitar operation.

  In addition, when the technique described in Patent Document 1 is applied to, for example, drum practice, as in the case of a guitar, in order to perform like a performer, how to grip a drumstick that changes the tone, It is also necessary to understand in advance the magnitude of arm movement that changes the volume of the sound. However, since the video of the player's hand showing how to hold the drumstick and the magnitude of the arm movement is displayed only in accordance with the timing of the drumstick down operation, the user knows the player's performance technique. This can be done at the same time as the drum stick down operation. For this reason, as in the case of guitar practice, the user cannot prepare for performance. Also, it is difficult to understand the performer's performance style before the drum operation timing.

  It is an object of the present invention to provide a performance information display device and a performance information display program for displaying information such as a fingering image for playing a musical instrument like a performer modeled by a user.

  In order to achieve the above object, the performance information display device according to claim 1, recording information recording a performance sound obtained by playing a musical piece with a musical instrument, recording information recording a performance state of a performer, and the music Storage means for storing performance information including the identification information, and extraction means for extracting a performance timing from the recording information and extracting a fingering image corresponding to the performance timing and indicating a fingering action from the recording information; Reproducing means for reproducing the accompaniment sound of the music corresponding to the identification information; display means having a first predetermined area for displaying the fingering image; and a second predetermined area for displaying the recording information; and the accompaniment sound Display control means for causing the recording information to be displayed in the second predetermined area in synchronization with the playback tempo and to display the fingering image in the first predetermined area prior to the performance timing corresponding to the fingering image. When, Wherein the fingering image, characterized in that the two hands of the player, a still image including the arms, and fingering information.

  The performance information display device according to claim 2 is the performance information display device according to claim 1, wherein the first predetermined area is adjacent to the second predetermined area in the display area of the display means. And the display control means displays the fingering images in the first predetermined area in order of the performance timing from a position adjacent to the second predetermined area, and from the first predetermined area to the second predetermined area. The fingering image is scroll-displayed toward the area, and when the performance timing arrives, the fingering image corresponding to the arrived performance timing is displayed in the second predetermined area.

  Further, the performance information display device according to claim 3 is the performance information display device according to claim 1 or 2, wherein the extraction means obtains a performance code corresponding to the performance timing from the recording information. Generating means for extracting and generating a fingering code image according to the performance code, wherein the display control means adds the fingering code image corresponding to the performance timing to the fingering image to add the first predetermined area; It is characterized by being displayed.

  Further, the performance information display device according to claim 4 is the performance information display device according to claim 3, wherein the storage means stores the performance timing and chord information including the performance chord, Comparing means for comparing the code information with the model code information previously associated with the music, the display control means emphasizes the display of the fingering code image based on the comparison result of the comparing means Features.

  Further, the performance information display device according to claim 5 is the performance information display device according to any one of claims 1 to 4, wherein a user performance sound in which a user plays a musical piece with a musical instrument is input from the outside. And an evaluation means for comparing the performance sound with the user performance sound for evaluation, and the display control means precedes the performance timing when the evaluation means has a high evaluation. The display period for displaying the fingering image is shortened, and when the evaluation by the evaluation means is low, the display period for displaying the fingering image is extended prior to the performance timing.

  According to a sixth aspect of the present invention, there is provided a performance information display program including recording information obtained by recording a performance sound obtained by playing a musical piece by a musical instrument, recording information obtained by recording a performance state of a performer, and identification information of the musical piece. A performance information display device comprising: a storage means for storing performance information; and a display means having a first predetermined area and a second predetermined area. An extraction step for extracting a fingering image corresponding to the performance timing, a reproduction step for reproducing the accompaniment sound of the music corresponding to the identification information, and the recording information in synchronism with the reproduction tempo of the accompaniment sound. A display control step of displaying in a region and displaying the fingering image in the first predetermined region prior to the performance timing corresponding to the fingering image, and Finger image is characterized in that the two hands of the player, a still image including the arms, and fingering information.

  According to the performance information display device according to claim 1, the extraction means extracts performance timing from the recording information, and is an image showing a fingering action, and extracts a fingering image corresponding to the performance timing from the recording information. To do. The display control means displays the recording information in the second predetermined area in synchronization with the playback tempo of the accompaniment sound, and displays the fingering image in the first predetermined area prior to the performance timing corresponding to the fingering image. The fingering image is a still image including information on the player's hands, arms, and fingering. For example, the fingering image represents the position of the finger that the performer presses on the string and the pick or plucking of the string. In addition, from the movement of the arm, it can be inferred that the power to pluck the string. As a result, the user can clearly know a performance technique for performing like a performer as a model. Thereby, the user can perform a performance incorporating the performance technique of the performer as a model. Also, the performance information display program according to the sixth aspect exhibits the same effect as the performance information display apparatus according to the first aspect.

  According to the performance information display device of claim 2, the display control means scrolls the fingering image from the first predetermined area toward the second predetermined area, and when the performance timing has arrived, the display control means has arrived. A fingering image corresponding to the performance timing is displayed in the second predetermined area. The fingering image is scroll-displayed so as to overlap the recording information at the performance timing. Thereby, even when the user is gazing at the fingering image, the user can know the performance timing by predicting the timing at which the fingering images overlap. Also, as the performance timing approaches, the fingering image overlaps the recorded information, so that the user can see the finger and arm movements until the fingering indicated by the fingering image just by looking at the fingering image. Can see. Thereby, the user can know the performance timing from the fingering image, and can also see the movement of the performer's fingers and arms, so that the performance technique of the performer can be known.

  According to a third aspect of the present invention, there is provided a performance information display device comprising a generating means for generating a fingering chord image according to the chord information. The generated fingering code image is added to the fingering image and displayed. By adding the fingering chord image to the fingering image and displaying it in the first predetermined area, the performance chord and the fingering can be viewed simultaneously. Accordingly, the user can view the fingering image in a state where the performance code is understood, and thus can easily understand the fingering corresponding to the performance code. Since fingering can be easily understood, not only fingering but also more detailed information such as the positional relationship between the frets and fingers of the guitar can be looked at. This makes it easier for the user to obtain information for performing like a performer.

  When practicing a musical instrument, a user generally practices performance according to model code information, which is a model prepared in advance for music. However, there is a case where the performance of the performer who wants to serve as a model is played with an arrangement different from the model code information. When performing according to the arrangement by the performer, the user may be confused because it is different from the typical chord information for performance practice. Therefore, according to the performance information display device of the fourth aspect, the display control means emphasizes the display of the fingering code image based on the comparison result of the comparison means. By highlighting the fingering chord image, the difference between the model chord information and the chord information is clarified, and the user can know whether it is an arrangement performance by the performer or the performance of the original music. As a result, the user can perform while paying attention to the difference between the performer's performance and normal music performance.

  According to the performance information display device of the fifth aspect, the evaluation means compares and evaluates the performance of the user and the performance of the performer as a model. The evaluated result is an index indicating how close the user's performance is to the performer. When the evaluation is low, the display period of the fingering image that is displayed in advance is extended, so that the user can perform the performance while understanding the performance technique of the performer by carefully observing the performer's fingering image. If the evaluation is high, the display period of the fingering image that is displayed in advance will be shortened, so the user can confirm how far the player's performance technique was understood with the fingering image that is displayed only immediately before While playing.

It is a functional block diagram of the communication karaoke system provided with the performance information display apparatus which concerns on one Embodiment of this invention. FIG. 6 is a diagram for explaining various types of information stored in an HDD 103. It is a figure explaining the contribution code information in the analysis result preserve | saved at HDD103. 4 is a diagram for explaining a display period calculation table among tables stored in the HDD 103. FIG. 4 is a diagram for explaining a display area of a monitor. FIG. 4 is a flowchart showing main routine processing executed by a control unit 100. 3 is a flowchart showing a code information analysis process executed by another thread of the control unit 100. 5 is a flowchart of a subroutine representing display period determination processing executed by a control unit 100. It is a figure explaining the method of the scroll display of a fingering image in this embodiment. It is a figure explaining the display area of the monitor in which a fingering image is displayed about a person with a standard performance level in this embodiment. It is a figure explaining the display area of the monitor in which a fingering image is displayed about a person with a high performance level in this embodiment. It is a figure explaining the display area of the monitor in which a fingering image is displayed about a person with a low performance level in this embodiment. It is a figure explaining the extraction method of the contribution code information in case this recording information is except MIDI format in this embodiment.

  An embodiment of the present invention will be described with reference to the drawings. In this embodiment, the commander of the karaoke device is embodied as an example of the performance information display device, but the invention is not limited to this embodiment and can be implemented in various modes.

[Composition of karaoke system]
FIG. 1 is a functional block diagram of a communication karaoke system including a commander according to the present embodiment. The communication karaoke system includes a commander 1 and peripheral devices. As peripheral devices, an information terminal 31, a singing microphone 32A, a musical instrument 32B, a speaker 33, a monitor 34, and a karaoke host 35 are provided.

  The commander 1 includes a control unit 100, a ROM 101, a RAM 102, an HDD 103, an acoustic interface unit 111, an acoustic control unit 112, a video reproduction unit 121, a video control unit 122, an operation unit 131, and a communication interface unit 141. The control unit 100 is an information processing device including a CPU, and executes processing of a program stored in the ROM 101. The RAM 102 temporarily stores parameters necessary for processing in the program. The HDD 103 stores information for service provision.

  The programs stored in the ROM 101 are, for example, programs that respectively realize a main routine process shown in FIG. 6 described later, a code information analysis process shown in FIG. 7 described later, and a display period determination process shown in FIG. 8 described later. The control unit 100 constitutes a computer that executes various programs together with storage means such as the ROM 101 and the RAM 102.

  The HDD 103 is a mass storage device such as a hard disk or a solid state drive. Many pieces of MIDI format information for playing back karaoke music are stored, and other information such as guitar chords and user scoring results analyzed for providing services is stored. Further, the HDD 103 stores information such as an image and a telop to be displayed on the display screen as a model.

  The acoustic interface 111 is an acoustic input control device that receives inputs from the singing microphone 32A and the musical instrument 32B. For example, the acoustic interface 111 includes an infrared light receiving unit for receiving input from a wireless microphone, and further includes a wired microphone and a sound input terminal for inputting guitar sound. As a result, the acoustic interface 111 receives the user's performance sound and singing sound input.

  The sound control unit 112 includes a sound reproduction device that reproduces the karaoke music stored in the HDD 103, an acoustic signal synthesis device that synthesizes the input singing sound and karaoke music, and encodes the input sound into the MIDI format. Provide an encoder to perform.

  The video reproduction unit 121 includes a decoder that reproduces compressed moving image information. Video information obtained by reproducing the recording information stored in the HDD 103 is sent to the video control unit 122 and displayed in the display area of the monitor 34.

  The video control unit 122 includes a graphic engine for displaying the video information sent from the video playback unit 121 on the monitor 34. Further, the video control unit 122 receives the telop information and the information on the image to be displayed sent from the control unit 100, combines them with the video information, and outputs a graphic to the monitor 34.

  The operation unit 131 includes a touch panel, a switch, and a dial. Information input by the operation of the operation unit 131 by the user is sent to the control unit 100. The user can select a desired music piece or select an operation mode of the commander 1 by operating the operation unit 131. The commander 1 has an operation mode for playing musical instruments in addition to an operation mode for singing karaoke. The operation unit 131 sends information indicating an operation mode selection operation by the user to the control unit 100. The control unit 100 identifies the operation mode selected by the user operation according to the information sent from the operation unit 131, and starts executing the operation mode.

  The communication interface 141 is an external communication device including an infrared communication port for performing short-range communication with the information terminal 31 and a LAN port for performing Internet communication. The infrared communication port receives operation mode and music selection operation information from the information terminal 31 operated by the user, and further sends operation information to the control unit 100. The LAN port is connected to the Internet, and can communicate with the karaoke host 35 via the network.

[Configuration of data stored in HDD 103]
Next, information stored in the HDD 103 will be described with reference to FIG. The HDD 103 stores post information, music information, analysis results, and table contents.

  The posted information is information such as a performance sound when the performer has played the musical instrument previously and a video recording video, etc., and is posted to the karaoke host 35 through a network from a commander (not shown) used for the performance. It is remembered. Then, post information on the music designated by the user is transmitted from the karaoke host 35 to the commander 1.

  The posting information includes posting identification information, recording information, recording information, music identification information, performance speed information, key information, and instrument type. The post identification information is information for identifying a large number of post information. The recording information is information of a sound obtained by recording a musical performance by a musical instrument, and is information in a MIDI format in this embodiment. The recorded information is information on a moving image that records the performance status of the performer, and includes information on the performer's hands, arms, and fingering. Since the recording information and the recording information are associated with the elapsed time from the start of the performance, both can be reproduced in synchronization by matching the elapsed time. The music identification information is information for identifying the music. The performance speed information is information including a performance tempo at the time of musical instrument performance. The key information is information including the tonality of the music during the musical instrument performance. The type of instrument is information on the instrument name included in the recording information.

  The music information is information on accompaniment of the performance of the musical instrument 32B by the user. The music information includes accompaniment information that is an accompaniment sound of the music, model code information, and telop information. For example, the music information is karaoke music data distributed from the storage unit 201 of the karaoke host 35 via the Internet and stored in the HDD 103. The accompaniment information is information on the accompaniment sound of the music, and is information in the MIDI format in this embodiment. The model chord information is information including chord symbols representing guitar chords extracted in advance from event information of music accompaniment information. At the time of performance, the telop information is information for displaying lyrics, and the color of the lyrics can be changed at the color change timing included in the telop information.

  The analysis result includes post code information, performance code information, display position, and scoring information. The post code information is information obtained by extracting the guitar code from the recording information. The performance chord information is information obtained by extracting the guitar chord from the performance sound of the musical instrument by the user. The display position is information including a position where the posted code information is displayed on the screen. The scoring information is information that represents the degree of matching between the posted code information and the performance code information (hereinafter, the degree of coincidence) as an evaluation.

  The table includes a code image table and a display period calculation table. The chord image table stores a chord symbol image M31 and a fingering chord image M35 corresponding to each guitar chord. The code image table, the code symbol image M31, and the fingering code image M35 will be described later in the step of extracting the code image. The display period calculation table stores a ratio of changing the display period of the code symbol image M31, the timing bar M32, and the fingering image M33 (hereinafter, display period change ratio) corresponding to the scoring information. The display period calculation table will be described later with reference to FIG.

  In FIG. 3, the posted chord information includes a performance order, performance timing, chord symbols representing guitar chords, and a background color flag that determines the background color of the chord symbol image. The performance timing, chord symbol, and background color flag are associated with each guitar chord and arranged in the order of performance. For example, for a specific musical piece, the first guitar chord shown in FIG. 3 is associated with performance timing “6487 (mS)”, chord symbol “C # m”, and background color flag “0”. The numerical value “6487 (mS)” of the performance timing represents the elapsed time from the performance start time of the specific music to the performance timing of the guitar chord. The background color flag “0” indicates that the background color of the code symbol image M31, timing bar M32, and fingering image M33 displayed on the screen is not emphasized, and the background color flag “1” indicates the code symbol image M31. , The background color of the timing bar M32 and the fingering image M33 is emphasized.

  In FIG. 4, the display period calculation table records points (P), display period change ratios, and parameters in association with each other. The score (P) is a score obtained by evaluating the degree of coincidence between the performance of the model and the performance of the user as a performance level of the user between 0 points and 100 points. According to the score (P), the control unit 100 classifies the performance level in five stages. When the performance level of the user is classified into five levels according to the score (P), the control unit 100 determines a display period change rate according to the performance level.

  The display period change ratio indicates a ratio of changing the display period of the chord image according to the performance level. The parameter represents a parameter corresponding to the display period change ratio, and is information for determining the display position of the code image and the display period. For example, when the score (P) is in the range of more than 90 points and less than 100 points (90 <P ≦ 100), the display period change rate is “−20%”, and the corresponding parameter is converted to a real number. “−0.2”, and this ratio and parameter are associated with the score (P). Further, for example, when the score (P) is in the range of more than 70 points and 80 points or less (70 <P ≦ 80), the display period change ratio is “0%”, and the corresponding parameter is the display period change ratio as a real number. The display period is determined according to the tempo of the music. The case where the score (P) is in the range of higher than 70 points and lower than 80 points (70 <P ≦ 80) and displayed in accordance with the tempo of the music is “standard”. The calculation method of the display period using the parameter will be described in detail in the subroutine processing described later.

[Main operations of Commander 1]
Next, main operations related to the acoustic control, display control, and communication control of the commander 1 will be described.

  The operation of the acoustic interface 111 and the acoustic control unit 112 related to the acoustic control of the commander 1 will be described. The acoustic interface unit 111 includes an input port for inputting voice from the singing microphone 32 </ b> A and performance sound from the musical instrument 32 </ b> B, and outputs the input sound to the acoustic control unit 112. The acoustic control unit 112 performs a process of reproducing a music according to a reproduction request including the music identification information sent from the control unit 100 and a process of synthesizing a sound input from the acoustic interface unit 111 with an accompaniment sound. The sound signal which is the information of the sound is output to the speaker 33. In reproducing the music, the performance tempo and tonality are changed according to the performance speed information and key information stored in the HDD 103 and reproduced. The acoustic control unit 112 converts the elapsed time from the start of the performance and the performance sound into a MIDI format and sends the result to the control unit 100 during the reproduction of the music.

  The operations of the video playback unit 121 and the video control unit 122 related to the display control of the commander 1 will be described. The video playback unit 121 receives the recording information from the control unit 100 and the elapsed time from the start of playback of the music played back by the sound control unit 112, and sends the recorded information and the elapsed time to the video control unit 122 in synchronization. The video control unit 122 converts the recording information sent from the video reproduction unit 121 into information suitable for display and displays the information on the monitor 34.

  The video control unit 122 receives the telop, the fingering image M33, and the like from the control unit 100 together with information indicating the display position and the color. The video control unit 122 displays the telop received from the control unit 100, the fingering image M33, and the like at predetermined positions on the monitor 34. The display of images on the monitor 34 will be described later with reference to FIG.

  The operation of the communication interface unit 141 related to the communication control of the commander 1 will be described including the karaoke host 35. For example, when the information terminal 31 is an external operation device such as a mobile phone equipped with a remote controller, karaoke navigation, and a dedicated application, the user operates the commander 1 by operating the information terminal 31 instead of the operation unit 131. It is possible to operate. Furthermore, the communication interface unit 141 can communicate with the karaoke host 35 via the Internet. For example, when the information desired by the user is not stored in the HDD 103, the control unit 100 sends an information request to the communication interface unit 141. The communication interface unit 141 searches for the karaoke host 35 on the Internet based on the information request from the control unit 100 and transmits the information request to the karaoke host 35. When receiving the information request, the karaoke host 35 takes out the requested information from the storage unit 201 and sends it to the communication interface unit 141 of the commander 1. Information sent from the karaoke host 35 is sent from the communication interface unit 141 to the control unit 100. The control unit 100 sequentially stores information sent from the karaoke host 35 in the HDD 103. The commander 1 provides the service by downloading the information lacking in the service provision from the karaoke host 35.

[Display of image on monitor 34]
The configuration of the screen display displayed on the monitor 34 will be described with reference to FIG. The screen of the monitor 34 has a code symbol display area V11 at the upper right of the screen, a timing display area V12 below the code symbol display area V11, and a telop display area V16 below the timing display area V12. In the code symbol display area V11, the timing display area V12, and the telop display area V16, the long sides and the long sides of the screen are arranged in parallel. The recording information display area V14 is arranged at the upper left of the screen, the fingering display area V13 is in contact with the bottom of the image information display area V14, and is arranged in an L shape along the short side and the long side of the screen. The performance area V15 is arranged at the left end in the timing display area V12.

  In FIG. 5, an image and a moving image corresponding to the display area are displayed in each display area. The code symbol image M31 is an image that displays characters representing a code symbol in a square frame. The timing bar M32 is a vertically long bar-shaped image indicating the performance start timing, and is an image in which a band indicating that there is a performance after the performance starts is displayed. The fingering image M33 is fingering information indicated by the recording information at the performance timing extracted from the recording information, and is a still image including information about the performer's hands, arms, and fingering. Furthermore, a square frame having the same length as the bottom side of the fingering image is displayed at the bottom of the fingering image M33, and a code symbol corresponding to the fingering indicated by the fingering image is shown in the square frame. A fingering code image M35 is displayed. In the present embodiment, the color in the square frame of the code symbol image M31, the color of the band of the timing bar M32, and the color in the square frame of the fingering code image M35 are associated with each other and displayed in the same color.

  Each image sent from the video control unit 122 is displayed in each area arranged on the screen. In the chord symbol display area V11, chord symbol images M31 are displayed in order of performance from the left end of the area. In the timing display area V12, the timing bar M32 is scroll-displayed from the right end to the left end of the area. In the fingering display area V13, a fingering image M33 is displayed in an L shape in order of performance from an area in contact with the lower side of the recording information display area V14. In the recording information display area V14, the recording information M34 is displayed in synchronization with the performance tempo. The telop is displayed in the telop display area V16, and the display color of the telop is changed according to the color change timing.

  The size of the chord symbol display area V11 is such that the number of chord symbol images M31 displayed is the maximum when the score (P) evaluated by the user is in the range of 0 to 60 (0 ≦ P ≦ 60). It is only necessary to have a sufficient size in the length direction of the long side of the screen. For example, in the case of this embodiment, it is only necessary to have an area of a size that can display the six code symbol images M31 shown in FIG. 10C. Also, the size of the fingering display area V13 is the same as the chord symbol display area V11, when the score (P) evaluated by the user performance is in the range of 0 to 60 points (0 ≦ P ≦ 60) It is only necessary that the display area has a sufficient size so that the number of fingering images M33 displayed is the maximum. For example, in the case of the present embodiment, it is only necessary to have an area that is large enough to display the five fingering images M33 shown in FIG. 10C.

  In a period in which each image is displayed in each display area, when the timing bar M32 is displayed from the right end of the area in the timing display area V12, each corresponding image is displayed. In the timing display area V12, when the timing bar M32 passes the left end of the area, the display of the portion that has passed is erased. When the band of the timing bar M32 finishes passing through the left end of the area, the chord symbol image M31 and the timing bar M32 are erased as if the guitar chord corresponding to the timing bar M32 has been played. In this embodiment, since the fingering image M33 is an image extracted at the performance timing, the fingering image M33 and the fingering code image are erased at the moment when the timing bar M32 enters the performance area V15.

[Description of each process in the flowchart]
The operation and action of the present embodiment will be described with reference to FIGS. 6, 7, and 8. FIG. The process of each step in the flowcharts shown in FIGS. 6, 7, and 8 is a process executed by the control unit 100.

(Main routine processing)
When the user performs an operation mode selection operation using the operation unit 131 or the information terminal 31 and selects an operation mode for musical instrument performance using post information as a model, the processing of the main routine shown in FIG. 6 is started.

  In step S111, post information modeled by the user is selected according to the user's operation. Specifically, the operation unit 131 or the information terminal 31 sends the music identification information and the type of musical instrument to the commander 1 in accordance with a user operation. The commander 1 sends a posting information request including the music identification information and the type of musical instrument to the karaoke host 35. The karaoke host 35 receives the post information request from the commander 1 and extracts all pieces of post information that matches the music identification information and the type of the instrument from the large number of post information stored in the storage unit 201. . The karaoke host 35 transmits post identification information, representative images of recording information (details will be described later), performance speed, and key information of the extracted post information to the commander 1. The representative image is an image that is displayed on the monitor 34 as a part of the posted information as a preview image before downloading the posted information in order to select one posted information as a model from the extracted posted information. It is. For example, as the representative image, the image having the earliest display start time corresponding to the image from the start of the performance may be used in the recording information. When the commander 1 receives a representative image of the same music from the karaoke host 35, the representative image is stored in the HDD 103, and the stored representative image is displayed on the monitor 34. While viewing the displayed representative image, the user operates the operation unit 111 or the information terminal 31 to designate one post information as a model. In accordance with the user's operation, post information as a model is selected. Thereafter, the process proceeds to step S112.

  In step S112, it is determined whether post information has already been stored in the HDD 103 (S112A), and post information selected in accordance with a user operation is downloaded to the HDD 103 (S112B). If the selected posting information is already stored in the HDD 103, the posting information stored in the HDD 103 is used without downloading. Therefore, if the selected post information is already stored, the process proceeds to step S113. If the selected posting information is not stored in the HDD 103, the selected posting information is downloaded from the karaoke host 35 to the HDD 103, and when storing the posting information in the HDD 103 is completed, the process proceeds to step S 113.

  In step S113, it is determined whether or not the music information corresponding to the music identification information included in the posted information is already stored in the HDD 103 (S113A), and the selected music information is downloaded to the HDD 103 (S113B). . If music information corresponding to the music identification information is already stored in the HDD 103, the music information stored in the HDD 103 is used without downloading. Therefore, if music information corresponding to the selected music identification information is stored in the HDD 103, the process proceeds to step S114. When the music information corresponding to the music identification information is not stored in the HDD 103, the music information corresponding to the music identification information is downloaded from the karaoke host 35 to the HDD 103, and when the storage in the HDD 103 as the music information is completed, the process proceeds to step S114. Processing is transferred.

  In step S114, the start of a subroutine process (see FIG. 7) for analyzing the guitar chord in the posted information is instructed. The processing of the subroutine whose start is commanded in step S114 is performed in parallel with the processing of the main routine. In the guitar chord analysis of the subroutine shown in FIG. 7, post chord information is extracted from post information, and a chord image and a fingering image corresponding to the chord symbol of the extracted guitar chord are generated. Also, the posted code information and the model code information are compared, and a background color flag is set according to whether or not they match each other. Subroutine processing is performed until analysis of all sections of the posted information is completed. Subroutine processing will be described later with reference to the flowchart of FIG.

  In step S115, whether or not to perform the accompaniment sound minus one is selected according to the user's operation. Specifically, the user operates the operation unit 131 or the information terminal 31 to determine whether or not to reproduce the musical instrument part to be played and the recorded information of the posted information. When it is selected whether or not to perform minus one according to the user's operation, the process proceeds to step S116.

  In step S116, it is determined whether or not the guitar chord analysis for a certain section is completed by the processing of the guitar chord analysis subroutine instructed to start in step S114. When the analysis of the certain section is completed (S116: Y), the process is moved to step S121. On the other hand, when the analysis of the certain section is not completed (S116: N), the process of step S116 is performed until the analysis of the certain section is completed.

  In step S116, the fixed section is a section in which post code information that is one more than the number of fingering images M33 displayed in the fingering display area V13 for the person with the lowest evaluation is extracted, for example. In the present embodiment, the maximum number of fingering images M33 displayed is five. However, one more code symbol image M31 is displayed than the fingering image M33 at the performance timing. Therefore, if post code information including six chord symbols and six performance timings can be extracted from the post information, it is determined that the analysis of a certain section has been completed.

  In step S121, scoring information is initialized. Since the scoring information determines the ratio of changing the display period, the code image display period at the start of the performance is determined by initialization. In this embodiment, in order to set the display period to the standard display period, the initial score of scoring information is 75 points, and the initial score is stored in the HDD 103. When the storage of the scoring information is completed, the process proceeds to step S122.

  In step S122, the video reproduction unit 121 is instructed to reproduce the recorded information, and the sound control unit 112 is instructed to reproduce the accompaniment information. Specifically, a reproduction request is sent to the acoustic control unit 112 to reproduce the accompaniment information. The sound control unit 112 reproduces accompaniment information in accordance with a reproduction request from the control unit 100. The playback tempo at this time may be a tempo defined in the accompaniment information or a performance speed included in the posted information. When reproducing at the tempo defined in the accompaniment information, the control unit 100 and the sound control unit 112 may reproduce according to the performance timing included in the accompaniment information. However, when reproducing according to the performance speed information included in the posted information, the control unit 100 calculates a change rate indicating how much the performance tempo changes from the reference tempo from the performance speed information, and uses the change rate in the accompaniment information. Adapt and play. The sound control unit 112 sends an elapsed time from the start of reproduction to the control unit 100 when the reproduction is started. In order to reproduce the recording information, the elapsed time and the recording information are sent to the video reproduction unit 121. The video reproduction unit 121 associates the elapsed time from the control unit 100 with the recording information, and sends it to the video control unit 122. Further, in order to display a telop according to the elapsed time from the start of reproduction, the elapsed time and telop information are sent to the video control unit 122. The video control unit 122 synthesizes the telop from the control unit 100 and the recording information, converts the information into information suitable for display, and sends the information to the monitor 34. The monitor 34 displays an image on the screen according to the video information.

  In step S131, it is determined whether or not the performance of the music requested to be played in step S122 has been completed. If the accompaniment information and the recorded information have been reproduced (S131: Y), the process proceeds to step S140. However, if the accompaniment information and the recorded information have not been reproduced (S131: N), the process proceeds to step S132.

  In step S132, an instruction to start a subroutine process for determining a code image display period is given. In this subroutine, processing for determining the moving speed of the display position of the timing bar M32 is mainly performed according to the contents of the scoring information. Thereby, the display period of the code image is determined. Furthermore, a process of determining a display position corresponding to the elapsed performance time for the display period of the chord image is performed. Detailed processing of the subroutine for which activation is instructed in step S132 will be described later with reference to the flowchart of FIG.

  In step S133, the display is updated over time. In step S133, the display position of the code symbol image M31, the display position of the timing bar M32, and the display position of the fingering image M33 are updated according to the information on the display position of the timing bar M32 determined in step S132. Further, the color of the code symbol image M31, the color of the timing bar M32, and the color of the fingering code image M35 are displayed in colors corresponding to the background color flag. Specifically, the elapsed time from the start of reproduction of the accompaniment information requested for reproduction in step S122 is sent from the acoustic control unit 112. When display is performed in accordance with the elapsed time, the code symbol image M31, the timing bar M32, the fingering image M33, the display position, and the background color flag are sent from the control unit 100 to the video control unit 122.

  The video controller 122 displays the timing bar M32 at the display position determined in step S132 in the timing display area V12. However, when the display position of the timing bar M32 is outside the timing display area V12, or when the performance timing has passed, the image of the timing bar M32 and information on the display position are sent to the video control unit 122 by the control unit 100. Therefore, it is not displayed on the monitor 34.

  Next, a code symbol image M31 corresponding to the timing bar M32 is displayed in the code symbol display area V11. The code symbol image M31 is sent to the video control unit 122 by the control unit 100 so that the same number as the timing bar M32 is displayed including the one displayed in the performance area V15. The chord symbol image M31 displayed in the chord symbol display area V11 is displayed in order from the left end to the right end of the area except for one displayed in the performance area V15 in the performance order corresponding to the timing bar M32. When the display of the timing bar M32 is started, the display of the corresponding code symbol image M31 is also started. When the display of the timing bar M32 is turned off, the display of the corresponding code symbol image M31 is also turned off.

  For example, as shown in FIG. 5, when four timing bars M32 are displayed in the timing display area V12, four code symbols “F # m”, “D7”, “A7”, and “C” are displayed on the screen. An image M31 is displayed. At the timing of playing the instrument, the display area of the chord symbol image M31 moves from the chord symbol display area V11 to the performance area V15, and the timing bar M32 also moves to the performance area V15. In the performance area V15, the chord symbol image M31 and the display of the timing bar M32 overlap. Except for the chord symbol image M31 of “F # m” displayed in the performance area V15 indicating the timing of playing the instrument, the chord symbol display area V11 has three symbols “D7”, “A7”, and “C”. A code symbol image M31 is displayed. When the performance timing has passed and the timing bar M32 corresponding to “F # m” is erased from the timing display area V12, the chord symbol image M31 corresponding to “F # m” is also erased. When the display of the chord symbol image M31 of “F # m” is erased from the performance area V15, the next chord symbol image M31 of “E” is displayed in the performance area V15, and the chord symbol images of “A7” and “C” are displayed. The display position of M31 is moved by one to the left.

Next, the fingering image M33 is displayed in the fingering display area V13 in the order of performance along the L-shape of the fingering display area V13 from a position close to the recording information display area V14. Regarding the display of the fingering image M33, when the timing bar M32 is displayed, the corresponding fingering image M33 is displayed. The fingering image is stopped and displayed in the fingering display area V13. However, as the performance timing approaches, the fingering image M33 is scroll-displayed toward the recording information display area V14.
The display of the fingering image M33 will be described later with reference to FIG.

(Display control by background color flag)
The colors of the code symbol image M31, the timing bar M32, and the fingering image M33 are determined by the background color flag. When the background color flag is “0” in the later-described guitar chord analysis subroutine processing, the posted code information matches the model code information, and white indicating normal display is used as the background color. To the video controller 122. When the background color flag is “1”, the posted code information and the model code information do not match, and blue indicating highlighting is sent from the control unit 100 to the video control unit 122 as a background color. The video control unit 122 changes the background color of the code symbol image M31, the band color of the timing bar M32, and the background color of the fingering code image M35 according to the color information, and displays them on the monitor 34.

  In step S134, performance chord information is extracted from the user's performance sound. The performance sound of the user's musical instrument 32B is sent to the acoustic control unit 112 via the acoustic interface 111. The sound control unit 112 generates a sound signal by synthesizing the performance sound of the music by the user's musical instrument 32 </ b> B with the accompaniment sound, and sends the sound signal to the speaker 33. Furthermore, the sound control unit 112 converts the input performance sound into a MIDI format and sends it to the control unit 100. In the guitar chord analysis performed by the control unit 100, the performance timing and the guitar chord are extracted from the performance sound converted into the MIDI format, and performance chord information is generated. The generated performance chord information is stored in the HDD 103 by adding the performance timing and guitar chord further extracted with the passage of the performance time to the performance chord information.

  In step S135, the score information of the musical performance of the user's musical instrument is calculated according to the comparison result (comparison procedure will be described later) comparing the degree of coincidence between the performance code information and the posted code information. The performance timing and chord symbol information of the performance chord information are respectively compared with the performance timing and chord symbol information of the posted chord information, and a score is calculated according to the degree of coincidence between the performance chord information and the posted chord information. Specifically, at the performance timings of the performance chord information and the posted chord information, 0 points are used when the chord symbols are different, and 100 points when the performance timings and chord symbols are the same. If the chord symbols match at different performance timings in the performance chord information and the posted chord information and the performance timings are different, points are deducted as the time difference between the performance timings increases. Furthermore, it may be calculated by a calculation method in which the deduction points become larger as the number of appearances of non-matching code symbols in the scored section increases. For example, in the case of scoring from 6 chord symbols and performance timing in performance chord information and post chord information, 3 out of 6 are 100 points, 2 are 60 points, 1 is 0 points Then, {600− (100−100) × 3− (100−60) × 2− (100−0) × 1} ÷ 6 = 70. At this time, the scoring information becomes 70 points and is stored in the HDD 103. When step S135 is completed, the process proceeds to step S131.

  In step S131, it is determined whether or not the music performance has ended. If the performance of the music has not ended (S131: N), the processing from step S132 to step S135 is performed. Through the processing from step S132 to step S135, the posted code information and scoring information are updated, and the display is also updated. The display period of the timing bar M32 is calculated according to the updated scoring information, and the length of time for displaying the post code information as a model is changed while maintaining the performance speed. When the performance of the music is finished (S131: Y), the process is moved to step S140.

  In step S140, the musical performance of the user's musical instrument is evaluated. The calculation of the scoring information in step S135 is the calculation of scoring information during the performance. In step S140, the final scoring information for one piece of music is calculated. The final scoring information is displayed on the screen to quantitatively inform the user of the performance level. Further, the final scoring information may be sent to the karaoke host 35 in association with the user identification information of the user and stored in the storage unit 201. When the process of step 140 is completed, the process of the main routine ends.

(Guitar chord analysis subroutine processing)
Next, the guitar chord analysis subroutine process started in step S114 shown in FIG. 6 will be described with reference to FIG. The subroutine shown in FIG. 7 performs guitar chord analysis of post information, and generates post code information and a chord image to be displayed as a model.

  In step S201, performance timing and guitar chord are extracted from the recording information. In the present embodiment, since the recording information is MIDI format information, performance timing and chord symbols are extracted from the MIDI event information. In step S201, a guitar chord is extracted and stored in the HDD 103 as post chord information. When the post code information is stored, the process proceeds to step S202.

  In step S202, posted code information and model code information are compared. The chord symbols at the respective performance timings of the posted code information and the model code information are compared. The processing in the next step S203 branches depending on the code symbol comparison result.

  If the code symbol of the posted code information matches the code symbol of the model code information (S203: N), the process proceeds from step S203 to step S204. In step S204, since the result of comparing the code symbol of the post code information and the code symbol of the model code information matches, the background color flag is set to “0” indicating normal display. Further, a code symbol image M31, a timing bar M32, and a fingering code image M35 that match the code symbol in the posted code information are extracted from the code image table and stored in the RAM 102.

  If the code symbol of the posted code information is different from the code symbol of the model code information (S203: Y), the process proceeds from step S203 to step S205. In step S205, since the result of comparing the code symbol of the post code information and the code symbol of the model code information is different, the background color flag is set to “1” indicating the highlighted display. Further, a code symbol image M31, a timing bar M32, and a fingering code image M35 that match the code symbol in the posted code information are extracted from the code image table and stored in the RAM 102.

  When the process of step S204 or step S205 ends, the process proceeds to step S206. In step S206, a fingering image M33 is extracted from the recording information. In step S201, the fingering image M33 is extracted according to the performance timing included in the post code information extracted from the recording information. The performance timing is determined by the elapsed time from the start of reproduction of the recorded information, and therefore coincides with the elapsed time from the start of reproduction of the recorded information. Thus, the fingering image M33 is extracted from the recording information according to the performance timing and stored in the RAM 102. When the fingering image M33 is stored in the RAM 102, the process proceeds to step S207.

  In step S207, it is determined whether or not the guitar chord analysis for one song has been completed in the processing from step S201 to step S206. If it is determined that the analysis process for one song has not been completed (S207: N), the process returns to step S201, and the portion of the recording information that has not been analyzed for the guitar chord is analyzed. If it is determined that the guitar chord analysis for one piece of music has been completed (S207: Y), the processing of the guitar chord analysis subroutine ends. As a determination method in step S207, it is determined that the guitar chord analysis for one piece of music is completed when the performance timings are the same by comparing the last performance timings of the recording information and the posted chord information. Since the execution time of the series of processing from step S201 to step S207 is sufficiently shorter than the performance time for one music piece, the series of processing from step S201 to step S207 is repeatedly executed.

(Display period determination subroutine processing)
Next, the display period determination subroutine (S132) will be described with reference to FIG. The subroutine shown in FIG. 8 determines the scroll speed and display position of the timing bar M32 using the scoring information. The display period determining subroutine determines the number of chord images to be displayed and the display speed of the timing bar M31 according to the performance level of the user.

  In step S301, the scoring information is read and stored in the RAM 102. The scoring information is the scoring information calculated in step S135 and stored in the HDD 103. When the scoring information is stored in the RAM 102, the process proceeds to step S302.

  In step S302, parameters are extracted from the display period calculation table according to the scoring information stored in the RAM 103 in step S301. For example, as the scoring information, when step S302 is executed for the first time, the initial value of 75 points is set. The parameter corresponding to 75 points is “0”, and “0” is stored in the RAM 102 as a parameter.

In step S303, the scroll speed of the timing bar M32 is calculated using the parameters stored in the RAM 102 in step S302.
V0 = X15 / T (1)
V = V0 × (1-P) (2)
In the arithmetic expressions (1) and (2), V0 is a reference scroll speed of the timing bar M32. X15 is the number of pixels in the left-right direction within the range set in the performance area V15. T is an average performance time obtained by calculating an average length of performance time among performance sounds in the music. V is the scroll speed of the timing bar M32. P is a parameter stored in the RAM 102 in step S302. For example, when the parameter stored as the initial value is “0”, the scroll speed V of the timing bar M32 is the reference scroll speed V0, and the display period at this time is the reference display period. The parameter range is -20% of the reference display period when the maximum score is 100 points, so the parameter is "-0.2", and when the minimum score is 0, the parameter range is relative to the reference display period. Since the display period is + 20%, the parameter is “0.2”.

  In this embodiment, the standard speed V0 is a solution obtained by dividing the number of pixels X15 in the X-axis direction by the average performance time T from the arithmetic expression (1). The scroll speed V is calculated by multiplying the standard speed V0 by the difference obtained by subtracting the parameter P determined from the scoring information from the value “1”. The X-axis direction is the horizontal axis direction of the performance area V15 in FIG.

  The calculated scroll speed V is stored in the RAM 102 and used as the scroll speed of the timing bar M32 until step S303 is executed next time. However, when the accompaniment information is reproduced according to the performance speed information in step S122, the speed V1 is used instead of the standard speed V0 in the calculation formula (2). The speed V1 is calculated by multiplying the change rate calculated from the performance speed information by the standard speed V0.

  Assuming that the number of pixels in the X-axis direction of the timing display area V12 is X12, dividing the number of pixels X12 by the scroll speed V displays the code symbol image M31, timing bar M32, and fingering image M33 at the scroll speed V. It becomes a period.

In step S304, the display position of the timing bar M32 is calculated. (The right side of the screen shown in FIG. 5 is the positive direction.)
X = X0 + V × (TON−TNOW) (3)
In the calculation formula (3), X is a display position of the timing bar M32. X0 is the number of pixels in the left-right direction from the left end of the display screen of the monitor 34 to the left end of the timing display area V12. V is the scroll speed obtained in step S303. TON is the performance timing of the posted code information. TNOW is the elapsed time from the start of performance to the present time.

  In the present embodiment, the display position X of the timing bar M32 is calculated by adding the display position of the timing bar M32 in the timing display area V12 to X0 indicating the number of pixels at the left end of the timing display area V12 from the left end of the display screen. . The display position of the timing bar M32 in the timing display area V12 represents the coordinates of the display position determined with reference to the left end of the timing display area V12. The difference between the performance timing TON and the elapsed time TNOW is multiplied by the scroll speed V. Is calculated.

  The display position of the timing bar M32 corresponding to the chord symbol of each performance timing in the posted code information is stored in the HDD 103, and the process returns to step S133.

(Scroll display of fingering image M33)
In FIG. 9, an example of the scroll display of the fingering image M33 performed in step S133 will be described. The minimum performance time with the shortest performance time among the performance sounds in the music is extracted from the posted code information and stored in the RAM 102. The timing for starting the scroll display of the fingering image M33 at each performance timing is set earlier than the performance timing by the minimum performance time. The speed of scroll display is determined by the distance between the bottom of the fingering image M33 to be displayed and the bottom of the recorded information M34 and the minimum performance time.

  FIG. 9 shows an enlarged view of the periphery of the boundary between the fingering display area V13 and the recording information display area V14 where the fingering image M33 and the recording information M34 are displayed. Elapsed times T0, T1, and T2 represent elapsed times from the start of performance. The elapsed time T2 is the performance timing of the guitar chord “D7” indicated by the fingering image M33. The elapsed time T1 is a timing that is earlier than the performance timing T2 of the guitar chord “D7” indicated by the fingering image M33 by a minimum performance time, and is a timing at which scrolling is started. The elapsed time T0 is the timing when the performance of the guitar chord before the guitar chord “D7” indicated by the fingering image M33 is finished, and is earlier than the elapsed time T1. At the elapsed time T0, the control unit 100 sends a control command to the video control unit 122. In accordance with this control command, the video control unit 122 deletes the display of the guitar chord after the performance timing and displays the fingering image M33 in a stationary state. Next, when the elapsed time T1 is reached, the control unit 100 sends a movement command to a new display position to the video control unit 122. In accordance with this movement command, the video control unit 122 shows the fingering image M33 corresponding to the guitar chord “D7” to be played next, and the fingering chord image M35 indicated by a two-dot chain line toward the recording information M34. The scroll display is started so as to move in the direction of 41. At this time, a new display position of the fingering image M33 is calculated by the control unit 100. When the elapsed time T2 is reached, the guitar chord “D7” becomes the performance timing, and the fingering image M33 corresponding to the guitar chord “D7” reaches the position where it overlaps with the recording information M34. When the elapsed time T2 is reached, the control unit 100 sends a control command to the video control unit 122. In accordance with this control command, the video control unit 122 deletes the display of the fingering image M33 corresponding to the performance timing T2. Further, the video control unit 122 stops the scrolling display of the fingering image M33 that has been scrolled, and displays the fingering image in a stationary state until a timing that is earlier than the performance timing of the next guitar chord by a minimum performance time. Let

When the scroll display shown in FIG. 9 is performed, the display position of the fingering image M33 is calculated in the control unit 100 as follows. (The screen upward direction shown in FIG. 9 is the positive direction.)
V33 = (Y34−Y33) / (Tmin) (4)
Y = Y33 + V33 × (TON−TNOW)) (5)
As shown in the arithmetic expression (4), in the state where the fingering image M33 corresponding to the guitar chord to be played next is displayed statically, from the display position (Y33) of the bottom side of the fingering image M33, the bottom side of the recording information M34 is displayed. The scroll speed (V33) is calculated by dividing the distance to the display position (Y34) by the minimum performance time (Tmin). Further, the display position (Y) of the fingering image M33 during the scroll display is a time difference obtained by subtracting the elapsed time TNOW from the performance timing TON with reference to the display position (Y33) of the bottom side of the fingering image M33 in the stationary display state. Is multiplied by the scroll speed V33.

[Specific Example of Embodiment]
In the performance information display device of the present embodiment, the display state of the monitor 34 is determined according to the performance level of the user. With respect to the display state of the monitor 34, the performance level of the user is classified into three levels: “standard”, “high”, and “low”. Each of the three-stage display states will be described with reference to FIGS. 10A to 10C where the elapsed time from the start of the performance is the same.

(Person with a standard performance level)
A person with a standard performance level represents a person whose scoring information is higher than 70 points and in the range of 80 points or less. For example, in the case of a person who is performing at a performance level where the scoring information is 72 points, the display period change rate is 0%, so the scroll speed of the timing bar M32 is the tempo of the music and is the standard speed V0. The screen is displayed at a standard speed V0 as shown in FIG. 10A.

(Person with high performance level)
A person with a high performance level represents a person whose scoring information is higher than 80 points and in the range of 100 points or less. For example, in the case of a person who performs with 90 points of scoring information, the display period change rate is −20%, so the scroll speed of the timing bar M32 is 20% faster than the standard speed V0. That is, the display on the screen is as shown in FIG. 10B, and the display period of the fingering image M33 is shorter than the standard case of FIG. 10A, and the number of fingering images M33 that can be viewed in advance is reduced.

(Person with low performance level)
A person whose performance level is low represents a person whose scoring information is 70 points or less. For example, in the case of a person who is playing with 55 scoring information, the display period change rate is 20%, so the scroll speed of the timing bar M32 is 20% slower than the standard speed V0. That is, the display on the screen is as shown in FIG. 10C, the display period of the fingering image M33 is longer than in the standard case of FIG. 10A, and the number of fingering images M33 that can be viewed in advance is increased.

(Example when scoring information is updated)
As the performance of the music progresses, the scoring information also changes, so the display on the screen changes accordingly. For example, when the scoring information is 75 points and the performance level is in the display form of a standard person, if the scoring information is updated and the score increases to 90 points, the display state of the monitor 34 is a person with a high performance level. The display format is changed. On the other hand, when the score is reduced to 60 points when the scoring information is updated, the performance level is changed from the normal person display form to the low performance level person display form. The display state of the monitor 34 changes every time the scroll speed is changed by the update of the scoring information and the calculation of the scroll speed. In this embodiment, the display period does not change for 60 points or less.

[Modification]
(Modification 1)
In the present embodiment, the recording information is information in the MIDI format. However, even if it is a raw sound (for example, information such as PCM or MPEG audio), post code information can be extracted by the following procedure. When the volume level of the performance sound is determined in advance, and the volume level of the performance sound equal to or higher than the threshold is detected, the performance timing of the posted code information is determined based on the performance. A well-known frequency analysis is performed during the performance period from the performance timing of the posted code information until the level of the performance sound falls below the threshold, and the frequency component at the performance timing of the posted code information is specified. Furthermore, the chord symbol of the guitar chord corresponding to the specified frequency component is specified. The identified chord symbol and the performance timing of the posted code information are associated with each other and stored in the HDD 103 as posted code information. The above procedure is stored in the ROM 101 as a program executed by the control unit 100, and the control unit 100 can extract post code information from the recorded sound information by executing the above procedure. As a result, it is possible to extract code information from the raw sound and create a fingering image M33.

Specifically, the description will be made with reference to the graph shown in FIG. 11 showing the time change of the voltage level of the recording information of the live sound. Since the recording information is input if the performance is performed, it is determined that the performance timing of the posted code information is input when the recording information exceeding the threshold is input. Each performance timing is indicated as t1, t2, and t6.
Since the voltage level of the recording information exceeds the threshold at the performance timing t1, the performance timing of the posted code information is determined as the performance timing t1. The period from the performance timing t1 to below the threshold is defined as the performance period of the chord symbol “C # m”, and frequency analysis is performed during the performance period. As an example of the frequency analysis method, fast Fourier transform is performed as an example, and a frequency component is specified. A chord is specified by the specified frequency component, and a chord symbol “C # m” indicating a guitar chord is specified from the chord. The performance timing t1 is associated with the chord symbol “C # m” and stored as post code information. Similarly, guitar chords are specified and stored in the HDD 103 at performance timings t2 and t6. t1, t2, and t6 correspond to the performance timings of the posted code information in FIG. 3, “6487 (mS)”, “18318 (mS)”, and “25923 (mS)”. A fingering image M33 is also extracted based on the stored post code information.

(Modification 2)
In this embodiment, the fingering display area V13 displays five fingering images M33 prior to the performance timing of the chord symbol “F # m” as shown in FIG. 10C for a person with a low performance level. However, the number of displays is not limited. It is sufficient that the display number increases in the order of a person with a high performance level, a person with a standard performance level, and a person with a low performance level, and the fingering display area V13 has a size that can display the fingering image M33 of this display number. I just need it. Further, the size of the code symbol display area V11 in the left-right direction can be determined in the same manner as the size of the fingering display area V13.

(Modification 3)
In the present embodiment, the code symbol image M31 and the timing bar M32 are displayed on the monitor 34 according to the fingering image M33, but the code symbol image M31 and the timing bar M32 may not be displayed. Further, the display form of the monitor 34 may be the fingering display area V13 and the recording information display area V14, and the recording information display area V14 may be arranged in the center, and there may be a fingering display area V13 surrounding it. There may be a recording information display area V14 at the bottom of the screen, and there may be a fingering display area V13 in contact with that area toward the top of the screen.

(Modification 4)
In the present embodiment, in the scoring information initialization process (S121), the initial value is set to 75 points so as to obtain a standard display form as shown in FIG. 10A. Scoring information may be extracted from the history, and the score may be used as scoring information. By performing initialization with the user's performance history, the display of the fingering image M33 corresponding to the user's performance level can be provided from the start of the performance. For example, as an initial value based on the performance history, an average value, a maximum value, or a minimum value of performance history evaluation can be considered.

(Modification 5)
In the present embodiment, when highlighting is performed, the posted code information is compared with the model code information, and the case where they do not match is highlighted, but the case where they match may be highlighted. In addition, the background color is set to blue so that the display is emphasized to the user as compared to normal white, but various colors may be used. In order to display it with emphasis, in addition to changing the background color of the code image, the code image blinks, the code image is trimmed, the code image is bolded, Or you may perform emphasis by either transforming a code image or the combination.

(Modification 6)
In the present embodiment, when performing with changing the tonality of the music, the chord symbols are converted with reference to how much the tonality of the music has been changed from the key information including the tonality of the music. For example, the HDD 103 includes a chord conversion table for converting guitar chords according to key information, and the chord conversion table includes a guitar chord conversion table according to key information. In accordance with the conversion table of the code conversion table, the code symbol of the posted code information is converted according to the key information and stored in the HDD 103. Similarly, in step S202, the code symbol of the model code information is converted according to the code conversion table, and the posted code information and the converted model code information are compared.

  As a processing method of scoring information calculation (S135), scoring information may be calculated including not only performance timing and chord symbol information but also performance technique information such as staccato. In the present embodiment, the performance timing and the degree of coincidence of chord symbols are calculated from the performance code information and the posted code information, and the scoring information is calculated from the degree of coincidence. However, as a method for calculating the scoring information, not only the guitar chord extracted from the performance sound but also scoring or scoring according to the performance technique may be performed. By adding a performance technique to the scoring item, the scoring information is a detailed evaluation of the performance level. In order to realize this, a performance technique is extracted from the posted information and stored in the posted code information. Further, the performance technique is extracted from the chord image and stored in the performance chord information. By evaluating the performance level in more detail, it is possible to realize a display form corresponding to a more accurate performance level of the user.

[Correspondence between Configuration of Claim and This Embodiment]
The correspondence relationship between the configuration described in the present embodiment and the configuration described in the claims is as follows. The monitor 34 is an example of a display unit. Of the processes executed by the control unit 100, steps S132 and S133 are examples of display control means and display control steps, step S135 is an example of evaluation means, steps S201 and S206 are examples of extraction means, and extraction steps. , And step S204 and step S205 are an example of the generating means. The HDD 103 is an example of a storage unit. The acoustic interface 111 is an example of an input unit. The sound control unit 112 is an example of a reproduction unit and a reproduction step. The music identification information is an example of identification information. The posted information is an example of performance information. Posted code information is an example of code information. Accompaniment information is an example of an accompaniment sound. The fingering display area V13 is an example of a first predetermined area, and the recording information display area V14 is an example of a second predetermined area. The scoring information calculated in step S135 is an example of the evaluation result.

DESCRIPTION OF SYMBOLS 1 ... Commander, 100 ... Control part, 101 ... ROM, 102 ... RAM, 103 ... HDD, 111 ... Sound interface, 112 ... Sound control part, 121 ... Video reproduction part, 122 ... Video control part, 131 ... Operation part, 141. Communication interface

Claims (6)

Storage means for storing performance information including recording information for recording a performance sound of a musical piece played by a musical instrument, recording information for recording a performance status of the performer, and identification information of the musical piece;
Extraction means for extracting a performance timing from the recording information and extracting a fingering image corresponding to the performance timing, which is an image showing a fingering action from the recording information;
Reproducing means for reproducing the accompaniment sound of the music corresponding to the identification information;
Display means having a first predetermined area for displaying the fingering image and a second predetermined area for displaying the recording information;
The recording information is displayed in the second predetermined area in synchronization with the playback tempo of the accompaniment sound, and the fingering image is displayed in the first predetermined area prior to the performance timing corresponding to the fingering image. Display control means;
With
The performance information display device, wherein the fingering image is a still image including information on both hands, arms, and fingering of the performer. The first predetermined area is disposed adjacent to the second predetermined area in the display area of the display means,
The display control means includes
Displaying the fingering images in the first predetermined area in order of the performance timing from a position adjacent to the second predetermined area;
Scrolling the fingering image from the first predetermined area toward the second predetermined area;
The performance information display device according to claim 1, wherein when the performance timing has arrived, the fingering image corresponding to the performance timing that has arrived is displayed in the second predetermined area. The extraction means extracts a performance code corresponding to the performance timing from the recording information,
Generating means for generating a fingering code image according to the performance code;
The display control means includes
The performance information display device according to claim 1 or 2, wherein the fingering code image corresponding to the performance timing is added to the fingering image and displayed in the first predetermined area. The storage means stores the performance timing and chord information including the performance chord,
Comparing means for comparing the code information with the model code information previously associated with the music,
The display control means includes
The performance information display device according to claim 3, wherein display of the fingering chord image is emphasized based on a comparison result of the comparison means. An input means for externally inputting a user performance sound when the user has played a musical piece with an instrument;
An evaluation means for performing an evaluation by comparing the performance sound and the user performance sound;
With
The display control means includes
If the evaluation of the evaluation means is high, shorten the display period for displaying the fingering image prior to the performance timing,
5. The performance information display device according to claim 1, wherein when the evaluation by the evaluation unit is low, a display period for displaying the fingering image is extended prior to the performance timing. . Storage means for storing performance information including recording information for recording a performance sound of a musical piece played by a musical instrument, recording information for recording a performance status of a performer, and identification information of the music piece, a first predetermined area, A display means having a second predetermined area, and a computer of a performance information display device comprising:
An extraction step of extracting performance timing from the recording information and extracting a fingering image corresponding to the performance timing from the recording information;
A reproduction step of reproducing an accompaniment sound of the music corresponding to the identification information;
The recording information is displayed in the second predetermined area in synchronization with the playback tempo of the accompaniment sound, and the fingering image is displayed in the first predetermined area prior to the performance timing corresponding to the fingering image. Display control step, and
The performance information display program according to claim 1, wherein the fingering image is a still image including information on both hands, arms, and fingering of the performer.