JP2022164934A - Program, method, electronic apparatus, and musical performance data display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a change of the color tone of a picture created by playing a music.

SOLUTION: The program of the present invention causes an information processor to determine at least one of a plurality of types of attributes on the basis of input musical performance data, to generate a first image including a first color according to the determined type, to generate a second image including a second color according to a type which is not the determined type; and to display the first image and the second image in a display device.

SELECTED DRAWING: Figure 3

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to a program, method, electronic device, and performance data display system.

Electronic musical instruments such as digital keyboards are equipped with processors and memories, and can be said to be embedded computers with keyboards. There is also known a model that can be connected to an information processing device such as a tablet through an interface such as a USB (Universal Serial Bus) to use various extended functions. For example, a technology that analyzes MIDI (Musical Instrument Digital Interface) data generated by playing an electronic musical instrument and creates and displays moving images that change along with the performance and still images (pictures) that reflect the content of the performance. It is under development (see Patent Document 1, for example).

Playing an instrument is difficult to practice, and many people get bored and give up halfway through. Recording one's own performance and checking which parts are not playable is a matter of being able to play to a certain extent, and many people give up before that. In order to motivate such people to practice, techniques using visual effects are attracting attention.

Japanese Patent Application Laid-Open No. 2019-101168

“Music Tapestry”, [online], [searched on November 20, 2019], Internet, <URL: https://news.mynavi.jp/article/20190724-casio_music_tapestry/>

In the technique of creating a picture based on musical performance, it is basic to create a picture by arranging characters corresponding to musical notes on a background of a color selected according to the key of the music being played. The background color is determined by applying the synesthesia relationship between sound and color, known by Scriabin, in association with the result of key determination by music analysis, such as red for C major and blue for G major.

However, users (performers, etc.) are not always satisfied with the finished picture. This is because, although the color tone of the picture may match the tone in theory, the performer does not necessarily like the color. With existing technology, the background color of the completed picture cannot be changed, which sometimes reduces the interest of performers.

The present invention has been made in view of these facts, and its object is to provide a program, method, electronic device, and musical performance capable of arousing user's interest by making it possible to change the color tone of a picture created by musical performance. To provide a data display system.

To achieve the above object, a program according to an embodiment of the present invention causes an information processing device to determine any one type of an attribute including a plurality of types based on input performance data, generating a first image containing a first color corresponding to the determined type; generating a second image containing a second color corresponding to a type different from the determined type; 2. Display the image on the display device.

According to the present invention, for example, it is possible to provide a program that can change the color tone of a picture created by performance and that can arouse the user's interest.

FIG. 1 is a diagram showing a state in which an information processing device TB according to the first embodiment is connected to a digital keyboard 1. As shown in FIG. FIG. 2 is a block diagram showing an example of the digital keyboard 1 according to the embodiment. FIG. 3 is a functional block diagram showing an example of the information processing device TB. FIG. 4 is a flow chart showing an example of the processing procedure of the information processing device TB. FIG. 5 is a diagram showing an example of a still image. FIG. 6 is a diagram showing an example of a print icon displayed on a still image. FIG. 7 is a diagram showing an example of a frame image. FIG. 8A is a flow chart showing an example of a processing procedure in playback/edit/print/setting mode. FIG. 8B is a flow chart showing an example of a processing procedure in playback/edit/print/setting mode. FIG. 9 is a diagram showing an example of screen transition on the display unit 52 when the "<" or ">" mark is tapped. FIG. 10 is a diagram showing an example of a setting screen displayed when the "∧" mark is tapped. FIG. 11 is a diagram showing an example of a transpose setting screen displayed when the "V" mark is tapped. FIG. 12 is a diagram showing an example of a thumbnail image of the first image. FIG. 13 is a diagram showing an example of thumbnail images of frame images. FIG. 14 is a diagram showing an example of a print icon displayed on a still image. FIG. 15 is a diagram showing an example of a print setting screen. FIG. 16 is a diagram showing an example of a print setting screen.

An information processing apparatus TB according to a first embodiment of the present invention will be described below with reference to the accompanying drawings.
<Configuration>
FIG. 1 is a diagram showing a state in which an information processing device TB of the first embodiment is connected to a digital keyboard 1. As shown in FIG. The digital keyboard 1 is, for example, an electronic keyboard instrument such as an electronic piano, a synthesizer, or an electronic organ. As shown in FIG. 1, the digital keyboard 1 includes a plurality of keys 10, a display section 20, an operation section 30, and a music stand MS. As shown in FIG. 1, the information processing device TB connected to the digital keyboard 1 can be placed on the music stand MS.

A key 10 is an operator for a player to designate a pitch, and when the player presses and releases the key 10, the digital keyboard 1 produces and silences a sound corresponding to the designated pitch. I do.

The display unit 20 has, for example, a liquid crystal display (LCD) with a touch panel, and displays messages accompanying the operation of the operation unit 30 by the performer. In addition, in the present embodiment, the display unit 20 has a touch panel function, so the display unit 20 can serve as one end of the operation unit 30 .

The operation unit 30 has operation buttons for the performer to perform various settings, and is a part for performing various setting operations such as volume adjustment.
The sound generator 40 is a part that outputs sound, and has output units such as a speaker 42 and headphone out.
FIG. 2 is a block diagram showing an example of the digital keyboard 1 according to the embodiment. The digital keyboard 1 includes a USB interface (I/F) 216, a RAM (Random Access Memory) 203, a ROM (Read Only Memory) 202, a display unit 20, an LCD controller 208, an LED (Light Emitting Diode) controller 207, a keyboard 101, Operation unit 30, key scanner 206, MIDI interface (I/F) 215, system bus 209, CPU (Central Processing Unit) 201, timer 210, sound source 204, digital/analog (D/A) converter 211, mixer 213, D A /A converter 212 , a speech synthesis LSI 205 and an amplifier 214 are provided. The sound source 204 and the speech synthesis LSI 205 are implemented as DSPs (Digital Signal Processors), for example.

CPU 201 , sound source 204 , voice synthesis LSI 205 , USB interface 216 , RAM 203 , ROM 202 , LCD controller 208 , LED controller 207 , key scanner 206 and MIDI interface 215 are connected to system bus 209 .

A CPU 201 is a processor that controls the digital keyboard 1 . That is, the CPU 201 reads the programs stored in the ROM 202 into the RAM 203 as a working memory and executes them to realize various functions of the digital keyboard 1 . The CPU 201 operates according to clocks supplied from the timer 210 . The clock is used, for example, to control sequences of automatic performance and automatic accompaniment.

A ROM 202 stores programs, various setting data, automatic accompaniment data, and the like. The automatic accompaniment data may include preset rhythm patterns, chord progressions, bass patterns, or melody data such as obbligato. The melody data may include pitch information of each sound, pronunciation timing information of each sound, and the like.

The sounding timing of each sound may be the interval time between soundings or the elapsed time from the start of the automatically played music. A tick is often used as the unit of time. A tick is a unit based on the tempo of a song, which is used in a general sequencer. For example, if the resolution of the sequencer is 480, one tick is 1/480 of the time of a quarter note.

The automatic accompaniment data may be stored not only in the ROM 202 but also in an information storage device or information storage medium (not shown). The format of the automatic accompaniment data may conform to the file format for MIDI.

The tone generator 204 is, for example, a so-called GM tone generator conforming to the GM (General MIDI) standard. This kind of sound source can change the timbre by giving a program change as a MIDI message, and can control a predetermined effect by giving a control change.

Sound source 204 is capable of producing, for example, up to 256 voices simultaneously. The tone generator 204 reads musical tone waveform data from, for example, a waveform ROM (not shown) and outputs it to the D/A converter 211 as digital musical tone waveform data. A D/A converter 211 converts the digital musical tone waveform data into an analog musical tone waveform signal.

Speech synthesis LSI 205 receives text data of lyrics and information on pitch as singing voice data from CPU 201 , synthesizes voice data of corresponding singing voice, and outputs the synthesized voice data to D/A converter 212 . A D/A converter 212 converts the audio data into an analog audio waveform signal.

A mixer 213 mixes the analog musical waveform signal and the analog voice waveform signal to generate an output signal. This output signal is amplified by the amplifier 214 and output from an output terminal such as the speaker 42 or headphone out.

The key scanner 206 constantly monitors the key depression/key release state of the keyboard 101 and the switch operation state of the operation unit 30 . The key scanner 206 then informs the CPU 201 of the states of the keyboard 101 and the operation unit 30 .

The LED controller 207 is, for example, an IC (Integrated Circuit). The LED controller 207 illuminates the keys of the keyboard 101 according to instructions from the CPU 201 to navigate the performer's performance. The LCD controller 208 is an IC that controls the display state of the display unit 20 .

The MIDI interface 215 inputs MIDI messages (performance data, etc.) from external devices such as the MIDI device 4, and outputs MIDI messages to external devices. The digital keyboard 1 can exchange MIDI messages and MIDI data files with an external device using an interface such as a USB (Universal Serial Bus). The received MIDI message is passed to the tone generator 204 via the CPU 201 . The tone generator 204 produces sound according to the tone color, volume, timing, etc. specified by the MIDI message.

An information processing device TB is connected to the system bus 209 via the USB interface 216 . The information processing device TB can acquire MIDI data (performance data) generated by playing the digital keyboard 1 via the USB interface 216 .
Furthermore, a storage medium (not shown) may also be connected to the system bus 209 via the USB interface 216 . Examples of the storage device 3 include a USB memory, flexible disk drive (FDD), hard disk drive (HDD), CD-ROM drive, and magneto-optical disk (MO) drive. If the program is not stored in the ROM 106, the program can be stored in a storage medium and read into the RAM 105 to cause the CPU 111 to perform the same operation as when the program is stored in the ROM 106. can.

FIG. 3 is a functional block diagram showing an example of the information processing device TB. The information processing device TB of the embodiment is, for example, a tablet-type information terminal, and an application for displaying an image on the display unit 52 as described later is installed. Note that the information processing device TB is not limited to a tablet-type portable terminal, and may be a notebook personal computer PC or the like. The information processing device TB may also include a sequencer or the like that receives MIDI data from the digital keyboard 1 and reproduces music data.

The information processing device TB mainly includes an operation unit 51 , a display unit 52 , a communication unit 53 , a sound output unit 54 , a control unit 56 (CPU), and a memory 55 . Each unit (operation unit 51, display unit 52, communication unit 53, sound output unit 54, control unit 56, and memory 55) is communicably connected via a bus 57 so that necessary data can be exchanged between the units. It has become.

The operation unit 51 includes, for example, switches such as a power switch for turning on/off the power. The display unit 52 has a liquid crystal monitor with a touch panel and displays images. Since the display unit 52 also has a touch panel function, it can serve as one end of the operation unit 51 .

The communication unit 53 includes a wireless unit and a wired unit for communicating with other devices. In the embodiment, the information processing device TB is wired to the digital keyboard 1 via, for example, a USB cable or the like, so that the information processing device TB can exchange various digital data with the digital keyboard 1 .

The sound output unit 54 has a speaker, an earphone jack, etc., and reproduces and outputs analog voices and musical tones, and outputs audio signals.

The control unit 56 includes a processor such as a CPU, and controls the information processing device TB. The CPU of the control unit 56 executes various processes according to the control programs stored in the memory 55 and installed applications.

Memory 55 includes ROM 60 and RAM 80 .
The ROM 60 stores, for example, a program 70 executed by the control unit 56, various data tables, and the like.

The RAM 80 stores data necessary for operating the program 70 . The RAM 80 also functions as a temporary storage area for data created by the control unit 56, MIDI data sent from the digital keyboard 1, and for developing applications. In this embodiment, the RAM 80 stores performance data 80a as MIDI data, as well as first image data 80b and second image data 80c derived from this performance data 80a.

By the way, in the embodiment, the program 70 comprises a music analysis routine 70a, a first image creation routine 70b, a second image creation routine 70c, and an output control routine 70d.

The music analysis routine 70a causes the control section 56 to determine any one type of attributes including a plurality of types based on the input performance data 80a. That is, the music analysis routine 70a analyzes the performance data 80a and determines the key of music based on the performance data 80a.

Here, tonality and key are distinguished as follows. Tonality can be understood as one of the attributes of a song and includes multiple keys. The key is the so-called key of a song, such as C major (C), D minor (Dm), B flat major (B♭), and so on. Tonality can be understood as a plurality of types included in tonality.
In addition to tonality, attributes include, for example, rhythm, tempo, form (rondo, sonata, etc.).

Although the method for determining the tonality, chord type, etc. is not particularly limited, for example, the method disclosed in Japanese Patent No. 3211839 or the like can be used.

The first image creation routine 70b causes the control unit 56 to create a first image that corresponds to the performance data 80a and that includes, for example, a background color that corresponds to the determined tone. The first image in the embodiment is a still image with a background color that matches the key of the played song, which is generated after the performance ends. The generated first image data is stored in the RAM 80 (first image data 80b).
It should be noted that the "first image" disclosed in Patent Document 1 corresponds to a moving image displayed in real time during a performance or an image of one frame thereof (such as the final frame). different from Also, the "second image" disclosed in Patent Document 1 corresponds to a still image generated after the performance ends, and corresponds to the "first image" of the present embodiment.

The [second image] in this embodiment is an image that includes, for example, a background color that corresponds to a tone different from the tone determined by the music analysis routine 70a. That is, the [second image] is a group of 11 images having a background color different from that of the [first image].

The second image creation routine 70c transposes the performance data 80a to create new performance data, and causes the control unit 56 to generate a second image based on the results of musical analysis of the new performance data. . That is, the second image creation routine 70c creates a second image with a background color corresponding to a key different from the key actually played.

The [second image] is also the same as the [first image] in that it is a still image after the performance is finished. However, the background color of the second image does not match the key of the piece being played, but is a color that matches the background of another key. That is, the second image is image data created based on performance data corresponding to each of 11 tones obtained by transposing the performance data from which the first image is derived. The generated second image data is stored in the RAM 80 (second image data 80c).

Both the first image data 80b and the second image data 80c can be created based on the performance data 80a using the technique disclosed in Patent Document 1, for example. By the way, under the 12-tone equal temperament, there are 12 major and minor keys, and there are 24 keys in total. If the same background color is applied to major and minor keys that are parallel to each other (for example, C major and A minor), there are 12 background colors.

The output control routine 70d causes the control unit 56 to display the created first image and second image on the display unit 52 as a display device.

<Action>
Next, the operation of the above configuration will be described. In the following description, it is assumed that the communication unit 53 of the information processing device TB and the communication unit 90 of the digital keyboard 1 are connected by wire. It is also assumed that an application for displaying an image on the display unit 52 is activated in the information processing device TB.

FIG. 4 is a flow chart showing an example of the processing procedure of the information processing device TB. In FIG. 4, the control unit 56 (CPU) of the information processing device TB waits for transmission of performance data from the digital keyboard 1 (step S1). If there is no performance data input (step S1: NO), the control unit 56 performs performance end determination processing (step S4).

In step S4, when a predetermined period of time has passed without performance data input (step S4: YES), the control unit 56 determines that the performance has ended. In other words, if the reception input of performance data is not detected for a certain period of time, the process proceeds to step S5. Incidentally, in the form in which the accompaniment data is automatically reproduced and the melody is specified by the performer, it may be judged that the performance is finished when the automatic reproduction of the accompaniment data is finished, and the process may proceed to step S5. In other words, the reason why the control section 56 determines that the performance is finished may be any reason. In short, when the control section 56 determines that the performance is finished, the processing procedure should be shifted to step S5.

In step S5, the control section 56 creates a still image reflecting the analysis result of the performance data 80a, and displays and outputs it on the display section 52 (step S5). This still image is the first image in this embodiment.

FIG. 5 is a diagram showing an example of a still image. In the embodiment, an operation icon (widget) is displayed superimposed on the image itself. In the center of the still image in FIG. 5, a "playback (rightward black arrow)" mark is displayed, and "∧", "V", "<", and ">" marks are displayed on the top, bottom, left, and right, respectively. Any mark can be turned on (tapped) on the touch panel of the display unit 52, and is an interface for conveying the user's intention to the system. For example, "<" or ">" can be used to increment the still image being displayed. Alternatively, it can also be used to switch between still images being displayed.

FIG. 6 shows a still image in which an icon for printing (print mark) is displayed in the center instead of the "playback" mark. The print mark is displayed after a short period of time has passed since the user selected the desired still image. When the print mark is tapped, the print mode is entered.

FIG. 7 is a diagram showing another example of a still image. For example, one frame (last frame or the like) of the moving image displayed in real time on the display unit 52 (FIG. 3) along with the performance can be taken out and used as a still image. An image created in this manner will be referred to as a "frame image". In other words, the frame image can be said to be an image that is displayed after the performance ends and before the first image is displayed. "Play" and the marks "∧", "V", "<", ">" displayed in FIGS. 5-7 can be used to enter the edit mode.

Returning to FIG. 4 again, the description is continued. In FIG. 4, if the end of the performance is not detected (step S4: NO), the processing procedure returns to step S1. If there is performance data input in step S1 (step S1: YES), the processing procedure proceeds to the next step S2.

At step S2, the control unit 56 executes performance determination processing (step S2). In step S2, the control unit 56 determines, for example, key (for example, 24 types from C major to B minor), chord type (for example, Major, minor, sus4, aug, dim, 7th, etc.) based on the acquired performance data. ) and note names (for example, do, re, mi, etc.). The note name can be determined by the note number or the like included in the performance data. The tone determined here is reflected in the background color of the first image (still image) generated in step S5.

As the performance determination process, any one of tonality determination, chord type determination, and pitch name determination may be performed. Conversely, in addition to these, for example, syncopation, chord progression, etc. may be determined.

When the performance determination process is finished, the control unit 56 outputs and displays a moving image corresponding to the performance data on the display unit 52 at the timing of receiving the performance data (step S3). It should be noted that the timing of receiving performance data does not mean at the same time in terms of time, but is understood to mean each time performance data is received.

For example, a basic image (basic image) can be stored in the memory 55 in advance, the basic image can be processed by computer graphics (CG) to create a moving image, and the display unit 52 can display the moving image. The first image to be displayed on the display unit 52 is not limited to this, and may be a moving image corresponding to the performance data.

Now, when step S5 is completed, the control section 56 waits for transition to the reproduction/edit/print/setting mode (step S6). That is, when any one of the images shown in FIGS. 5 to 7 is displayed, tapping the marks "∧", "V", "<", ">" or the print marks (step S6: YES), the control unit 56 shifts the information processing device TB to the reproduction/edit/print/setting mode (step S7).

8A and 8B are flow charts showing an example of the processing procedure in the playback/edit/print/setting mode. In FIG. 8A, the control unit 56 checks whether any key on the keyboard 101 (FIG. 2) has been pressed (step S11). If a key is pressed (YES), the system enters a performance standby state and returns to the calling source, that is, the processing procedure of FIG. 4 (RETURN). If no key has been pressed (NO), the control unit 56 determines whether or not the "playback" mark has been tapped on the screen of FIG. 5 (step S12). If YES, the processing procedure jumps to the reproduction process (step S15), and the performance based on the performance data 80a stored in the RAM 80 (FIG. 3) is started. Such a procedure makes it possible to reproduce music from frame images.

Next, the control unit 56 determines whether or not the "<" or ">" mark has been tapped on the screen of FIG. 5 (step S13).
FIG. 9 is a diagram showing an example of screen transition on the display unit 52 when the "<" or ">" mark is tapped. In step S13 of FIG. 8A, when ">" is tapped, the screen transitions one by one in the increment direction, that is, in the direction of the arrow in FIG. 9 (step S16). When "<" is tapped, the screen transitions in the decrement direction (reverse direction) (step S16). As shown in FIG. 9, images corresponding to a plurality of songs are cyclically (repeatedly) displayed. FIG. 9 shows image data for four songs played in the past, and each song has a set of [frame image] and [first image].

Next, the control unit 56 checks whether or not a key has been pressed (step S17). RETURN). If no key is pressed (NO), the control unit 56 determines whether or not the "playback" mark has been tapped (step S18), and if YES, the processing procedure jumps to playback processing (step S19). With this procedure, it is possible to select and reproduce the performance data 80a and the image data 80b stored in the RAM 80 up to that time.

On the other hand, if NO in step S13, the control unit 56 determines whether or not the "∧" mark has been tapped (step S14), and if YES, the setting screen is displayed (step S20).
FIG. 10 is a diagram showing an example of a setting screen displayed when the "∧" mark is tapped. The setting contents include, for example, the degree of difficulty of the music to be played, the setting and appearance ratio of reward pictures (bouquet, abstract, kids), presence/absence of reward, and the like. In addition, it is possible to set the details of the portion related to overall music analysis, the portion related to evaluation, and the portion related to expression, such as what type of picture (character) is to be used.

Even after the setting screen is displayed, the control unit 56 checks whether a key has been pressed (step S21), and if there is a key pressed (YES), returns to the processing procedure (FIG. 4) of the caller ( RETURN). If no key has been pressed (NO), the control section 56 determines whether or not the cancel button (FIG. 10) has been tapped (step S22). is repeated again.

If NO in step S22, the control unit 56 determines whether or not the OK button has been tapped (step S23), and if YES, the settings are reflected in the system (step S24). If NO, the processing procedure returns to step S21 again. Such a procedure allows to change various settings such as representation, rating, music analysis, and so on.

FIG. 8B shows a processing procedure to be jumped to when NO is determined in step S14 of FIG. 8A. At step S25 in FIG. 8, the control unit 56 determines whether or not the "V" mark has been tapped (step S25), and if YES, a transpose setting screen is displayed (step S28).
FIG. 11 is a diagram showing an example of a transpose setting screen displayed when the "V" mark is tapped. A total of 12 images are thumbnail-displayed on this screen, including the first images in colors corresponding to the keys of the played music (original keys) and the first images in colors corresponding to the other 11 keys. . If the key of the song played before this screen is displayed is C (Key=C), the thumbnail image of C is surrounded by a rectangular frame (cursor) and highlighted. The cursor can be moved with each mark of "∧", "V", "<", and ">", and the user can select the desired image.

As shown in FIG. 12, the thumbnail images may be a group of images corresponding to the first image (still image) generated in step S5 (FIG. 4). In the thumbnail display, major/minor keys that are parallel to each other are combined into one reduced image, so there are 12 images for all keys. After selecting a favorite picture with "<" or ">" in FIG. 5 and tapping "V", the screen in FIG. 11 appears. 11 can be said to focus on the image selected in FIG. 5 among the thumbnail images in FIG.

As shown in FIG. 13, the thumbnail image may be an image group obtained by reducing the frame image corresponding to each key. If you select your favorite picture with "<" or ">" in Fig. 7 and tap "V", the transpose setting screen corresponding to the frame image appears. That is, the thumbnail image in FIG. 13 and each button on the right side in FIG. 11 are displayed on the same screen.

When specifying a favorite picture, some people want to see the first image, while others want to see the frame image immediately after the performance. From any image, you can tap the "V" to display the transpose setting screen.

Returning to FIG. 8B, the description continues. After step S28, the control unit 56 displays thumbnail images of all tones transposed to the original key C and other keys as shown in FIG. 13 (step S29). In FIG. 13, since the cursor is on the image of C (because it is focused), it can be seen that the original key is selected. When "<" or ">" is tapped to select another key, the Key is changed and another key is selected.

Next, the control unit 56 determines whether or not printing is on (step S30). This is a determination as to whether or not the [Print with this key] button in FIG. 13 has been tapped. If YES, the control unit 56 registers the image of the selected key in the buffer for printing (step S34), the processing procedure jumps to the print setting screen, and printing is executed (step S39). In this way, an image with the user's favorite color tone is printed. Moreover, the image reflects the content of the performance.

Since the performance data 80a is MIDI data, it is fully possible to transpose the data in all tones and develop it into an image by calculation of the CPU. An output control routine 70d (FIG. 3) transposes the original key MIDI data to another key, and a second image creation routine 70c creates a second image, or frame image, based on the transposed MIDI data. The transposed MIDI data and the created second image data are stored in RAM 80 (FIG. 3).

If NO in step S30, the control section 56 determines whether or not to listen to the performance (step S31). This is a determination as to whether or not the [Listen with this Key] button in FIG. 13 has been tapped. If YES, the control unit 56 determines whether or not the Key has been changed (step S35), and if there has been a change (YES), reproduces the MIDI data with the selected key (step S36). It is also possible to listen and compare in various tones on the transpose screen of FIG. If there is no key change (step S35: NO), the MIDI data of the original key is reproduced (step S37).

If NO in step S31, the control unit 56 determines whether or not the key has been changed (step S32). If YES here, the control unit 56 changes the current key display and cursor position (step S38). When the cancel button in FIG. 11 is tapped, the editing mode is exited and the transpose setting screen returns to the screen in FIG. 5 or 7 . In the case of saving the transposed image and the transposed performance, it is sufficient to return to the transposed frame image when returning to FIG.

In this way, by transposing the MIDI data that is the basis of the original tone image, all the pictures that match the background color of each tone are displayed without destroying the relationship between the color and the sound that match the tone. be able to. In addition, by making it possible to reproduce with transposed keys, it is possible to provide the picture desired by the performer.

I will continue to explain. If NO in step S25, the control unit 56 determines whether or not the print icon (FIG. 6) is tapped when it is displayed (step S26). If YES, the processing procedure jumps to the display of the print setting screen and printing is executed (step S39).

15 and 16 are diagrams showing examples of print setting screens. FIG. 15 is for determining the items to be displayed on the picture to be printed, and arbitrary items can be selected by check boxes. When a performer's name or a song title is selected, a GUI (Graphical User Interface) shown in FIG. 16 is displayed so that characters can be entered. Once the date, song length, key, etc. are selected, these items are automatically printed.

The control unit 56 determines whether or not the print settings have been changed (step S40). If there are changes (YES), the changed settings are saved (step S41), and the changed information is superimposed on the buffered image data. and print (step S43). If there is no change (step S40: NO), the control section 56 adopts the default setting (step S42), and the process proceeds to step S43. When the printing is completed, the processing procedure of the caller in FIG. 4 is returned to (RETURN).

If NO in step S26, the control unit 56 checks whether or not the key has been pressed (step S27), and if the key has been pressed (YES), returns to the processing procedure (FIG. 4) of the caller ( RETURN). If no key is pressed (NO), the process returns to step S11 in FIG. 8A.

As described above, in the present embodiment, the MIDI data of the original key picture is transposed to change the background color of each key by selecting a key without destroying the relationship between colors and sounds that match the key. Made it possible to display all matching pictures. In addition, it is possible to reproduce transposed keys using transposed MIDI data. By doing so, it is possible to provide a picture that meets the needs of the performer while maintaining the consistency of the correlation between sound and color.

Images created from transposed MIDI data are displayed as thumbnails, and users can print their favorite pictures. The color of the picture created by the performance is determined by music analysis, but many users are interested in what kind of color it will be in other tones. In addition to simply changing the color, it is also possible to listen with a transposed key in the embodiment, and the correlation between the sound and the color is consistent, so there is an advantage that confusion does not occur.

In the embodiment, the final screen (first image or frame image) created from the performance is provided with four directional icons and a central icon, allowing simple operations to select and play songs, select pictures by transposing, and so on. It is now possible to perform playback, printing, various analyzes and expression settings. Therefore, playback and selection of music, editing of pictures by transposition, printing, various analyzes and expression settings can be easily understood and edited.

Accordingly, according to the embodiments, it is possible to provide a program, a method, an electronic device, and a performance data display system that can change the color tone of a picture created by a performance, thereby arousing the user's interest. becomes possible. As a result, playing and practicing musical instruments becomes even more enjoyable.

It should be noted that the present invention is not limited to specific embodiments. For example, in the embodiment, a tablet-type portable terminal separate from the digital keyboard 1 is assumed as the information processing device TB. Not limited to this, a desktop or notebook computer may be used. Alternatively, the digital keyboard 1 itself may also function as an information processing device. In addition, the technical scope of the present invention includes various modifications and improvements within the range in which the object of the present invention is achieved, which will be apparent to those skilled in the art from the description of the scope of claims. be.

The invention described in the scope of claims originally attached to the application form of this application is additionally described below.
The claim numbers of the claims described in the supplementary notes are as in the claims originally attached to the request of this application.
<Claim 1>
information processing equipment,
determining any one type of an attribute including a plurality of types based on the input performance data;
generating a first image including a first color corresponding to the determined type;
generating a second image including a second color corresponding to a type different from the determined type;
displaying the first image and the second image on a display device;
program.
<Claim 2>
The attribute is a tonality including a plurality of tonalities as the type,
generating both the first image and the second image based on the input performance data;
A program according to claim 1.
<Claim 3>
including the first color in the background color of the first image;
including the second color in the background color of the second image;
3. A program according to claim 1 or 2.
<Claim 4>
causing the display device to display thumbnails of a plurality of images including at least the first image and the second image;
4. A program according to any one of claims 1 to 3.
<Claim 5>
giving the input performance data to a performance device in accordance with the designation of the first image to cause it to play;
In response to the specification of the second image, the performance device outputs second performance data generated based on the input performance data, the second performance data corresponding to a type different from the determined type. to play,
A program according to any one of claims 1 to 4.
<Claim 6>
information processing equipment,
determining any one type of an attribute including a plurality of types based on the input performance data;
generating a first image including a first color corresponding to the determined type;
generating a second image including a second color corresponding to a type different from the determined type;
displaying the first image and the second image on a display device;
Method.
<Claim 7>
a display device;
a processor, said processor comprising:
determining any one type of an attribute including a plurality of types based on the input performance data;
generating a first image including a first color corresponding to the determined type;
generating a second image including a second color corresponding to a type different from the determined type;
instructing the display device to display the first image and the second image;
Electronics.
<Claim 8>
comprising an electronic musical instrument and a display device,
The electronic musical instrument
generating performance data according to a performance operation by a user, and outputting the generated performance data to the display device;
The display device
determining any one type of an attribute including a plurality of types based on the input performance data;
generating a first image including a first color corresponding to the determined type;
generating a second image including a second color corresponding to a type different from the determined type;
displaying the first image and the second image;
Performance data display system.

DESCRIPTION OF SYMBOLS 1... Digital keyboard 3... Storage device 4... MIDI device 10... Key 20... Display unit 30... Operation unit 40... Sound generator 42... Speaker 51... Operation unit 52... Display unit 53... Communication unit 54 Sound output unit 55 Memory 56 Control unit 57 Bus 60 ROM 70 Program 70a Music analysis routine 70b Image creation routine 70c Image creation routine 70d Output control routine 80...RAM 80a...performance data 80b...first image data 80c...second image data 90...communication section 101...keyboard 105...RAM 106...ROM 111...CPU 201... CPU 202 ROM 203 RAM 204 Sound source 206 Key scanner 207 LED controller 207 Controller 208 LCD controller 209 System bus 210 Timer 211 D/A converter 212 ... D/A converter, 213 ... Mixer, 214 ... Amplifier, 215 ... MIDI interface, 216 ... USB interface, 205 ... Voice synthesis LSI.

In order to achieve the above object, a program according to an embodiment of the present invention causes an information processing device to generate a first image based on input first performance data, and generates a tone corresponding to the first performance data. second performance data is generated by transposing the tone, and a second image is generated based on the second performance data .

Claims (8)

information processing equipment,
determining any one type of an attribute including a plurality of types based on the input performance data;
generating a first image including a first color corresponding to the determined type;
generating a second image including a second color corresponding to a type different from the determined type;
displaying the first image and the second image on a display device;
program. The attribute is a tonality including a plurality of tonalities as the type,
generating both the first image and the second image based on the input performance data;
A program according to claim 1. including the first color in the background color of the first image;
including the second color in the background color of the second image;
3. A program according to claim 1 or 2. causing the display device to display thumbnails of a plurality of images including at least the first image and the second image;
4. A program according to any one of claims 1 to 3. giving the input performance data to a performance device in accordance with the designation of the first image to cause it to play;
In response to the designation of the second image, the performance device outputs second performance data generated based on the input performance data, the second performance data corresponding to a type different from the determined type. to play,
A program according to any one of claims 1 to 4. information processing equipment,
determining any one type of an attribute including a plurality of types based on the input performance data;
generating a first image including a first color corresponding to the determined type;
generating a second image including a second color corresponding to a type different from the determined type;
displaying the first image and the second image on a display device;
Method. a display device;
a processor, said processor comprising:
determining any one type of an attribute including a plurality of types based on the input performance data;
generating a first image including a first color corresponding to the determined type;
generating a second image including a second color corresponding to a type different from the determined type;
instructing the display device to display the first image and the second image;
Electronics. comprising an electronic musical instrument and a display device,
The electronic musical instrument
generating performance data according to a performance operation by a user, and outputting the generated performance data to the display device;
The display device
determining any one type of an attribute including a plurality of types based on the input performance data;
generating a first image including a first color corresponding to the determined type;
generating a second image including a second color corresponding to a type different from the determined type;
displaying the first image and the second image;
Performance data display system.

Images