JP2003107986A - Information processor, and control method, control program and recording medium therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processor which is suitable for learning playing by freely writing into a displayed image such as an electronic score, and to provide a control method, a control program and a recording medium therefor. SOLUTION: The information processor displays a score image generated from score data by the unit of the page on a first layer L1 to display a handwritten image corresponding to the locus of a position inputted via a touch panel on a second layer L2 to display an image obtained by overlapping these layers on display screens 2 and 3. Further, the information processor stores data on the hand written image with additional information including the page of the score image displayed at the time in advance. When there are data on the handwritten image on the page of a changed score based on the additional information in the case of changing the page of the score image to be displayed, the information processor displays the image of the data on the second layer L2.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus, a control method therefor, a control program, and a recording medium, and more particularly to a technology for performing handwriting input on a display image and storing and transmitting / receiving the image.

[0002]

2. Description of the Related Art Conventionally, as software that can be used in a personal computer (hereinafter referred to as "PC"), there is music software for displaying a musical score. Also, this type of music software includes MIDI (Musical Instrument Digital Interface) connected to a PC.
There is a device that can input MIDI data of a performance from a keyboard and display a mistaken part (note) on a score displayed on the screen (hereinafter referred to as "electronic score"). As a result, the user of the PC can practice the keyboard performance by himself.

In recent years, with the background of the spread of personal computers (hereinafter referred to as "PCs") to general households, services for performing musical instrument lessons using the Internet have begun to be provided. With this service, when a student plays an instrument using the above-mentioned software for music and sends their performance MIDI data and questions to the teacher by e-mail, the teacher sends advice by e-mail. Is to come.

[0004]

However, in comparison with a general paper-based musical score, the electronic musical score has a portion that is inferior in usability because it is not possible to freely write notes and memos. Further, in the above-mentioned lesson of playing musical instruments on the Internet, the teacher and the student exchange information by e-mail, so that the lesson cannot be conducted in real time.

The present invention has been made in view of the above-mentioned circumstances, and an information processing apparatus which can freely write on a display image such as an electronic musical score and is suitable for performance practice, a control method therefor, a control program, and a recording. It is intended to provide a medium.

[0006]

In order to solve the above-mentioned problems, the invention according to claim 1 provides an information processing apparatus,
Display device, display data recording means for recording display data, image creating means for creating image data based on the display data, and position on the screen of the display device designated by the operator Position detecting means for detecting
By managing the image data created by the image creating unit and the handwriting data corresponding to the locus of the position detected by the position detecting unit in different layers, the image corresponding to the image data and the handwriting Display control means for displaying a handwritten image corresponding to data on the display device in an overlapping manner, the display control means displaying an image corresponding to the image data on a predetermined layer in page units, and the handwriting The handwritten image corresponding to the data is displayed in real time on another layer different from the layer, and the handwritten data is stored in association with the page of the image displayed at that time, When the image is redisplayed on the layer, a handwritten image corresponding to the handwritten data is displayed on the other layer. It is a symptom.

According to this structure, the image of the image data recorded in advance and the handwritten image of the handwritten data corresponding to the locus of the position detected by the position detecting means are displayed in different layers in real time, and The handwritten data is stored in association with the page of the image displayed at that time, and the handwritten image corresponding to the handwritten data is displayed when the image of the page is displayed again, so that the displayed image Can be written page by page, and when the page of the image in which the handwritten image is written is displayed again, the handwritten image can be displayed again. Further, since the image of the image data and the handwritten image of the handwritten data are displayed in different layers, it is possible to easily hide only the handwritten image by switching the display / non-display of the layer. it can.

According to the present invention, in a control method of an information processing apparatus, an image creating step of creating image data based on display data recorded in the display data recording means, and a display device specified by an operator. A position detection step of detecting a position on the screen, the image data, and handwriting data corresponding to the locus of the detected position are managed in different layers, respectively, and an image corresponding to the image data, And a display control step of displaying a handwritten image corresponding to handwritten data in an overlapping manner on the display device, and in the display control step,
An image corresponding to the image data is displayed on a predetermined layer in page units, and a handwritten image corresponding to the handwritten data is displayed in real time on another layer different from the layer, and the handwritten data is displayed at that time. Stored in association with the page of the image displayed in
When the image of the page is re-displayed on the layer, the handwritten image corresponding to the handwritten data is displayed on the other layer.

According to this structure, as in the case described above, writing can be performed for each page of the displayed image, and when the page of the image in which the handwritten image is written is displayed again, the handwritten image is re-displayed. Can be displayed. Further, it is possible to easily hide only the handwritten image.

Also, the present invention provides a computer having a display device in a control program and a position detecting means for detecting a position on the screen of the display device designated by an operator, for displaying data for display. Data recording means, image creating means for creating image data based on the display data, image data created by the image creating means, and handwriting corresponding to the locus of the position detected by the position detecting means A control program for causing the image corresponding to the image data and the handwritten image corresponding to the handwritten data to function as a display control unit that is displayed on the display device by managing the data in different layers. The display control means is configured to set an image corresponding to the image data in a predetermined layer for each page. The handwritten image corresponding to the handwritten data is displayed in real time on another layer different from the layer, and the handwritten data is stored in association with the page of the image displayed at that time. When the image of the page is re-displayed on the layer, the hand-written image corresponding to the hand-written data is caused to function on the other layer.

According to this configuration, when this control program is installed in the computer, the image of the image data previously recorded in the computer and the handwritten data corresponding to the locus of the position detected by the position detecting means are stored. While displaying the handwritten image in different layers in real time, the handwritten data is stored in association with the page of the image displayed at that time, and the handwritten data is displayed when the image of the page is redisplayed. Since it is possible to display the handwritten image corresponding to, it is possible to write on each page of the displayed image, and when the page of the image on which the handwritten image is written is redisplayed, the handwritten image is redisplayed. Can be made. Further, it is possible to easily hide only the handwritten image.

Further, the present invention provides a computer-readable recording medium in which a computer having a display device and a position detection means for detecting a position on the screen of the display device designated by an operator is provided with data for display. Data recording means for recording, image creating means for creating image data based on the display data, image data created by the image creating means, and a position detected by the position detecting means. By managing the handwritten data corresponding to the locus in different layers, the image corresponding to the image data and the handwritten image corresponding to the handwritten data are caused to function as a display control unit that is displayed on the display device in an overlapping manner. And a display control unit for displaying an image corresponding to the image data. Place in to be displayed on a predetermined layer,
A handwritten image corresponding to the handwritten data is displayed in real time on another layer different from the layer, and the handwritten data is stored in association with the page of the image displayed at that time, It is characterized in that a control program for recording a handwritten image corresponding to the handwritten data when the page image is redisplayed on the layer is recorded as a control program.

According to this configuration, if the control program is read from the recording medium and executed by the computer, it is possible to write the displayed image page by page in the computer in the same manner as described above. When the page of the image in which is written is re-displayed, the handwritten image can be re-displayed. Further, it is possible to easily hide only the handwritten image.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. The embodiments described below show one aspect of the present invention, and do not limit the present invention, and can be arbitrarily modified within the scope of the present invention.

(1) Configuration of the Embodiment (1.1) External Configuration of Musical Score Type Information Processing Device FIG. 1 is an external view of a musical score type information processing device according to an embodiment of the present invention. The musical score type information processing apparatus 1 is formed to have a size substantially the same as a general paper-based musical score, and can be placed on a music stand or the like during performance and used by a performer in place of the paper-based musical score. It is a device.

As shown in FIG. 1, the musical score type information processing apparatus 1 is composed of a thin body having a substantially rectangular shape, and has display screens 2 and 3 for displaying a musical score or the like on the left and right. The display screens 2 and 3 are displayed on the liquid crystal display panel 10 as shown in FIG.
Transparent touch panels 11A and 11B on the upper surfaces of A and 10B
Are provided respectively, and the touch operation and the touch position of the display screens 2 and 3 with a finger or a stylus pen indicate the touch panel 11.
It can be detected by A and 11B. Further, in FIG. 1, 4 is an operation switch group including a power switch and the like. That is, the musical score type information processing apparatus 1 is configured to input an instruction from the player through the operation of the operation switch group 4 and the pressing operation of the display screens 2 and 3.

The housing 5 on the left side and the housing 6 on the right side of the musical score type information processing apparatus 1 are rotatably connected by a hinge mechanism S, and can be opened and closed like a notebook or a musical score (musical score book). It is configured to be possible. As a result, the musical score type information processing apparatus 1 can be reduced in overall shape by closing the left and right casings 5 and 6, as shown in FIG.
It can be easily carried. Further, in this closed state, the display screens 2 and 3 are not exposed to the outside, so that it is possible to prevent the display screens 2 and 3 from being damaged.

Further, the musical score type information processing apparatus 1 has a camera 7, a microphone 8 and a speaker 9.
The camera 7 is provided in front of the musical score type information processing apparatus 1 in the open state (the state shown in FIG. 1), that is, for photographing a performer who plays the musical composition while viewing the displayed musical score. A microphone having directivity is applied to the microphone 8 and is provided to input a voice from the front surface of the musical score type information processing apparatus 1, that is, a performance sound or voice of a performer in an open state. The speaker 9 is for emitting a sound generated from the MIDI data or the sound data recorded in the musical score type information processing apparatus 1. Further, the musical score type information processing apparatus 1 has a built-in memory card reader / writer 12, and has a memory card slot at a side surface position (not shown).

(1.2) Electrical Configuration of Musical Score Type Information Processing Device Next, the electrical configuration of the musical score type information processing device 1 will be described with reference to FIG. In the figure, a CPU (Central Proc
The essing unit) 20 controls the operation of each unit connected via the bus (BUS) 21 to control the entire score type information processing apparatus 1. Further, when the CPU 20 performs display control for driving the liquid crystal display panels 10A and 10B to display various images, the CPU 20 performs display control for displaying the display image in a plurality of layer structures. The layer display will be described in the processing content of the musical score type information processing apparatus 1 in the subsequent stage.

A RAM (Random Access Memory) 22 is
The memory is used as a buffer memory or a work memory of the CPU 20, and temporarily stores image data, program data, and the like. In addition, the control program memory 23, the musical score data memory 24, the user data memory 25
Is, for example, a magnetic disk or an EEP-ROM (Elec
trically Erasable Programmable Read Only Memory)
The control program memory 23 stores a control program group showing a processing procedure for causing the CPU 20 to perform various kinds of processing (musical score display processing, etc.). Further, the musical score data memory 24 stores musical score data of a plurality of music pieces, and the user data memory 25 records image and sound data such as photographed data and recorded data. .

The wireless communication section 26 wirelessly performs data communication with an external device via an antenna (not shown) under the control of the CPU 20, and a wireless module such as Bluetooth (registered trademark) is applied to the wireless communication section 26. In the musical score information processing apparatus 1, the wireless communication unit 26 connects to the Internet via a router (not shown), and performs data communication with other musical score information processing apparatuses 1 and PCs.

Further, the MIDI interface 27 is
It is for communicating MIDI data between the musical score type information processing apparatus 1 and an electronic musical instrument or a PC having a MIDI interface. Further, the sound source 28 is the CPU 20.
Under the control of (1), an audio signal is generated from audio data or MIDI data, and the sound is emitted from the speaker 9 via a digital-analog conversion circuit (not shown).

(1.3) Structure of Musical Score Data Next, the musical score data used in this embodiment will be described.
As shown in FIG. 5, the musical score data DM includes fields of pitch information, rhythm information, and page information, and the pitch information further includes scale information and octave information.

The scale information represents scales such as "do (C)" using numbers, and specifically, the scale is set with "1" as the width of a semitone based on "do" = "0". Is specified. For example, “1” represents “Do # 1” that is a semitone higher than “Do”, and “2” represents “Re” that is one tone higher than “Do”.

In the octave information, the pitch of the octave with respect to the reference tone is represented by positive and negative signs "+" and "-", and the interval of the octave with respect to the reference tone is represented by numbers. For example, “+1” designates a tone one octave higher than the reference tone, and “−1” designates a tone one octave lower than the reference tone. Therefore, the scale information is "1".
When the octave designation information is "+1", it represents "do #" which is one octave higher than the reference "do".

On the other hand, in the rhythm information, a quarter note, which is a reference for the duration of the note, is "1" to represent the duration of the note by a numeral. For example, "0.5" is a half note length of a quarter note, so an eighth note is specified, and "4" is a quarter note length of a quarter note. All notes are specified.

The page information specifies the page of the musical score data to which the pitch information and the rhythm information belong. In addition to the above-mentioned information, the musical score data includes tempo information for designating the tempo at the time of performance, information indicating the title of the song, composer name, lyricist name, and the like.

The musical score type information processing apparatus 1 is C
Each page of the musical score is displayed on the display screens 2 and 3 by the PU 20 reading out the musical score data DM of the corresponding musical composition based on the instruction of the performer and generating the image data of the musical score of the musical notation in page units by a predetermined algorithm. It can be displayed.

(1.4) Processing Content of Musical Score Type Information Processing Apparatus Next, various kinds of processing performed by the musical score type information processing apparatus 1 will be described. The musical score type information processing apparatus 1 has a plurality of operation modes, and performs musical score display, shooting, recording, or data communication by switching the operation modes. Further, the musical score type information processing apparatus 1 according to the present embodiment has a function of using these functions to perform a remote lesson via a network such as the Internet. Specifically, the musical score type information processing apparatus 1 has a musical score display mode,
It has operation modes such as shooting mode, recording mode, communication mode and lesson mode. Hereinafter, each operation mode will be described.

A. Musical score display mode The musical score display mode is an operation mode for displaying a musical score for playing a musical instrument, and when this mode is selected, the CPU 2
0 displays a file name of the musical score data DM stored in the musical score data memory 24 on the display screens 2 and 3 by reading and executing a predetermined program recorded in the control program memory 23, and prompts the operator to do so. Prompts the user to select the score data DM to be displayed. The player may directly input the file name of the musical score data DM, or the musical score data DM stored in the musical score data memory 24 may be input.
May be displayed as a thumbnail and selected from among them.

When the operator selects the musical score data DM, the CPU 20 reads the corresponding musical score data DM from the musical score data memory 24 and generates the musical score image data of the first and second pages. The score of the first page and the score of the second page are displayed on the display screen 2 and the display screen 3 page by page. As shown in FIG. 6 as a display example in this case, operation button groups 30 and 40 for instructing various operations are further displayed on the display screen 2. Specifically, the operation button group 30 is used to perform a score operation such as page turning, page returning, returning to the beginning of a page, saving a score and displaying another score,
Software buttons for instructing processing such as erasing, saving, and various settings of handwritten images, and the operation button group 40 is a software button for instructing switching of operation modes.

Further, in this score display processing, the CPU
As shown in FIG. 7, 20 performs a layer display process of displaying the display image P in association with a plurality of layers. In this figure, the score image and the operation button groups 30, 4 are displayed on the first layer L1.
The case where an image of 0 is displayed is shown. Here, no image is displayed on the second layer L2 to the nth layer Ln. Then, the CPU 20 drives the liquid crystal display panels 10A and 10B to display the image Pl in the left area of the display image P on the liquid crystal display panel 10A, and displays the image Pr in the right area on the liquid crystal display panel 10B.
The score of the first page is displayed on the display screen 2, and the score of the second page is displayed on the display screen 3.

Next, the CPU 20 controls the touch panel 11
When it is detected that the operation button of the score operation of the operation button group 30 has been pressed based on the pressed position detected via A and 11B, the page of the score displayed on the first layer L1 is changed to display the image. The process of updating P is performed. For example,
When the page forward button is pressed, the score image displayed on the first layer L1 is updated to the score image after two pages, so that the score displayed on the display screens 2 and 3 is changed to the score after two pages. Change to. Further, when the CPU 20 detects that one of the operation button groups 40 is pressed via the touch panels 11A and 11B, the CPU 20 performs a process of switching the operation mode.

Further, the CPU 20 uses the touch panel 11
The pressed position detected through A and 11B is the operation button group 3
In the case of a position other than 0 and 40, image data of the locus of this position (hereinafter referred to as “handwritten data”) is generated, and as shown in FIG.
By displaying in real time on 2, the score image and the locus of the position are overlapped and displayed on the display screens 2 and 3. Further, as shown in FIG. 9 or FIG. 10, the CPU 20 attaches to this handwritten data additional information for specifying the page of the musical score at that time, the file name of the musical score data, the display layer (second layer L2), and the like. If there is handwritten data corresponding to the page after the change, the image of the locus at this position is displayed in the second layer. L2
, And nothing is displayed on the second layer L2. Note that FIG. 9 is a conceptual diagram when handwritten data for one page is added with additional information and stored, and FIG.
It is a conceptual diagram in the case where additional information is added to handwritten data for pages and stored, and any method may be adopted. As a result, the player can directly write a handwritten image such as a character or a pattern on the electronic musical score with a stylus pen or the like, similar to a general paper-based musical score.

Here, in the present embodiment, the case where the handwritten image (the locus of the pressed position) is displayed on the second layer L2 is shown, but the CPU 20 is a layer for displaying the handwritten image according to the setting change by the operator. Is arbitrarily changed, and the same processing is performed when displaying on other layers (third layer L3 to nth layer Ln). Therefore, it is possible to write a handwritten image in a plurality of layers. The operator may arbitrarily set the color and the size of the dots (line thickness) when the handwritten image is displayed for each layer. Further, the layer for displaying the score image and the operation button groups 30 and 40 is not limited to the first layer L1, and it is possible to display the score image and the handwritten image in any layers in different ranges. Note that the score image and the images of the operation button groups 30 and 40 are not limited to the same layer, and may be displayed in different layers.

Further, the CPU 20 controls the display / non-display of the first layer L1 to the nth layer Ln according to the operation and setting of the operation button group 30 to thereby display the score and the handwritten image according to the operation of the operator. Are displayed at the same time, only the score is displayed, only the handwritten image is displayed, only the handwritten image of a predetermined layer is displayed, and the switching display is performed. That is, since the musical score type information processing apparatus 1 displays the musical score and the handwritten image in different layers, it is possible to easily perform the non-display process of the musical score or the handwritten image.

When the musical score saving operation button of the operation button group 30 is operated, the CPU 20 reads out the handwritten data and the additional information displayed on the second layer L2 to the nth layer Ln, and stores them as these data. First layer L1
The musical score data displayed in 1 is recorded in the user data memory 25 as one musical score data. And the CPU 20
When displaying this score data, if it is determined that there is handwriting data corresponding to the page of the score to be displayed, based on the additional information included in the score data, the image of this handwriting data is assigned to the specified layer. By displaying the score, it is possible to display the score including the previously written handwritten image.

Further, the CPU 20 controls the operation button group 30.
When the operation button for saving the handwritten image of is operated,
The handwritten data of the designated layer is read out together with the additional information, and is stored in the user data memory 25 or the memory card. When the CPU 20 is instructed to read the handwritten data recorded in the user data memory 25 or the memory card, the score data corresponding to the score data memory 24 is recorded based on the additional information included in this data. If the score data is recorded, the score data is read and displayed. If there is handwritten data corresponding to the displayed page of the score, the image of the handwritten data is read as described above. Is displayed on the specified layer. This allows
If the operator stores only the handwritten data of himself / herself in a memory card and carries it, the operator can display the score including the handwritten image written by his / her own score type information processing apparatus 1 on another score type information processing apparatus 1. be able to.

As a result, in this score display mode, not only is it possible to display successive pages of the score at the same time and pages are turned over, but the displayed score is the same as for a general paper-based score. You can freely write in. Further, the image data of the handwritten image is managed in a layer different from that of the image data of the score, and the image data of the handwritten image is subjected to display processing and storage processing with additional information describing the correspondence with the image data of the musical score. Therefore, a handwritten image can be displayed according to the page of the displayed score, and the score including the previously written handwritten image is displayed not only on the same score type information processing device 1 but also on another score type information processing device 1. be able to.

B. Shooting Mode The shooting mode is a mode for shooting the performer, and when this mode is selected, the CPU 20 displays the shot image 50 of the camera 7 on the display screen 3. As shown in the display example of the shooting mode in FIG. 11, the shot image 50 is displayed on the display screen 3.
In addition, an operation button group 60 for instructing processing such as recording start, recording stop, and reproduction is displayed, and the score displayed on the display screen 2 before switching to the shooting mode or the performer. Score data D specified by
Any of the musical scores corresponding to M is displayed.

Then, in response to the operation of the operation button group 60, the CPU 20 causes the user data memory 25 to store the moving image data corresponding to the photographed image 50 input via the camera 7 and the voice data input via the microphone 8. A recording process for recording in synchronization with the recording process, a recording stop process, a reproduction process of moving image and audio data recorded in advance in the user data memory 25, or the like is performed. In this recording process, the moving image data is MPEG
(Moving Picture Experts Group) 2 or motion JPEG (Joint Photographic Experts Group) or other video encoding algorithm is used for compression encoding processing, while audio data is compression encoded by MPEG1 (layer 2) or other audio encoding algorithm. May be performed and recorded. Further, only moving image data may be recorded.

Also in this mode, similar to the score display mode, the score operation can be performed by operating the operation button group 30, and the handwritten image can be written in the electronic score or only the handwritten data can be saved. Is. In this mode, since the electronic musical score is displayed only on the display screen 2, when the page feed button or the page return button is pressed, unlike the case of the score display mode,
The electronic musical score is changed page by page. The captured image 50 may be displayed on the display screen 2 and the electronic musical score may be displayed on the display screen 3, or the operator may select which display screen to display the captured image 50. Good.

As a result, in this photographing mode,
Since the electronic musical score and the photographed image 50 can be displayed at positions adjacent to each other, the performer plays an instrument while watching the electronic musical score displayed on one display screen 2, and at the same time displays the other display screen 3. You can easily shoot while monitoring your own performance displayed in.

C. Recording mode The recording mode is a mode for recording performance sounds and singing voices.
When this mode is selected, the CPU 20 starts recording,
An operation button group (not shown) for instructing processing such as recording stop and reproduction is displayed on the display screen 3, and voice data input through the microphone 8 in response to the pressing operation of these operation buttons is recorded in the user data memory 25. Or reproduce the voice data recorded in the user data memory 25.

Also in this mode, similarly to the shooting mode, the display screen 2 displays the score displayed before switching to the recording mode or the score corresponding to the score data DM designated by the performer. To be done. This allows
The performer plays the musical instrument while looking at the electronic score displayed on the display screen 2 on one side, and at the same time, operates the operation buttons displayed on the display screen 3 on the other side to simplify his / her performance sound and singing voice. Can be recorded.

D. Communication Mode The communication mode is a mode for transmitting moving image data, audio data, etc. recorded in the user data memory 25 to the external device or receiving various data from the external device in the shooting mode or the recording mode. When selected, C
The PU 20 displays an operation button group (not shown) for performing communication on either or both of the display screens 2 and 3. In this case, the electronic musical score may be displayed on any of the display screens 2 and 3 as in the above mode. Then, data communication is performed with the destination PC specified by the operator or another musical score information processing apparatus 1 in accordance with the operation of these operation button groups. It is also possible to browse the Internet by activating a browser according to the operation of the operation button group and connecting to the Internet via the router. Further, the handwritten data may be communicated with the external device together with the additional information.

E. Lesson mode The lesson mode is a mode in which bidirectional real-time communication of shooting data, voice data, MIDI data, and handwritten data is performed with another musical score type information processing apparatus 1. Also in this mode, similarly to the shooting mode, the score and the operation button group 30 for instructing page operations of the score are displayed on the display screen 2, and the player plays the musical instrument while watching the score. be able to.

Here, when using this mode, it is necessary to connect the musical score type information processing apparatus 1 and the MIDI interface of the musical instrument to each other. In the example shown in FIG. 12, this musical score type information processing apparatus is used. 1 is placed on a music stand of a piano (MIDI musical instrument) so that MIDI connection is possible. In this case, the musical score type information processing apparatus 1 and the musical instrument may be connected by a cable, but if a short-distance wireless communication method such as infrared rays, microwaves, electromagnetic induction is applied, it can be simply placed on the music stand. MIDI
It is possible to complete the connection.

As shown in FIG. 12, in this mode, another musical score type information processing apparatus 1 (musical score information on the teacher or student side) designated by the performer (student or teacher) via a network such as the Internet. When connected to the processing device), each CPU 20 causes the musical score data processing device 1 of the connection destination to transmit the moving image data input via the camera 7 in real time, and receives the musical score data processing device 1 of the connection destination. The moving image data is reproduced in real time and displayed on the display screen 3. In this case, by adjusting the direction of the camera 7 to the direction of photographing the keyboard of the musical instrument,
As shown in, both teachers and students can see the other party's keyboard performance.

At the same time, each of the CPUs 20 outputs M from the musical instrument.
When IDI data is input, this MIDI data is transmitted to the musical score type information processing apparatus 1 of the connection destination, while MIDI data is received from the musical score type information processing apparatus 1 of the connection destination,
The MIDI data is transmitted to the musical instrument to drive the keyboard of the musical instrument or perform automatic performance. As a result, when the student plays the musical instrument, the teacher's musical instrument is automatically played (automatically accompanied by keyboard drive). Similarly, when the teacher plays the musical instrument, the student's musical instrument is automatically played. You can actually see it with your eyes and ears.

Further, each CPU 20 determines the signal level of the voice input through the microphone 8. If the signal level exceeds a predetermined value, it is determined that the voice is emitted and the voice is input during the determination period. The voice data of the generated voice is transmitted to the musical score type information processing apparatus 1 of the connection destination, while the voice data received from the musical score type information processing apparatus 1 of the connection destination is reproduced and emitted from the speaker 9. With this, it is possible to have a conversation between the teacher and the student.

Further, when each CPU 20 detects that a position other than the operation button groups 30 and 40 on the display screen 2 has been pressed via the touch panel 11A, the pressed position and the position are the same as in the score display mode. The handwritten image is displayed by displaying the locus of No. at the same position on the display screen 2. Also in this case, like the score display mode,
The handwritten image is displayed in a layer different from the layer displaying the score and the reproduced image, and only the handwritten image can be saved.

Each CPU 20 transmits handwritten data (including additional information) displayed in the layer in response to the operation of the operation button group 30 to the musical score type information processing apparatus 1 of the connection destination, while the connection destination of the connection destination. When handwritten data is received from the musical score type information processing apparatus 1, the corresponding handwritten image is displayed on a preset layer. In this case, the received handwritten image is a preset layer different from the layer displaying the score and different from the layer displaying the handwritten image input by the operator of the musical score information processing apparatus 1. Displayed in the layer.

As a result, not only can the student and the teacher write a handwritten image on the score displayed on their own score type information processing apparatus 1, but also the handwriting that the other party (teacher or student) at the connection destination wrote on the score. Since the image is automatically displayed on the same page of his own score, for example, the student can obtain his own score in which the lesson instruction from the teacher is described. In addition, the handwritten image written by the other party of the connection destination is displayed in a separate layer, so if you change the display / non-display settings of this layer, you will only see the score including only your own handwritten image and the handwritten image of the teacher. It is possible to freely display the score including the.

As a result, in this lesson mode, not only the distance between the teacher and the student can be visually and audibly confirmed by the teacher and the student in spite of the distance lesson, but also the teacher can listen to the score of the student. You can write the lesson instructions and evaluations directly in, and you can conduct private lessons of teachers and students in real time.

(2) Second Embodiment The musical score type information processing apparatus 100 according to the second embodiment has an orchestra mode suitable for use in an orchestra performance, and the musical score data DM used in this mode.
Since it is the same as the musical score type information processing apparatus 1 according to the above-described first embodiment except that is different, the same portions are denoted by the same reference numerals, and overlapping description will be omitted.

As shown in FIG. 13, the musical score data (hereinafter referred to as "musical score data for orchestra") DMO used in the orchestra mode includes the first musical score data for each musical instrument part (part 1, part 2, ... Part m) of the orchestra. It is configured to have the score data DM described in one embodiment. As a result, in the musical score type information processing apparatus 100, the CPU 20
However, by reading out pitch information, rhythm information, and page information of one preset musical instrument part from the orche score data DMO, image data of the musical score of the musical score is generated as in the first embodiment. It is possible to display successive pages of the score on the display screens 2 and 3.

Further, in the musical score type information processing apparatus 100, the CPU 20 reads out pitch information and rhythm information of a plurality of preset musical instrument parts from the musical score data DMO for the orchestra, and generates a musical score of a musical notation for each musical instrument. Then, by generating image data of a musical score of musical notation in which measures of each musical instrument part are arranged vertically, as shown in FIG. 14, a plurality of musical instrument parts (flauti, Oboi, Clarinetti, ..., Co
ntrabasso) showing successive pages of the score screens 2 and 3
It can be displayed on. Note that the musical score data also includes information indicating the name of the musical instrument, the title of the song, the name of the composer, and the like.

Next, the orchestra mode will be described. The orchestra mode is a mode in which a plurality of musical score type information processing devices 100 form a wireless network (for example, a “piconet” in Bluetooth), and music score data DMO (DM) and handwritten data are communicated within this wireless network. The wireless network may be configured in advance, or may be configured with a plurality of score-type information processing apparatuses 100 adjacent to each other in the same mode when the orchestra mode is selected.

When this mode is selected, in each musical score information processing apparatus 100, the CPU 20 displays an operation screen (not shown) for selecting an instrument part, prompting the operator to select an instrument part. It is like this. Then, from any score type information processing device 100 in this wireless network, the score data DMO for the orchestra.
Is broadcast to all score-type information processing devices 1 in this wireless network, and the score data DM for
O is transmitted, and the musical score of the musical instrument part preset by each CPU 20 is displayed on each musical score type information processing apparatus 100.

Therefore, the musical score type information processing apparatus 10 of the performer
At 0, the musical score of the musical instrument part played by the performer is displayed, and in the musical score type information processing apparatus 100 of the conductor, FIG.
As shown in, the score including all the instrument parts is displayed. In this case, in each musical score type information processing apparatus 100, as in the musical score display mode, the page of the musical score is changed according to the operation of the operation button group 30 or the handwritten image input by the operator is displayed. It has become. Even in this mode, the handwritten image is
Similar to the score display mode, score images and operation buttons 3
It is displayed in a layer different from the layer displaying 0 and 40, and it is possible to switch between non-display / display of handwritten images and save.

In the musical score type information processing apparatus 100 displaying a plurality of musical instrument parts, that is, the conductor's musical score type information processing apparatus 100, when a handwritten image is input, the CPU 20 is operated by handwriting. Based on the position (the detected pressed position), the processing for detecting the musical instrument part of the musical score that is close to the position is performed. When the CPU 20 detects a musical instrument part close to the position, the CPU 20 broadcasts corresponding handwritten data with additional information including the musical instrument part as shown in FIG. In this case, the score displayed on the conductor's score information processing apparatus 1 (see FIG. 14) and the score displayed on the performer's score information processing apparatus 1 (see FIG. 6) have different page values due to the difference in the amount of information. Since they are different, the additional information includes, in addition to the musical instrument part, information for specifying the display position of the handwritten data (information for specifying a bar, information for specifying a musical sound, etc.).

On the other hand, the musical score type information processing apparatus 100 of the conductor
In the musical score type information processing apparatus 100 other than the above, upon receipt of the handwritten data, whether each CPU 20 matches the musical instrument part included in the header with the musical instrument part preset in the musical score type information processing apparatus 100. Is judged,
If they do not match, the received data is deleted. On the other hand, if they match, the image of the handwritten data is displayed in the designated layer on the specified bar portion of the score to be displayed based on the additional information added to the handwritten data. That is, the handwritten image written by the conductor on the electronic musical score is automatically transmitted to the musical score type information processing apparatus 100 of the performer who is displaying the musical score of the musical instrument part closest to the writing position and is displayed at the same position. Therefore, the performer's instructions are written in the score displayed on his or her own score type information processing apparatus 1, so that the performer can quickly and accurately know the instruction contents and the position to be instructed by the conductor. it can.

As a result, in this orchestra mode, as shown in FIG. 16, the conductor's handwritten instructions are immediately displayed on the corresponding musical score type information processing apparatus 100 of the performer, so that the performer promptly conducts the command. The instruction of the conductor can be understood, and the instruction of the conductor can be stored in the musical score type information processing apparatus 100 so that the instruction can be confirmed later. Even in the musical score type information processing apparatus 100 of the performer, when a predetermined operation is performed, the handwritten image of the player can be transmitted to the musical score type information processing apparatus 100 of the conductor, and the musical score type information processing apparatus 1 of the conductor can be transmitted. It can be displayed.

(3) Supplement The invention of the present application is not limited to the above-described embodiment, but can be implemented in various modes. For example, the following modifications are possible. (3.1) In each of the above-described embodiments, when the musical score type information processing apparatuses 1 and 100 are in the lesson mode, shooting data, audio data, and MIDI data with the musical score type information processing apparatuses 1 and 100 of the connection destination. Although the case where two-way communication of handwritten data is performed was described, as shown in FIG. 17, when a teacher is on the student side, communication of shooting data, voice data, and MIDI data can be performed depending on the setting of the operator. Alternatively, only handwritten data may be communicated without performing the communication.
Further, the present invention can be applied not only to individual lessons but also to lessons given by a teacher to a plurality of students. In this case, the teacher may specify the musical score type information processing apparatus 1 of the student to whom the handwritten data is sent.

(3.2) In the orchestra mode of the second embodiment, the musical score type information processing apparatus 100 on the receiving side of the musical score data or handwritten data receives the musical piece based on a preset musical instrument part. The case where the score type information processing device 100 on the transmitting side is stored with the destination and the musical instrument part of the score type information processing device 100 accommodated in the wireless network is described. Based on the stored data, the pattern information processing apparatus 100 may transmit the musical score data or handwritten data for each musical instrument part to the musical score information processing apparatus 100 of a predetermined destination. It should be noted that the information about the destination and the musical instrument part of the musical score information processing apparatus 100 may be shared by transmitting and receiving their own data between the musical score information processing apparatuses 100 when configuring the wireless network.

(3.3) In each of the above embodiments,
This musical score type information processing device 1 or 100 is a musical score display mode,
I described the case of having shooting mode, recording mode, communication mode, lesson mode and orchestra mode, but at least score display mode is required so that score and handwritten image can be displayed in different layers, or only handwritten image can be saved. , Just insert it.

(3.4) In each of the above embodiments,
Although the case where handwriting is performed on the score image with the touch panels 11A and 11B has been described, various pointing devices (position detecting means) such as a tablet and a mouse other than the touch panel can be widely applied.

(3.5) In each of the above embodiments,
Although the case where the musical score operation and the like are performed with the software key has been described, a hardware key for performing these operations may be provided. Further, the various operations are not limited to being performed by the software key or the hardware key, and the various operations may be performed based on the voice input through the microphone 8.

(3.6) In each of the above-mentioned embodiments, the case where the musical score type information processing apparatus 1 or 100 is provided with a display device (display screen, liquid crystal display panel, etc.) and an electronic musical score or the like is displayed on this display device. As described above, the electronic musical score may be displayed on another display device connected to the musical score type information processing apparatus 1 or 100.

(3.7) In each of the above embodiments,
The case of writing, saving, inserting, and remote lesson of a handwritten image in the musical score type information processing apparatus 1 or 100 that displays an electronic musical score has been described. However, even when information other than the electronic musical score is displayed, the handwritten image can be written. It may be enabled, and not only the musical score type information processing apparatus but also a PC or PDA (Personal Digital Assy
Computers (including information processing devices) such as istants)
The present invention can be widely applied to.

Therefore, various teaching systems, for example, a test describing problems such as national language and arithmetic are displayed on this kind of device, and the answer handwritten in the test from the device on the student side is displayed on the device on the teacher side. The present invention can be widely applied to a learning system in which a scoring result is transmitted and handwritten and input from the teacher's device to the student's device. Also in this case, each device manages (displays, sends, and saves) the test, the answer (handwriting input), and the scoring result (handwriting input) in different layers, and the handwritten data includes additional information including the page corresponding to the test. By attaching it to the test and associating it with the test, it is possible to realize a learning system that uses a test of multiple pages, and you can review many times by hiding the previous answers and scoring results.

(3.8) Further, in each of the above embodiments,
The case where a control program for performing layer display of a handwritten image, storage of handwritten data of a specified layer, transmission / reception, insertion and the like is stored in the musical score information processing apparatus 1 or 100 in advance has been described. As shown in 18,
The control program may be recorded in a computer-readable recording medium 120 such as a magnetic recording medium, an optical recording medium, or a semiconductor recording medium, and the computer 130 may read and execute the program. Also, FIG.
As shown in FIG. 9, the control program is stored in the server 140, and the server 140 transmits the control program to an information processing apparatus such as the musical score information processing apparatus 1 or 100 or a computer (PC) 150 that has been requested to transmit via the network. You may do it.

[0074]

As described above, according to the present invention, it is possible to freely write on a display image such as an electronic score, save only the written handwritten image data, and send / receive it to / from another device. Therefore, it can be applied to various learning systems.

[Brief description of drawings]

FIG. 1 is an external view of a musical score type information processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a display screen of the musical score type information processing apparatus.

FIG. 3 is an external view showing a state in which the housing of the musical score information processing apparatus is closed.

FIG. 4 is a block diagram showing an electrical configuration of the musical score type information processing apparatus.

FIG. 5 is a diagram showing musical score data.

FIG. 6 is a diagram for explaining a score display mode of the score type information processing apparatus.

FIG. 7 is a diagram illustrating a layer structure of a display image when a musical score type information processing apparatus displays a musical score.

FIG. 8 is a diagram illustrating a layer structure of a score and a display image when a handwritten image is displayed by the score information processing apparatus.

FIG. 9 is a diagram for explaining a method for managing handwritten data.

FIG. 10 is a diagram for explaining another management method of handwritten data.

FIG. 11 is a diagram for explaining a shooting mode of the musical score type information processing apparatus.

FIG. 12 is a diagram for explaining a lesson mode of the musical score type information processing apparatus.

FIG. 13 is a diagram showing musical score data for an orchestra.

FIG. 14 is a diagram for explaining an orchestra mode of the musical score type information processing apparatus.

FIG. 15 is a diagram for explaining the transmission of handwritten data in the orchestra mode.

FIG. 16 is a diagram for explaining an example of use of an orchestra mode of the musical score information processing apparatus.

FIG. 17 is a diagram for explaining another usage example of the lesson mode of the musical score information processing apparatus.

FIG. 18 is a diagram for explaining a case where a control program is recorded in a computer-readable recording medium.

FIG. 19 is a diagram for explaining a case where a control program is transmitted from a server via a network.

[Explanation of symbols]

1, 100 ... Musical note type information processing apparatus, 2, 3 ... Display screen, 4 ... Operation switch group, 30, 40 ... Operation button group, 5, 6 ... Housing, 7 ... Camera, 8 ... ... microphone, 9 ... speaker, 10A, 10B ... liquid crystal display panel, 11A, 11B ... touch panel (position detecting means), 20 ... CPU (display control means), 21 ... bus, 22 ... RAM, 23 ...... Control program memory,
24 ... Score data memory, 25 ... User data memory, 26 ... Wireless communication unit, 27 ... MIDI interface, 28 ... Sound source, 50 ... Shooting image, DM, DM
O ... Music score data, L1 ... Ln ... Layer.

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G10H 1/00 G09G 5/36 520M F term (reference) 5B069 AA01 AA09 BA01 BA04 BB16 CA18 DD15 JA06 LA03 5C082 AA01 AA27 BA02 BA12 BA26 BB26 BB53 BD02 CA56 CB06 DA22 DA73 DA86 DA89 MM02 MM09 MM10 5D378 TT02

Claims (10)

[Claims] 1. A display device, display data recording means for recording display data, image creating means for creating image data based on the display data, and the display device designated by an operator. Position detection means for detecting the position on the screen, image data created by the image creation means, and handwritten data corresponding to the locus of the position detected by the position detection means are managed in different layers. An image corresponding to the image data, and a display control unit for displaying the handwritten image corresponding to the handwritten data in an overlapping manner on the display device, wherein the display control unit displays the image corresponding to the image data. It is displayed on a predetermined layer page by page, and the handwritten image corresponding to the handwritten data is displayed on another layer different from the layer. In addition to displaying the handwriting data, the handwriting data is stored in association with the page of the image displayed at that time, and the handwriting corresponding to the handwriting data when the image of the page is displayed again on the layer. An information processing apparatus for displaying an image on the other layer. 2. The display control means reads out each handwritten data stored in association with each page of the image,
2. The recording control means according to claim 1, further comprising recording control means for recording each handwritten data on a predetermined recording medium with additional information for specifying a corresponding page and the display data. Information processing equipment. 3. A reading unit for reading the handwritten data and the additional information recorded on the recording medium, wherein the display control unit displays the handwritten image corresponding to the handwritten data read by the reading unit for the additional information. The information processing apparatus according to claim 2, wherein when the image corresponding to the display data and page specified by is displayed on the layer, the image is displayed on the other layer. 4. The handwritten data corresponding to the handwritten image displayed on the other layer is read out, and at that time, additional information for identifying the page of the image displayed on the layer is added and transmitted to the outside. The information processing apparatus according to claim 1, further comprising a transmitting unit. 5. The receiver further includes a receiving unit that receives the handwritten data and the additional information transmitted from the outside, and the display control unit displays the handwritten image corresponding to the handwritten data received by the receiving unit as the additional information. The information processing apparatus according to claim 4, wherein when the image of the page specified by is displayed on the layer, the image is displayed on a predetermined layer. 6. The image creating means creates image data of a musical score including a plurality of musical instrument parts based on the display data, and the display control means creates a plurality of musical instruments based on the image data. The score including the part is displayed in page units on the layer, and the transmitting unit displays the score on the layer based on the position detected by the position detecting unit when transmitting the handwritten data to the outside. 5. The information processing apparatus according to claim 4, wherein the additional part information is detected by detecting an instrument part close to the position, and the additional information includes information specifying the detected instrument part. 7. The receiving means determines, based on the information for identifying the musical instrument part included in the received additional information, whether or not the musical instrument part matches a musical instrument part preset in the information processing apparatus, and determines that the musical instrument parts match. The information processing apparatus according to claim 6, wherein the received additional information and the handwritten data are deleted when not performed, and the received additional information and the handwritten data are output to the display control unit when they match. 8. An image creating step of creating image data based on the display data recorded in the display data recording means, and a position detecting step of detecting a position on the screen of the display device designated by the operator. By managing the image data and the handwritten data corresponding to the detected locus of the position in different layers, an image corresponding to the image data and a handwritten image corresponding to the handwritten data are displayed on the display device. And a display control step of displaying the image data in a layer that is predetermined in page units, the image corresponding to the image data is displayed in the display control step, and the handwritten image corresponding to the handwritten data is In addition to displaying in real time on another layer different from the layer, the handwritten data is displayed at that time. The information processing apparatus is characterized in that it is stored in association with a page of the image, and when the image of the page is displayed again on the layer, a handwritten image corresponding to the handwritten data is displayed on the other layer. Control method. 9. A display device, and a computer having a position detection means for detecting a position on the screen of the display device designated by an operator, a display data recording means for recording display data, and the display. Image creating means for creating image data based on the data for use, image data created by the image creating means, and handwritten data corresponding to the locus of the position detected by the position detecting means in different layers. It is a control program for causing an image corresponding to the image data and a handwritten image corresponding to the handwritten data to function as display control means for superimposing and displaying on the display device by managing the display control means. Displays an image corresponding to the image data on a predetermined layer for each page, and corresponds to the handwriting data. The handwritten image to be displayed on another layer different from the layer in real time, the handwritten data is stored in association with the page of the image displayed at that time, and the image of the page is stored as described above. A control program that causes a handwritten image corresponding to the handwritten data to be displayed on the other layer when redisplayed on the layer. 10. A display device, and a computer having a position detecting means for detecting a position on the screen of the display device designated by an operator, a display data recording means for recording display data, and the display. Image creating means for creating image data based on the data for use, image data created by the image creating means, and handwritten data corresponding to the locus of the position detected by the position detecting means in different layers. A computer-readable record that records a control program that causes the display device to superimpose an image corresponding to the image data and a handwritten image corresponding to the handwritten data on the display device by managing the image. A medium, wherein the display control unit preliminarily displays an image corresponding to the image data in page units. Displayed in a layer, the handwritten image corresponding to the handwritten data is displayed in real time on another layer different from the layer, and the handwritten data is associated with the page of the image displayed at that time. Computer-readable recording control program characterized in that when the image of the page is re-displayed on the layer, it functions as means for displaying the handwritten image corresponding to the handwritten data on the other layer. Possible recording medium.