JP5747974B2 - Information processing apparatus and program - Google Patents

Description

  The present invention is used in combination with a musical sound reproducing apparatus capable of displaying a display content such as a musical score or lyrics related to a musical sound generated (sounded) through a speaker or the like on a separately prepared external display. The present invention relates to an information processing apparatus having the above-mentioned external display suitable for performing, and a program used therefor. In particular, the present invention relates to a technique for updating display contents according to the progress of musical sounds and reflecting them on an external display.

  Conventionally, display contents such as musical scores and lyrics are displayed on a display provided in the main body, and the melody that is pronounced based on the reproduction of music data desired by the user and the progress of musical sounds such as automatic accompaniment, There is known an electronic musical instrument that automatically updates a score and lyrics displayed on a display. In such an electronic musical instrument, for example, musical score display control data is embedded in advance in music data (for example, MIDI data) for automatic accompaniment, and the musical score display control data is displayed in accordance with the reading of the musical sound. The score display on the instrument is automatically updated. According to this, since the user does not have to perform the so-called turning operation of turning the score by himself / herself as necessary, the user is devoted to practicing playing the melody etc. by actually operating the performance operator. can do.

  By the way, some electronic musical instruments do not have a display device in the first place, or there are cases where only a small display device that does not have sufficient display capability for displaying a musical score or the like is included. Of course, the score cannot be displayed. Therefore, in order to realize the user's desire to display a score even in such a case, the score can be displayed on an external display configured separately from the instrument body. There is something. An example of an apparatus for displaying a musical score related to a musical tone being reproduced on an external display device is, for example, an apparatus described in Patent Document 1 shown below.

  In the device described in Patent Document 1, a musical tone including musical instrument performance sounds and voices (sounds emitted) from a musical instrument main body through a speaker is acquired by a microphone, and beats (beats) are obtained by analyzing the acquired musical sounds. Position) and pitch are extracted to specify the musical performance position (playback position), and the display of the musical score is automatically updated based on the specified musical performance position (playback position). . In this case, the display update of the score on the display unit is performed by, for example, displaying a half-page score including one or more bars in a display area divided into two upper and lower stages, and a score in a range where the performance is finished. This is realized by switching control so that the score of the next half page following the score in the range displayed in the other display area is displayed with respect to one of the upper and lower display areas. That is, in the score display control in this apparatus, the score for one page is updated by half a page as the musical tone progresses.

  In addition, there has been proposed one in which music information is superimposed on audio data as a digital watermark and content is specified based on audio data on which the music information received on the external device side is superimposed (see Patent Document 2 shown below). .

  Furthermore, recently, for example, a music application program (software program) that realizes a predetermined music function is applied to an information processing apparatus having a touch-sensitive touch panel display such as a slate personal computer (also called a tablet terminal) or a smartphone. By installing the program and operating the program, the apparatus can be operated as an external display or the like for displaying the display contents such as the score and lyrics as described above.

JP 2001-265325 A Japanese Patent Laid-Open No. 2002-314980

As described above, in the conventional apparatus described in Patent Document 1, since the display control of the musical score is performed based on the musical sound emitted from the electronic musical instrument through the speaker, the performance is started without the musical sound being emitted. Previously, a score related to the musical tone to be played cannot be automatically selected and displayed. Therefore, the user needs to manually call the musical score of the desired musical tone and the musical score at the performance start position before starting the performance, but the operation for this must be performed separately from the operation for selecting the music data. It was troublesome.
In addition, according to the musical score display control based on the musical performance position specified by analyzing the musical sound emitted through the speaker, particularly when the performance tempo (or playback tempo) is not constant or the user If the performance technique of the music is immature, beat detection and pitch detection cannot be performed well, and it takes time to recognize the performance position and it is easy to misrecognize the performance position. There was a great risk that the update would not synchronize well. In addition, even when jumping to non-continuous performance positions due to fast-forwarding or rewinding, it takes time to re-recognize the performance position, and it takes time to make the score follow the performance position. .
In addition, the display of music scores was controlled based on a wide range of audible sound (music signal) that would be difficult to pick up with a microphone, depending on the noise generated in the surroundings. Accordingly, there is a problem that the automatic update of the score is likely to be delayed because the performance position of the score cannot be correctly extracted.

  On the other hand, since the device described in Patent Document 2 simply specifies the content and does not perform any linkage operation between the devices, the display content such as a score and lyrics displayed on the external display is originally used as a musical sound. It was difficult to update with progress.

  The present invention has been made in view of the above points, and when displaying and updating display content related to a musical sound emitted through a speaker or the like in accordance with the progress of the musical sound reproduction, it is in accordance with the musical sound reproduction position. An object of the present invention is to provide an information processing apparatus and a program used for the information processing apparatus that can reflect a display of an appropriate location on a display device at an appropriate timing.

An information processing apparatus according to the present invention includes a content storage unit that stores a large number of display contents including a plurality of pages, a display unit that displays display content, and a signal acquisition unit that acquires a signal of an audible band, the audible band The signal includes a sound signal modulated by an external device so as to include music information for specifying the music and position information indicating the performance position of the music specified by the music information. from the device and the another sound with can be output superimposed on the other sounds are separately signal alone can be outputted, and the signal acquisition means, said audio band signal acquired by the signal acquiring means, if the acoustic signal including the said song information the position information is included, an extraction means for extracting the music information and the position information from the acoustic signals, display configuration corresponding to the extracted song information Specifying means for specifying a Ntsu from many display content that is the storage, the determining the display page of the specified display content according to the extracted position information, on said display means on the basis of the display page Display control means for performing display control for updating a display mode of display content to be displayed.

According to the present invention, an audible band signal including an acoustic signal modulated by an external device as song information and curved information includes the position information indicating the playback position of the music to be specific for identifying music the signal acquired Ri by the acquisition means, by the extraction unit, when the acoustic signal in the acquired audible band signal is contained, it extracts the music information and the position information from the acoustic signal. Then, the identifying from among a large number of display content already stored the display content corresponding to the extracted song information, the determining the display page of the specified display content according to the extracted position information, the display page The display mode of the display content displayed on the display means is updated based on the above.

According to this, get an audible band signal including an acoustic signal modulated by an external device as song information and curved information includes the position information indicating the playback position of the music to be specific for identifying music Since the display content of the display device is controlled based on the music information and the position information included in the sound signal, the music performance starts as well as the music performance starts. Since display control of display content can be performed based on the music information and the position information (for example, information indicating the performance start position) included in the acoustic signal acquired previously, prompt display of the display content is possible. It becomes possible. Moreover, it takes less time than the conventional process of analyzing the musical sound signal and specifying the performance position (playback position) of the music . That is, since there is no fear that it takes a long time to recognize the playback position (musical performance position) or erroneously recognizes the playback position as in the past, in the present invention, especially when the playback tempo is not constant or in the middle of the music For example, when playback is started from the beginning, the display of the display content can be accurately followed by the playback position of the musical sound (musical performance position) . Furthermore, if the acoustic signal is a signal composed of only a high band, it is less susceptible to noise than the conventional technique for analyzing a musical sound signal composed of a wide band and specifying a performance position. For this reason, even if the reception environment is bad, it is difficult to be affected, and the correct position can be displayed. Therefore, when the display content related to the musical sound is updated in accordance with the progress of the musical sound, it is possible to reflect the display of an appropriate portion according to the musical performance position on the display device at an appropriate timing.

  The present invention can be constructed and implemented not only as a device invention but also as a method invention. Further, the present invention can be implemented in the form of a program of a processor such as a computer or a DSP, or can be implemented in the form of a storage medium storing such a program.

According to the present invention, an audible band signal including an acoustic signal modulated by an external device as song information and curved information includes the position information indicating the playback position of the music to be specific for identifying music Since the display content of the display device is controlled according to the acquired music information and position information included in the sound signal, the music performance is started as well as the music performance is started. Since display control of display content can be performed based on the music information and the position information (for example, information indicating the performance start position) included in the acoustic signal acquired before the display, prompt display of the display content Is possible. Further, when updating together display content related to music to the progress of the music can be reflected on the display to display the appropriate places in accordance with the performance position of the music at the right time. Moreover, it takes less time than the conventional process of analyzing the musical sound signal and specifying the performance position (playback position) of the music . That is, if it contains an acoustic signal in the acquired audio band signal, said extracting an acoustic signal song information and position information from, since the display based on the extracted, as in the conventional reproduction position ( In the present invention, particularly when the playback tempo is not constant or when playback is started from the middle of the song, etc. The display of the display content can be accurately followed by the performance position of the music .

1 is a block diagram of a hardware configuration showing an embodiment of an overall configuration of a music sound reproducing device according to the present invention. 1 is a block diagram of a hardware configuration showing an embodiment of an overall configuration of an information processing apparatus according to the present invention. It is a conceptual diagram which shows the example which displayed the score as display content displayed on the display of an information processing apparatus. It is a functional block diagram for demonstrating the outline | summary of the score control function in a musical tone reproduction apparatus. It is a functional block diagram for demonstrating the outline | summary of the score display function in information processing apparatus. It is a conceptual diagram which shows the data structure of a score display control table. It is a flowchart which shows one Example of a score control process. It is a flowchart which shows one Example of a score display process. It is a timing chart which shows an example of each operation | movement timing of the score control process according to a musical tone reproduction mode, and a score display process. It is a timing chart which shows an example of each operation | movement timing of the score control process according to the performance tempo of a musical tone, and a score display process.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a hardware configuration block diagram showing an embodiment of the overall configuration of a musical sound reproducing apparatus according to the present invention. The musical sound reproducing apparatus shown in the present embodiment is, for example, an electronic musical instrument capable of generating (sounding) musical sounds based on at least reproduction of arbitrary music data, and includes a microprocessor unit (CPU) 1, a read-only memory ( (ROM) 2 and a random access memory (RAM) 3. The CPU 1 controls the overall operation of the music sound reproducing device (corresponding to the control unit in FIG. 4). A ROM 2, a RAM 3, a performance operator 4, a display unit 5, a setting operator 6, a tone generator unit 7, a storage device 8, and a communication interface (I / F) 9 are connected to the CPU 1 via a data and address bus D. It is connected. Further, the CPU 1 is connected to a timer 1A for measuring the interrupt time and various times in the timer interrupt process (interrupt process). For example, the timer 1A generates a clock pulse, and gives the generated clock pulse to the CPU 1 as a processing timing command or to the CPU 1 as an interrupt command. The CPU 1 executes various processes according to these instructions.

  The ROM 2 stores various control programs executed by or referred to by the CPU 1, various data, and the like. The RAM 3 is used as a working memory for temporarily storing various data generated when the CPU 1 executes a predetermined program, or as a memory for temporarily storing a currently executing program and related data. The A predetermined address area of the RAM 3 is assigned to each function, and is used as a register, a flag, a table, a temporary memory, or the like.

  The performance operator 4 is, for example, a keyboard provided with a plurality of keys for selecting the pitch of a musical tone, and has a key switch corresponding to each key. The keyboard or the like) can be used not only for manual performance by the user's own playing but also for selection of music data to be reproduced.

  The display unit 5 is, for example, a display (display) configured by a liquid crystal display panel (LCD), a CRT, or the like. Various data or the control state of the CPU 1 is displayed. However, unlike the display unit 15 of the information processing apparatus to be described later, the display unit 5 provided in this apparatus is a display that can only perform simple display, that is, the number of characters / symbols and the number of characters that can be displayed are limited. This is a display device with a very narrow display area that is difficult to display complicated display or a lot of contents at once, specifically, it is difficult to display a score of multiple stages as shown in FIG.

  For example, the setting operator (switch, etc.) 6 is a mode selection switch for setting the apparatus to a musical sound reproduction mode for generating a musical sound based on desired music data, a music selection switch for selecting music data to be reproduced, Playback control switch for starting and stopping playback, rewinding and fast-forwarding of the music playback position, switches for determining various settings (parameters) related to performance environment / musical sound control such as tone and effect, display control information ( 4 (see FIG. 4), and various kinds of operators such as a switching button for switching output on / off. In addition to the above, the setting operator 6 includes various operators such as a numeric keypad for inputting numeric data and a keyboard for inputting character data for selecting, setting, and controlling pitches, tones, effects, and the like. Also good. When these setting operators 6 are operated, information and data corresponding to the operated switches are input to the CPU 1 (control unit) via the data and address bus D.

  The tone generator unit 7 can simultaneously generate musical tone signals in a plurality of channels, and is generated in response to the operation of the performance operator 4 given by the user via the data and address bus 1D and / or any desired one. A musical tone signal is generated based on various performance information generated in response to the reproduction of the music data. Further, in the present embodiment, not only the tone signal is generated, but also the display control information which is a digital signal using, for example, a direct spread spectrum technique or the like is used as a high frequency signal in an audible band (this is referred to as an acoustic signal in this specification) ) To generate a tone signal in which the tone signal is superimposed on the tone signal (which is referred to as a superimposed signal for convenience) or only the tone signal. (See FIG. 4B described later). These signals (musical sound signal, superimposed signal, acoustic signal) generated from the sound source unit 7 are sounded (sounded out) from a sound system 7A including an amplifier and a speaker. The sound system 7A corresponds to signal output means for outputting an audible band signal to the outside. Any conventional configuration may be used for the configuration of the sound source unit 7 and the sound system 7A. For example, the tone generator 7 may employ any of various tone synthesis methods such as FM, PCM, physical model, formant synthesis, etc., and may be configured with dedicated hardware, a DSP (Digital Signal Processor), You may comprise by the software process by CPU1.

  The storage device 8 stores various control programs executed by the CPU 1 in addition to various data prepared in advance such as music data. When the control program is not stored in the ROM 2 described above, the control program is stored in the storage device 8 (for example, a hard disk) and is read into the RAM 3 to store the control program in the ROM 2. It is possible to cause the CPU 1 to execute the same operation as in FIG. In this way, control programs can be easily added and upgraded. The storage device 8 is not limited to a hard disk (HD), but may be a flexible disk (FD), a compact disk (CD-ROM / CD-RAM), a magneto-optical disk (MO), or a DVD (Digital Versatile Disk). Any storage device may be used as long as it uses a storage medium in the form. Alternatively, a semiconductor memory such as a flash memory may be used.

  The communication interface (I / F) 9 is an interface for transmitting and receiving various information such as a control program and various data between the apparatus and an external device (not shown). The communication interface 9 may be, for example, a MIDI interface, a LAN, the Internet, a telephone line, or the like, and may include both wired and wireless ones.

In the above-described embodiment, the performance operator 4 is not limited to the form of a keyboard instrument, and may be any form such as a stringed instrument, a wind instrument, or a percussion instrument. In addition, the musical sound reproducing device is not limited to one in which the performance operator 4 and the sound source unit 7 are built in one device main body, but each device is configured separately, and each device is configured using a communication interface 9 such as a MIDI interface or various networks. Needless to say, it may be configured to connect the two.
Note that the musical sound reproducing apparatus according to the present invention is not limited to an electronic musical instrument, and emits musical sounds through a speaker or the like when music data is reproduced, such as a karaoke apparatus, a personal computer, a portable communication terminal such as a mobile phone, or a game apparatus. Any type of apparatus / apparatus may be used as long as it can be used. In the case of a mobile communication terminal, not only the case where a predetermined function is completed with only the terminal, but also a part of the function is provided on the server side so that the predetermined function is realized as a whole system including the terminal and the server. May be.

  Next, an information processing apparatus that is provided separately from the music sound reproducing apparatus and includes a display unit (external display) that displays display content will be described. FIG. 2 is a hardware configuration block diagram showing an embodiment of the overall configuration of the information processing apparatus according to the present invention. The information processing apparatus shown in FIG. 2 is controlled by a microcomputer including a CPU 11, a ROM 12, and a RAM 13, and a program (for example, a “score display process” program described later) that realizes a predetermined music function by the computer is used. It is an electronic device such as a tablet terminal or a smartphone, for example, which can execute processing related to score display and display update on the display unit 15 based on the execution. The CPU 11 controls the operation of the entire information processing apparatus (see the control unit in FIG. 5), and a timer 11A for measuring various times is connected to the CPU 1. The ROM 12 constantly stores control programs and data, and the RAM 13 temporarily stores them.

  The input operation unit 14 is a physical operation unit, for example, a power on / off switch, an adjustment switch for external output volume, microphone sensitivity, or the like. The display unit 15 is a touch panel display having a detection function for detecting (recognizing) a user touch operation (corresponding to a display unit). In response to detection of a user touch operation on the display unit 15, for example, selection / execution of a program to be operated on the device can be performed. In addition, the display unit 15 of the information processing apparatus according to the present embodiment can also function as an external display that displays display contents such as music scores and lyrics related to the musical sound being reproduced by the musical sound reproducing apparatus described above.

  Here, FIG. 3 shows an example of display content displayed when the display unit 15 of the information processing apparatus functions as an external display of the musical sound reproducing apparatus. FIG. 3 shows an example in which a score is displayed as display content on the display unit 15 of the information processing apparatus. However, in this example, the case where the score of the first page that has not been turned is displayed.

  As an example, for example, a title display Ha, a score display Hb, and other displays Hc are displayed on the display unit 15 in order from the top of the three display areas. In the title display Ha, a part of song information such as a song name and a composer name related to a musical sound being reproduced which is a musical score display target is displayed. The other display Hc includes a virtual operator I that functions in response to a user's touch operation such as a “menu” button for returning to a menu screen (not shown) for program selection and the like, and a speaker of the above-described music sound reproducing device. A reception indicator J or the like that is turned on when the superimposed signal or acoustic signal to be emitted is received via the voice input unit (microphone) 16 and is turned off when the signal is not received is displayed. The score display Hb displays a score composed of a plurality of bars in one row over a plurality of steps. Here, an example is shown in which a score of four bars is displayed in one row and displayed in three rows. Note that this one row consists of four bars, and a total of twelve bars of music score displayed as three rows is formed as one page of image data, and image data for a plurality of pages corresponding to the score of each song as a whole ( Are collectively stored in the storage device 17 for each song.

  Returning to the description of FIG. 2, the audio input unit 16 is a microphone, collects a musical sound signal (or a superimposed signal, an acoustic signal) emitted from the speaker of the above-described musical sound reproducing device, and collects the collected signal. It outputs to CPU11 (control part). The audio input unit 16 (microphone) corresponds to a signal acquisition unit that acquires an audible band signal. The storage device 17 stores various control programs executed by the CPU 11 and various data such as the above-described score data or a display control table described later (corresponding to content storage means). The communication interface (I / F) 18 is connected to a wired or wireless communication network such as a LAN, the Internet, or a telephone line, and is connected to a server computer or the like (not shown) via the communication network. This is an interface for fetching a control program or various data from the device.

  Also in the musical tone reproducing apparatus according to the present invention shown in FIG. 1, the musical tone reproducing apparatus according to the present invention is generated in accordance with the reproduction of the music data read from the ROM 2 or the storage device 8 according to the user selection, as in the conventional musical tone reproducing apparatus. Sound signal from the speaker. However, in the musical tone reproducing apparatus according to the embodiment of the present invention, the musical tone signal generated in response to the reproduction of the music data is not simply emitted from the speaker as it is, but the display control information is transmitted between the speaker and the microphone. A musical tone signal (superimposed signal) in which a modulated acoustic signal is superimposed on a signal that can be transferred using data or only the acoustic signal is emitted from a speaker, and displayed on the display unit 15 (external display) of the information processing apparatus. Display content display control (for example, turn control of a musical score) is realized. This will be described below.

  FIG. 4 is a functional block diagram for explaining the outline of the score control function in the musical tone reproduction apparatus. As shown in FIG. 4A, the control unit 1 can be roughly divided into a music data acquisition unit A, a music information acquisition unit B, and a display control information generation unit C. The music data acquisition unit A reads the music data to be played from the storage device 8 according to the music selection instruction output according to the operation of the setting operator 6 and outputs the music data read according to the playback instruction to the sound source unit 8. The song data includes a song ID unique to each song as one piece of information (song information) for specifying a song in addition to a song name, a singer's name, etc., and the song information acquisition unit B stores the read song data. The song ID extracted from the song information is sent to the display control information generation unit C. The display control information generation unit C is activated when the changeover switch of the setting operator 6 is turned on, and the song ID output from the song information acquisition unit B and the bar number output from the musical tone signal reproduction unit D in the sound source unit 7. Based on the above, the display control information is generated at predetermined time intervals (for example, every 0.5 seconds). This display control information is always generated regardless of the reproduction status of the music data if the changeover switch is on. When the changeover switch is turned off, the display control information generating unit C stops its operation and no display control information is generated. Further, the “turning function” is set to ON or OFF in accordance with ON / OFF of the changeover switch.

  The display control information is a digital signal, and header information (about 1 byte) including information indicating that the signal is a signal related to turning control of a score, and information for identifying the entire length of control data, a song ID, It consists of control data (about 2 bytes) consisting of measure information (position information) and footer information (about 1 byte) consisting of information indicating the end of the control data.

  The sound source section 7 can be divided into a musical sound signal reproduction section D (corresponding to reproduction means) and a modulation control section E (corresponding to control means). The music signal reproduction unit D, when instructed to start reproduction by the reproduction control switch of the setting operation element 6, reproduces the music data output from the music data acquisition unit A and generates a music signal. When a performance operation is performed with the performance operator 4, a musical tone signal is generated according to performance information corresponding to the performance operation. The generated musical sound signal is output to the modulation control unit E. Further, when the music signal reproducing unit D is reproducing music data, the musical sound signal reproducing unit D outputs position information indicating the performance position (reproduction position) on the progress of the currently reproduced musical tone to the display control information generating unit C. This position information is a measure number representing the measure where the musical sound currently being reproduced is located.

  The modulation control unit E modulates the display control information from the display control information generation unit C into an acoustic signal having a high band within the audible band, and when the musical tone signal is output from the musical tone signal reproduction unit D, the musical tone signal And a musical tone signal (superimposed signal) on which the acoustic signal is superimposed is output to the sound system 7A. The sound system 7A emits a superimposed signal through a speaker. On the other hand, when no music signal is output from the music signal reproduction unit D, only the sound signal is output to the sound system 7A (speaker) and emitted.

  In the technology of modulating display control information, which is a digital signal, into an acoustic signal and transmitting it, the display control information is modulated into an audible band high-frequency signal (acoustic signal), and can be reproduced by a general speaker (audible band). ), It is possible to transmit data using a high band within an audible band of about 18 kHz that is hardly audible to human (especially adult) ears. According to this, although the data transfer speed is not so fast as about 80 bps at the maximum, it is possible to transmit display control information to a place distant from 10 meters or more simultaneously to a plurality of receiving objects (one-to-many B) can be distributed, can use existing speaker equipment, and can control the transmission range by adjusting the volume on the speaker side. Note that the frequency band used for modulating the display control information into an acoustic signal is not limited to the above-described frequency band of about 18 kHz as long as it is a high band within an audible band that is not normally audible to humans.

  Note that, as described above, in the present embodiment, whether or not display control information and thus an acoustic signal is generated can be set according to ON / OFF of the changeover switch. That is, the acoustic signal can be set to be generated without being generated even though it is a high-frequency signal within the audible band that is not normally heard by humans. The first reason is that a high-band signal with a frequency band of about 18 kHz may be heard depending on the user's age (especially for younger children), and such a user needs automatic page turning control. This is because even if the sound signal is not generated, the sound signal is mixed with the musical sound signal and is very harsh when the sound signal is generated. Second, when multiple music playback devices are prepared, if sound signals are output from two or more music playback devices, interference will occur and it will not be possible to control the page turning well. Yes (in this case, you can turn on only one and turn off the rest). Third, when the page turning control is automatically performed even though the user does not want to perform the page turning control for some reason (for example, when the user wants to perform with a specific page kept open). This is because the user is troubled.

  Any method may be used to superimpose the sound signal on the musical tone signal in the modulation control unit E, but it is preferable to use a method in which the modulation component is difficult for a person to hear as described above. For example, a spreading code (PN code) such as an M sequence or a Gold sequence may be superimposed on a high band in the audible band at a weak level that does not cause a sense of incongruity. Here, an example of the configuration of the modulation control unit E is shown in FIG. The modulation control unit E includes an LPF 20, an adder 21, a spread code generation unit 22, a multiplier 23, an XOR circuit 24, a delay unit 25, an LPF 26, a multiplier 27, and a carrier signal generator 28.

  The spread code generating unit 22 periodically generates a spread code such as an M sequence in accordance with an instruction from the display control information generating unit C. The spread code generated by the spread code generator 22 and the display control information generated / output by the display control information generator C (a code string binarized to −1 and 1) are multiplied by the multiplier 23. Thereby, the spread code is phase-modulated. That is, when the bit data is 1, the phase is normal and when the bit data is 0, the phase is inverted.

  The spread code after phase modulation is input to the XOR circuit 24. The XOR circuit 24 outputs an exclusive OR of the code input from the multiplier 23 and the output code one sample before input via the delay unit 25. The signal after differential encoding is binarized by -1,1. By outputting the differential code binarized to −1 and 1, the demodulation side can extract the spreading code before differential encoding by multiplying the differential code of two consecutive samples.

  The differentially encoded spreading code is band-limited within the baseband in the LPF 26 and input to the multiplier 27. The multiplier 27 multiplies the carrier signal output from the carrier signal generator 28 (the carrier signal in the high frequency range in the audible band) and the output signal of the LPF 26, and frequency-shifts the spread code after differential encoding to the passband. . The spread code after differential encoding may be frequency shifted after up-sampling. The spread code after the frequency shift is added (synthesized) with the tone signal generated by the tone signal reproduction unit D by the adder 21. However, the tone signal is limited to a band different from the frequency component of the spread code by the LPF 20 before addition. In this way, the display control information is modulated into an acoustic signal having a high audible band and superimposed on the musical sound signal.

  FIG. 5 is a functional block diagram for explaining the outline of the score display function in the information processing apparatus. As shown in FIG. 5A, when a superimposed signal or an acoustic signal is collected by the voice input unit 16 (microphone), these collected signals are output to the demodulation unit F of the control unit 11. The The demodulator F demodulates the superimposed signal or the acoustic signal input from the audio input unit 16 and decodes the display control information, thereby extracting the song ID and the bar number included in the decoded display control information (extraction). Equivalent to means).

  FIG. 5B is a block diagram illustrating an example of the configuration of the demodulation unit F. The demodulation unit F includes an HPF 31, a delay unit 32, a multiplier 33, an LPF 34, a correlator 35, a peak detection unit 36, and a code determination unit 37. The superimposed signal or acoustic signal collected by the voice input unit 16 (microphone) is input to the HPF 31. The HPF 31 is a filter for extracting only the acoustic signal superimposed on the musical sound signal. The output signal of the HPF 31 is input to the delay unit 32 and the multiplier 33.

  The delay amount of the delay unit 33 is set to a time corresponding to one sample of the differential code. When the differential code is upsampled, it is set to the time for one sample after the upsampling. The multiplier 33 multiplies the signal input from the HPF 31 by the signal one sample before output from the delay unit 32, and performs delay detection processing. Since the differentially encoded signal is binarized to −1 and 1 and indicates a phase change from the code one sample before, the difference is obtained by multiplying the signal one sample before. A spreading code before dynamic coding is extracted.

  The output signal of the multiplier 33 is extracted as a baseband signal through the LPF 34 and input to the correlator 35. The correlator 35 includes an FIR filter (matched filter) in which the spreading code generated by the spreading code generator 22 is set as a filter coefficient, and obtains a correlation value between the input superimposed signal or acoustic signal and the spreading code. Since the spreading code uses a code having a high autocorrelation such as an M series or a Gold series, the correlation value output from the correlator 35 is positive or negative in the period of the spreading code (the period of the data code) by the peak detection unit 36. Are extracted. The sign determination unit 37 decodes each peak component as a data code of the display control information (a positive peak is 1 and a negative peak is 0).

  The display control unit G searches and specifies the corresponding musical score data and the display control table (see FIG. 6) from those stored in the storage device 17 in accordance with the song ID included in the decoded display control information (specifying means). Equivalent). When the musical score data corresponding to the song ID is found, the display control unit G reads the data from the storage device 17. Then, by referring to the specified display control table based on the bar number included in the decoded display control information, the display target page is determined for each stage as the display mode of the score displayed on the display unit 15, As a result, a score screen in which pages are switched for each level is generated and output to the display unit 15 (corresponding to display control means). At this time, it is preferable to perform display control so that the player can be shown as if the display of the stage where the page has been switched is turned over, that is, turned. Further, the display control unit G outputs a part of song information such as a song name included in the score data to the display unit 15 to display them on the screen.

  Here, a display control table and display control using the same will be described. FIG. 6 is a conceptual diagram showing the data structure of the display control table. However, here, an example has been shown in which display control is performed when a musical score including a repeat performance as shown in FIG. 3 is displayed in three stages.

  The display control table shown in FIG. 6 includes target pages to be displayed at each level of the musical score displayed on the display unit 15 for each measure of the musical tone being played (specifically, the measure number sent from the musical tone signal reproduction unit D). Is stipulated. For example, since bar numbers 1 to 4 are defined as “1” pages in all the first to third levels, one page is being played during the playback of musical sounds from the first bar to the fourth bar. Display control for displaying the first to third stages is performed. Measure number 5 is defined as “2” page in the first row, “1” page in the second and third rows, so when playing the musical sound of the fifth measure, the first row on the first page Display control is performed to switch only the first-stage musical score on which is displayed to the first-stage display of two pages. Therefore, the musical score displayed on the display unit 15 has a display mode in which the first stage of the second page and the second and third stages of the first page are mixed. Since bar number 6 to bar number 12 are the same as bar number 5, during the playback of musical sounds from bar 6 to bar 12, bar 2 and page 1 Display control for displaying the stage is performed.

  Also, bar number 13 is defined as “2” pages in the first and second tiers, and “1” page in the third tier. Therefore, when playing the musical sound of the thirteenth bar, 2 in one page. Display control is performed to switch the second-stage score displayed from the second stage to the second-stage display of two pages. Since bar number 16 defines “2” pages for all of the 1st to 3rd tiers, all of the 1st to 3rd tiers are the second The score display. That is, the score displayed on the display unit 15 is a display mode in which only two pages are displayed from a display mode in which two pages and one page are mixed. In this way, the turn-over control is repeatedly performed in order from the upper level of the score display according to the target page to be displayed for each level based on the measure number defined in the display control table. In other words, when the performance (playback) of all the bars included in a certain stage is completed and the performance (playback) proceeds to the next bar, the display control table displays the completed stage on the next page. It is set to update to the score. As a result, the reproduction position of the musical sound reproduced after the current musical sound reproduction position can be displayed on the score prior to the reproduction.

  Each control program for realizing the musical score control function in the musical tone reproduction apparatus and the musical score display function in the information processing apparatus as described above will be described with reference to FIGS. FIG. 7 is a flowchart showing an example of a score control process for realizing a score control function in the musical tone reproduction apparatus. This “musical score control process” is a process controlled by the CPU 1 of the musical tone reproducing apparatus, and is started when the musical tone reproducing apparatus is powered on.

  In step S1, it is determined whether or not the page turning function is set to ON according to the operation of the switching button by the user. If it is determined that the page turning function is not yet turned on (NO in step S1), the process waits until the page turning function is turned on. If it is determined that the page turning function is set to ON (YES in step S1), if the music data is selected in advance by the user, the music ID is acquired from the music information of the music data, or the music is set by the user. If no data is selected, the music data that has been initially set is automatically selected, the music ID is obtained from the music information of the music data, and the measure number corresponding to the current playback position of the music data is set. Obtain (step S2). If it is before the start of reproduction of the music data, the measure number (for example, “1” of the first measure) of the reproduction start position is acquired. However, when the user can designate the reproduction start position of the music data as an arbitrary position, the measure number corresponding to the designated reproduction start position is acquired.

  In step S3, the above-described display control information (see FIG. 4) including the acquired song ID and measure number is generated. In step S4, the generated display control information is supplied to the sound source unit 7. In step S5, it is determined whether a musical sound signal has been generated. When no musical sound reproduction start instruction is given and no music data is reproduced, and when the user does not operate the performance operator, it is determined that no musical sound signal is generated (NO in step S5). Control is performed to modulate the display control signal into an acoustic signal and output only the acoustic signal via the sound system 7A (step S7). Then, the process returns to step S1.

  On the other hand, when it is determined that a musical sound signal is generated, that is, when a musical sound signal is generated by the sound source unit 7 based on reproduction of music data, and / or by the sound source unit 7 according to the operation of the performance operator by the user. When the musical sound signal is generated (YES in step S5), the display control signal is modulated into an acoustic signal, and the superimposed signal obtained by superimposing the acoustic signal on the generated musical sound signal is output to the sound system 7A. (Step S6). Then, the process returns to step S1.

  Since the processing of step S1 to step S7 is repeatedly executed at predetermined time intervals (for example, every 0.5 seconds), the display control information is superimposed on the musical sound signal as an acoustic signal or independently for a predetermined time as an acoustic signal. Sound is emitted through the speaker at every interval. According to this, when the music data is not being reproduced, the same display control information whose bar number does not change is emitted through the speaker at predetermined time intervals and displayed on the display unit 15 of the information processing apparatus. The musical score is not turned over and the current display state is maintained. On the other hand, when the music data is being reproduced, the display control information with different measure numbers is emitted through the speaker as the musical sound reproduction position advances, and accordingly the display unit 15 of the information processing apparatus 15 The sheet music displayed in can be turned over.

  FIG. 8 is a flowchart illustrating an example of a score display process for realizing a score display function in the information processing apparatus. This “music score display process” is a process controlled by the CPU 11 of the information processing apparatus, and is started in response to reception of an acoustic signal or a superimposed signal. As an example, the CPU 11 repeatedly executes a process (not shown) for realizing the function of the demodulator F described above in a short cycle, and starts the “score display process” when the display control information can be extracted in the process. It is good to let them.

  In step S11, the song ID and the measure number are extracted from the display control information included in the superimposed signal or the sound signal emitted from the musical sound reproducing device through the speaker. In step S12, it is determined whether or not the extracted music ID matches the music ID stored in the RAM 13 (the music of the musical score currently displayed on the display unit 15). If it is determined that the extracted song ID matches the stored song ID (YES in step S12), it is determined whether or not the extracted measure number matches the stored measure number in the RAM 13 (step S13). ). If it is determined that the extracted bar number matches the stored bar number (YES in step S13), the process ends. That is, in this case, since it is not necessary to change the display position of the musical score being displayed on the display unit 15 in accordance with the progress of the musical sound reproduction position, the turning control of the musical score is not performed. Further, since the music being reproduced is not changed, it is not necessary to replace the musical score displayed on the display unit 15. When the reception of the acoustic signal or the superimposed signal is the first time, the bar number is not already stored in the RAM 13, so that the above step S13 is always determined as NO (does not match).

  When it is determined that the song ID extracted in step S12 does not match the stored song ID (NO in step S12), the score data matching the extracted song ID is read from the storage device 17 (step S14). . That is, when the music data to be reproduced is changed, the score displayed on the display unit 15 is replaced with the score of the changed music. In this case, a display control table (see FIG. 6) that matches the extracted music ID is specified. Further, the extracted music ID is written and stored in the RAM 13 (this is the stored bar number described above). That is, each time the measure number included in the display control information changes, the measure number stored in the RAM 13 is updated. If it is determined that the bar number extracted in step S14 or the bar number extracted in step S13 does not match the stored bar number (NO in step S13), the identified display control table is referred to, step S11. The display page of each level corresponding to the measure number extracted in step S15 is determined (step S15). In this case, the extracted measure number is written and stored in the RAM 13. In step S16, the page turning control is performed to update the display of the score for each stage in the display unit 15 based on the determination of the display page.

In addition, as an initial display of the display unit 15 when a superimposed signal or sound signal is not emitted from the musical sound reproducing device when the execution instruction of the program is instructed, for example, the score of the music displayed at the end of the previous program is the first Redisplaying from one measure, displaying a score of a predetermined demo song from the first measure, displaying a blank score (stave only), etc. may be used. Or you may display not only a score but a menu screen (not shown) etc., for example.
Although not shown in FIG. 7 described above, when a music selection operation is performed by the user in the musical sound reproducing device, the music ID before the music selection operation is determined according to the processing of steps S2 and S3 after the music selection operation. Display control information with different bar numbers) is newly generated. Then, the newly generated display control information is emitted through a speaker in accordance with the processing after step S4. On the other hand, with the execution of the processing shown in FIG. 8, the information processing apparatus displays the display instead of performing the page turning control for changing only a part of the plurality of scores displayed on the display unit 15. Display control is performed to change all of the multi-stage musical scores to a new musical score.

  As described above, in the musical sound reproducing apparatus according to the present embodiment, the superimposed signal (or only the acoustic signal) including the display control information is superimposed regardless of whether or not the musical sound reproduction progress state has reached the timing of turning the score. ) At a relatively short predetermined time (for example, every 0.5 seconds). That is, the musical sound reproducing device continues to output information related to the display control of the external display periodically. On the other hand, the information processing apparatus according to the present embodiment performs display control of the score based on the received display control information. As the display control of the score based on the display control information, a score composed of a plurality of bars in one stage is displayed in a plurality of stages on the display unit 15, and the display of the stage where all the bars have been reproduced is sequentially updated. Thus, the turn-over control is performed for each stage so that the display of the stage is completed before the bars included in the stage are reproduced. According to this, since the display control information can be acquired and an appropriate score can be displayed even before the reproduction of the musical sound is started, the user can manually add the desired score and the performance start position before the performance starts. There is no need to call out the score.

  Here, FIG. 9 shows a timing chart showing an example of each operation timing of the score control process and the score display process according to the musical tone reproduction mode. In order from the top, the playback position (measure number) of the music being played on the music playback device, the generation timing of display control information by the music playback device, the reception completion timing of display control information received by the information processing device, and the information processing device display The musical score position (measure number) to be displayed in the section 15 is shown.

  The operation of the musical sound reproducing apparatus will be described. Display control information (hereinafter simply referred to as information) is generated and transmitted at a certain timing (time t1). As soon as the transmission of the information t1 (here, a time is attached for convenience to distinguish the information generated at each timing), the information is generated and transmitted at the next timing t2. Here, if it takes a time of Δt (for example, 0.5 seconds) until one display control information is generated and transmitted, the interval at which the information t1 and the information t2 are transmitted is Δt time. is there. In the music reproducing apparatus, information is generated and transmitted one after another at this Δt time interval (see t3 to t11). Generation / transmission of such information is performed at Δt time intervals regardless of whether it is during normal playback or during rewinding or fast-forwarding operations.

  On the other hand, the operation of the information processing apparatus will be described. Reception of the information is completed after Δt time from the generation timing of certain information in the musical sound reproducing apparatus. For example, the reception of the information t1 transmitted from the musical sound reproducing device during the musical sound reproduction of the twelfth measure is completed at time (t1 + Δt). Since the bar number included in the information t1 is “12”, according to the display control table shown in FIG. 6, the score of the second page in the first row, the first page in the second row and the third row is displayed. Will be. Since the bar number included in the information t2 to the information t4 received after the information t1 is “12” and does not change with the bar number of the immediately preceding information, display control for updating the score is not performed during this period. . On the other hand, the bar number included in the information t5 is “13”, and according to the display control table, the second row is defined as “2 pages”, which is different from the bar number “12”. In this case, display control is performed in which the first and second tiers are updated to a score of 2 pages, and the third tier is a one-page score, that is, a score in which the second tier is turned.

  According to this, after Δt time when the display control information is transmitted from the musical sound reproducing device, the display control in the information processing device is performed, and the time for analyzing the performance position is not required unlike the prior art. Therefore, even if the musical sound playback position is moved to a non-continuous position by fast-forwarding or rewinding operation, it is possible to display the score in the corresponding range immediately after the fast-forwarding or rewinding operation compared to the conventional case. There is an advantage of becoming. For example, in FIG. 9, it is assumed that the rewinding operation from the 13th bar to the 4th bar is performed after the generation / transmission of the information t6 is started. At this time, the information t7 and the information t8 are generated / transmitted during the rewinding operation. In the information t7, the reproduction position of the musical sound corresponding to the rewinding operation is the eleventh measure, and in the information t8, the replaying operation is performed. Suppose that the playback position of the musical sound is the seventh measure. Then, since the measure number included in the information t7 received by the information processing apparatus is “11”, according to the display control table, the score of the first page is 2 pages, the second stage and the third stage are 1 page. It will be displayed again. The bar number included in the information t8 received following the information t7 is “7”. However, according to the display control table, the same rule as that of the bar number “11” is used. The display of the score of one page at the first level is maintained for the second page, the second level and the third level is maintained. Then, the bar number included in the information t9 generated / transmitted after the rewinding operation up to the fourth bar is completed and the normal playback is resumed is “4”. All of the third row displays a one-page score. Thus, the information t9 generated immediately after the end of the rewinding operation displays the score corresponding to the measure number 4 after the rewinding operation at the time (t9 + Δt) in the information processing apparatus. Since the time Δt is a short time of about 0.5 seconds, a musical score corresponding to the performance position is displayed immediately after the rewinding operation.

  In addition, by updating the display of the score for each stage in accordance with this, the information processing apparatus can minimize the adverse effects that occur when the reception status of the superimposed signal emitted from the musical sound reproduction apparatus is poor. It can be kept in. For example, if information t3 to information t5 cannot be received in FIG. 9, if the information is correctly received, the second stage is turned to page 2 after the completion of reception of information t5. Should be displayed, but the score corresponding to the bar number “12” included in the time t2 at which reception was successful can be displayed continuously on the first page, the second page, the second and third lines on the first page. become. However, as shown in FIG. 3, in the score display corresponding to the measure number “12”, the score of the thirteenth measure has already been displayed in the third row. Therefore, even if information t5 cannot be received and the score corresponding to the bar number “13” cannot be displayed immediately, the score corresponding to the bar number “13” can be displayed when the information t6 is received. There is no problem for the user to practice performance while viewing the score in accordance with the music playback.

  In addition, as shown in FIG. 3, since the reception indicator J is displayed on the display unit 15 of the information display device, whether the acoustic signal (or the superimposed signal) can be received depending on the lighting state of the reception indicator J. Alternatively, the user can grasp at an early stage, such as before the start of music sound reproduction, whether the situation has not been received. If the reception condition is poor, for example, if there is an obstacle that prevents the sound signal from being emitted from the music playback device or the sound collection by the information processing apparatus, remove the obstacle or place it in a place with good reception conditions. The user can take an action for improving the reception status, such as moving the information processing apparatus.

  In addition, there is an advantage that a musical score at an accurate position is always displayed and is not easily affected by the change of the music sound reproduction tempo. Here, FIG. 10 shows a timing chart showing an example of each operation timing of the score control process and the score display process according to the musical performance tempo.

  In the musical sound reproducing apparatus, information is generated / transmitted at every Δt interval without being influenced by the musical sound reproduction tempo. In order to make the explanation easy to understand, for example, the musical tone playback tempo from the twelfth bar to the thirteenth bar is temporarily accelerated, while the musical tone playback tempo from the fourteenth bar to the fifteenth bar is temporarily slowed. Has been. As can be understood from FIG. 10, when the musical sound reproduction tempo is accelerated, the number of times of transmitting information is reduced while the musical sound for one measure is reproduced, compared to before the tempo is accelerated. For example, information is sent four times from information t1 to information t4 during music playback from the twelfth bar to the thirteenth bar before the tempo is accelerated, but from the thirteenth bar after the tempo is accelerated. During music playback up to 14 bars, information t5 is sent only once. However, if the time interval (Δt) for sending information is a short time of less than 1 second, even if the tempo is speeded up, enough information is sent to update the musical score during music playback of one measure. Specifically, when playing a 4-beat musical tone at the tempo 120, if the time interval for sending information is 0.5 seconds, for example, information can be sent 3 to 4 times per measure. On the other hand, when the tempo is slowed down, the same information is simply sent repeatedly at predetermined time intervals (Δt), so there is no problem in the display control of the score. As described above, in the conventional technique for acquiring the tempo from the performance data and extracting the performance position, it is necessary to analyze the performance of at least the last few beats and extract the beats. Now, instead of acquiring tempo information and controlling the display of the score, as a display control information, periodically obtain a measure number indicating “how many measures are currently playing” and control the display of the score. Therefore, even if the musical tone playback tempo is changed, it is possible to control the display of the score with good follow-up.

As mentioned above, although an example of embodiment was demonstrated based on drawing, this invention is not limited to this, It cannot be overemphasized that various embodiment is possible. For example, the above-described score control processing program and score display processing program can be implemented not only in the form of computer software but also in the form of a microprogram processed by a DSP (digital signal processor). The present invention is not limited to the form, and may be implemented in the form of a dedicated hardware device configured to include a discrete circuit, an integrated circuit, a large-scale integrated circuit, or the like.
In the above-described embodiment, the score of the musical score displayed on the external display (the display unit 15 of the information processing apparatus) is obtained by emitting a superimposed signal or an acoustic signal through a speaker and receiving the signal with a microphone. Although an example of performing turning control is shown as an example, transmission and reception of a superimposed signal or an acoustic signal between the music sound reproducing device and the information processing device may be performed by wire.

In the above-described embodiment, a signal in which an acoustic signal is superimposed on a musical sound signal is emitted from the speaker. However, the present invention is not limited to this, and any sound (for example, singing voice input from a microphone, An operation sound when the setting operator is operated, a notification sound when an error occurs, etc.) may be emitted from the speaker together. That is, any other sound may be output from the speaker as long as the acoustic signal is output from the speaker.
Note that the display control of the display unit 15 is not limited to the turn-turning control for each stage, and may be display control for updating a part of the score in order, for example, for each bar, for each bar, for each bar, for each half page. Anything may be used.
Note that the reproduction position information may not be a measure number. It may be MIDI information representing measures / beats / clocks, or may represent the elapsed time from the beginning of the song in absolute time such as minutes / seconds / milliseconds. The music data may not be MIDI data but may be audio data. In this case, the reproduction position information should be the absolute time as described above.

In the embodiment described above, an example is shown in which digital display control information including a song ID and a bar number is generated, and this is modulated and converted into an acoustic signal. However, the present invention is not limited to this example. An acoustic signal may be prepared in advance, and the display control information may be converted into an acoustic signal by reading out the acoustic signal corresponding to the extracted song ID and measure number. In this case, the information processing apparatus stores in advance a conversion table in which the correspondence between the acoustic signal, the song ID, and the bar number is stored, and collates the conversion table on the basis of the received acoustic signal. It is recommended to extract measure numbers.
Note that the display content displayed on the external display (display unit 15 of the information processing apparatus) includes musical scores such as staff notation, tablature, chord notation, and the progress of musical sound reproduction such as lyrics and fingering instructions of performance operators. Any display may be used as long as it is necessary to switch the display according to the above. The display content may be in any data format such as image data or text data.

1,11.CPU, 2,12.ROM, 3,13.RAM, 4.performance operator 5,15.display unit 6.setting operation unit 7.sound source unit 8,17.storage device 9. 18, communication interface, 14 input operation unit, 16 voice input unit (microphone), D, 1D data and address bus, A music data acquisition unit, B music information acquisition unit, C display control information generation Unit, D / musical sound signal reproduction unit, E / modulation control unit, F / demodulation unit, G / display control unit

Claims (5)

Content storage means for storing a large number of display contents composed of a plurality of pages;
Display means for displaying display content;
Signal acquisition means for acquiring an audible band signal, wherein the audible band signal includes music information for specifying a song and position information indicating a performance position of the song specified by the song information. The signal acquisition is a signal that can be output by being superimposed on another sound from the external device and can be output separately from the other sound. Means ,
The audible band of the signal acquired by the signal acquiring means, when the acoustic signal including said music information and the position information is included, extracts of extracting the music information and the position information from the acoustic signals Means,
A specifying means for specifying a display content corresponding to the extracted music information from the stored many display contents;
Wherein said determining the display page of the specified display content according to the extracted position information, and display control means for controlling the display to update the display mode of the display contents to be displayed on said display means based on the display page Information processing apparatus provided.   The display control means displays display content by dividing the display means into a plurality of display areas, and the display control means performs turn-over control for changing pages in order for each partial area of the display means. The information processing apparatus according to claim 1.   The information processing apparatus according to claim 1, wherein an acquisition state of the acoustic signal by the signal acquisition unit is displayed on the display unit. Said extraction means, information processing apparatus according to any one of claims 1 to 3, characterized in that to execute a process for extracting said location information and said track information from the audio signal periodically. On the computer,
A procedure for obtaining an audible band signal, wherein the audible band signal includes music information for specifying a music piece and position information indicating a performance position of the music piece specified by the music piece information. The procedure, wherein the acoustic signal is a signal that can be output by being superimposed on another sound from the external device and can be output separately from the other sound, and
The signal of the audible band the acquired, if the acoustic signal including the said song information the position information is included, the procedure for extracting the music information and the position information from the acoustic signal,
A procedure for specifying display content consisting of a plurality of pages corresponding to the extracted song information from a number of display contents stored in advance;
Program for executing a procedure for performing display control in response to said extracted position information to determine the display page of the display contents and said specified, updates the display mode of the identified display content based on the display page.

Images