JP2006058577A - Data processor and program for processing two or more time-series data - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a means permitting to easily and accurately perform a job for judging whether or not consistency is maintained among different forms of data accompanied by time-series information. <P>SOLUTION: A decision part 124 retrieves a prescribed kind of music element data from logical music data acquired by a logical music data acquisition part 121, and retrieves performance element data including timing data showing the same timing or neighboring timing as the timing data included in the retrieved music element data from the performance control data acquired by a performance control data acquisition part 122. The decision part 124 decides whether or not the consistency is maintained between the contents of the retrieved music element data and performance element data. When deciding that inconsistency exists between those data, the decision part 124 creates a correction candidate data, and a permutation part 127 alters the logical music data and the performance control data according to the correction candidate data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a technique for processing a plurality of time series data, and more specifically, to a technique for detecting inconsistencies in a plurality of related time series data.

  For the purpose of supporting the acquisition of musical performance, the musical score is displayed on the display, and at the same time, the automatic performance device automatically performs the musical composition, or the metronome sounds for the musical performance master. There is technology.

For example, Patent Literature 1 and Patent Literature 2 disclose a technique for sequentially displaying a portion of a musical score of a musical piece currently being played on a display in accordance with the progress of the musical piece.
Japanese Patent Application Laid-Open No. 08-123416 JP 2001-047662 A

  By the way, when the musical score is displayed on a display or the like and at the same time, the automatic performance apparatus performs automatic performance of the musical composition, between the information indicated by the musical score and the performance control data that gives the automatic performance apparatus a performance instruction. There must be consistency. If the performance performed by the automatic performance device does not conform to the content of the score, the user who wants to perform the performance according to the score while using the performance performed by the automatic performance device as an exemplary performance may trust either the exemplary performance or the score. This is because they are confused and don't know how to play.

  However, the data for displaying the score and the performance control data for giving the player's instruction to the automatic performance device may be created by different processes. For example, this is a case where a music engineer creates data for displaying a score while viewing a score as a printed material, while another music engineer creates performance control data. As described above, there is a possibility that inconsistency may occur between data created by different processes due to a data input error or the like.

  Conventionally, the work of detecting and correcting such inconsistencies between data has been performed manually. In other words, the person in charge of the check visually confirms the score displayed on the display according to the display data of the score, and at the same time, confirms the performance sound automatically played by the automatic performance device according to the performance control data, and compares the contents. As a result, the data is checked for inconsistencies, and if there is an error, one of the data is corrected. Such conventional check work not only requires a lot of time and labor, but also has a problem that the quality of the check depends on the experience and ability of the checker.

  A similar problem occurs between, for example, data for displaying a musical score and moving image data in which an example performance by an experienced player is recorded. Furthermore, the present invention is not limited to music, but occurs, for example, between moving image data indicating the contents of a movie and character data indicating subtitles of the movie. That is, it can be said that this problem occurs widely between time series data of different formats that are required to be consistent with each other.

  In view of the above situation, the present invention makes it easy and accurate to determine whether or not consistency is maintained between time-series data of different formats that are required to be consistent with each other. It is an object of the present invention to provide means for enabling the above.

  In order to achieve the above object, the present invention provides first aggregate data including at least one element data that is a combination of timing data indicating timing and content data that is data related to timing indicated by the timing data. First data acquisition means, aggregate data including at least one element data that is a combination of timing data indicating timing and content data that is data related to the timing indicated by the timing data, the first data Second acquisition means for acquiring second aggregate data including content data in a format different from the aggregate data, and element data including timing data indicating a specific timing among the element data included in the first aggregate data Included in the first content indicated by the content data included in the second set data Determination means for determining whether or not the second content indicated by the content data included in the element data including the timing data indicating the specific timing among the element data satisfies a predetermined relationship. A data processing apparatus is provided.

  According to the data processing device having such a configuration, it is automatically determined whether or not the contents of the data corresponding to the same timing among the data included in the different types of data accompanied with time-series information are consistent. be able to. As a result, the user can obtain a result with a certain degree of accuracy without taking time and effort compared to the case where the same operation is performed manually.

  In a preferred aspect, the data processing device indicates the first content when the determination unit determines that the first content and the second content do not satisfy a predetermined relationship. Informing means for informing information specifying at least one of element data including content data and element data including content data indicating the second content is provided.

  According to the data processing apparatus having such a configuration, when there is a mismatch between data of different formats accompanied by time-series information, the user can easily correct the data portion causing the mismatch. .

  In another preferable aspect, when the data processing apparatus determines that the first content and the second content do not satisfy a predetermined relationship by the determination unit, the first processing unit And changing means for changing at least one of the content data indicating the content of the content and the content data indicating the second content according to a predetermined condition.

  According to the data processing apparatus having such a configuration, when there is a mismatch between data in different formats accompanied by time series information, the user can retrieve the corrected data without performing manual correction or the like. Obtainable.

  In another preferred embodiment, each of the first set data and the second set data processed by the data processing device is a performance for instructing an automatic performance device to control pronunciation of a tone included in a musical piece. Control data, logical score data that is a combination of data indicating the components of the musical score and timing data indicating the position of the component on the time axis, a plurality of character data arranged in time series, It is data including at least one of sound data indicating the sound or sound of music, a plurality of still image data and moving image data arranged in time series.

  According to the data editing apparatus having such a configuration, it is possible to check consistency at various timings between various audiovisual data such as a score and automatic performance of the same music, a movie image and subtitles.

  Furthermore, the present invention relates to element data included in first set data including at least one element data that is a combination of timing data indicating timing and content data that is data related to the timing indicated by the timing data. A combination of the first content indicated by the content data included in the element data including the timing data indicating the specific timing, and the content data which is the timing data indicating the timing and the data related to the timing indicated by the timing data Timing data indicating a specific timing among element data included in second set data including content data in a format different from that of the first set data. Specified by the content data included in the included element data And content of providing a program for causing execute processing for determining the computer whether it satisfies a predetermined relationship.

  According to such a program, the above-described data processing apparatus according to the present invention can be realized by a computer.

  According to the data processing device and the program according to the present invention, it is possible to obtain an accurate determination result regarding the presence or absence of consistency between different types of data accompanied by time-series information without requiring manual labor.

[1. First Embodiment]
The data processing apparatus (hereinafter referred to as “error check tool 10”) according to the first embodiment of the present invention is data used by an apparatus (hereinafter referred to as “trainer tool 20”) that supports learning of music performance for the user. This is a device that checks the consistency of data and corrects necessary data when there is a mismatch between the data. Therefore, hereinafter, after describing the functional outline of the trainer tool 20 and the structure of data used by the trainer tool 20, the configuration and operation of the error check tool 10 according to the present embodiment will be described.

[1.1. Configuration of trainer tool]
FIG. 1 is a diagram showing the configuration of the trainer tool 20 and a keyboard 30 connected to the trainer tool 20. The trainer tool 20 includes a control unit 201 that controls each component of the trainer tool 20, a display unit 202 that displays a musical score and the like for the user, a sound system 203 that includes a sound source unit and a speaker, etc. Used to synchronize the processing between the storage unit 204 that stores a program instructing the processing to be performed by the unit 201, the input / output I / F (Interface) 205 that transmits and receives data to and from the keyboard 30, and the components of the trainer tool 20. An oscillator 206 for transmitting a clock signal to be transmitted is provided. The keyboard 30 is a music keyboard provided with a keyboard or the like for the user to play music, and transmits data corresponding to the user's keystrokes to the trainer tool 20.

  In addition to a program for instructing the control unit 201 to perform various processes, the storage unit 204 performs logic score data 2041 indicating the content of the score displayed by the display unit 202 and the sound system 203 under the control of the control unit 201. A plurality of pieces of performance control data 2042 indicating the contents of the automatic performance are stored. For example, the logical score data 2041 related to a certain piece of music and the performance control data 2042 related to the same piece of music have the same file name excluding the extension, and are associated with each other. Hereinafter, the individual logical score data 2041 is expressed as logical score data 2041a, logical score data 2041b,..., And the individual performance control data 2042 is expressed as performance control data 2042a, performance control data 2042b,. Further, it is assumed that the logical score data 2041 and the performance control data 2042 having the same code end are data relating to the same music piece. Furthermore, it is assumed that both the logical score data 2041a and the performance control data 2042a are data related to the musical piece “Saint's March”.

[1.2. Logic score data]
Next, the structure and role of the logic score data 2041 will be described. FIG. 2 is a diagram showing a score displayed on the display unit 202 of the trainer tool 20 in accordance with the logical score data 2041a. FIGS. 3 and 4 are diagrams illustrating a part of the logical score data 2041a. FIGS. 3 and 4 correspond to the first and fourth levels of the score shown in FIG. 2, respectively. . Actually, the logical score data 2041a is a series of data including the data shown in FIGS.

  The logical score data 2041a is a set of element data (hereinafter referred to as “score element data”) shown as data in each row in FIGS. 3 and 4, and each score element data includes notes, dynamic symbols, and the like included in the score. Information for displaying each component (hereinafter referred to as "musical score element"). The score element data includes “ID (Identifier)” for identifying the score element data, “MBT (Measure / Beat / Tick) 1” indicating the time axis direction of the display position of the score element, that is, the position in the X axis direction, and “MBT2”, “Y1” and “Y2” indicating the display position of the score element in the Y-axis direction, “Type” indicating the type of the score element, and “Value” indicating various information according to the type of the score element Contains. In addition, “MBT1” and “Y1” and “MBT2” and “Y2” form a pair, and indicate one point on the score. Hereinafter, when it is necessary to distinguish each piece of musical score element data, an ID is attached and the musical piece element data is represented as “1”.

  “MBT1” and “MBT2” are composed of a set of three numerical values, each indicating a bar number, a beat number, and a tick number. The tick number is a tick number that is a time unit obtained by equally dividing one beat into a predetermined number. In the following description, for example, 1 tick = 1/480 beats. For example, “1: 1: 000” indicates the start timing of the first beat of the first measure, and “1: 1: 001” indicates only one tick (1/480 beat) from the start timing of the first beat of the first measure. Indicates the elapsed timing. In the logical score data 2041a, “MBT1” and “MBT2” indicate the timing from the beginning of each stage of the score. However, there are musical score elements that do not belong to any measure, such as clef. Accordingly, “MBT1” and “MBT2” indicate “0: 0: 000” to “0: 1: 479”, “#: 0: 000” (“0: 0000”) indicating the timing that does not actually exist according to “Type”. # Is a positive integer) and “#: X: 480” (# and X are positive integers).

  “0: 0: 000” to “0: 1: 479” indicate positions before the first measure, and are used for musical score element data for displaying clef symbols, tone symbols, time signature symbols, and the like. “#: 0: 000” indicates the position of a bar line indicating the start of the #th bar. “#: X: 480” indicates the position of a bar line indicating the end of the #th bar. In that case, X is the number of the last beat of the measure. For example, in the music of 4/4 time, X is “4”. Therefore, for example, “2: 4: 480” and “3: 0: 000” both indicate the positions of the bar lines that divide the second bar and the third bar. In the present embodiment, a normal bar line is treated as indicating the start of the subsequent bar, and for example, “3: 0: 000” is used as the bar line position between the second bar and the third bar. . Then, when there are no subsequent bars on the score, such as a bar line indicating the end of the last bar of each stage or a repeat symbol indicating the end point of the repetition period, the positions of those bar lines and the like are set to “4: 4: 480 ". By the way, the musical score element data in which “Type” is “bar line” and “MBT1” and “MBT2” are “#: X: 480” indicates the bar line at the end of each stage, that is, the break of each stage. . Thus, hereinafter, such musical score element data is referred to as “column break data”.

  In the musical score element data group corresponding to each stage of the musical score, the musical score element data are arranged in the order of the timing indicated by “MBT1”. When the timings indicated by “MBT1” are the same, the values indicated by “Y1” are arranged in ascending order. “Y1” and “Y2” are integer values. In the score, the position of the first line in the Y-axis direction is “100”, and the distance between adjacent lines is “10”. Show. For example, “Y1” and “Y2” of the musical score element data indicating the second line are “110”.

  All the musical score element data included in the logical musical score data 2041a have values of “MBT1” and “Y1”, and the combination of “MBT1” and “Y1” depends on “Type” of the musical score element data. Indicates the reference point or starting point of the score element. The reference point is a feature point of each musical score element whose position can be specified by one point, such as a point indicating the treble position of the treble clef or the center point of the note head. When the combination of “MBT1” and “Y1” indicates the reference point of the score element, the score element data does not have the values “MBT2” and “Y2”. On the other hand, for a musical score element that requires a start point and an end point to specify the display position, the combination of “MBT1” and “Y1” indicates the start point of the display position of the musical score element, and the combination of “MBT2” and “Y2” is the score. Indicates the end point of the display position of the element.

  The value of “Type” includes “score line” indicating the first to fifth lines, “clef” indicating the clef, “key” indicating the tone symbol, “time” indicating the time signature, and the speed of the music “Tempo” indicating the display, “Note head” indicating the note head, “Stem” indicating the note tail, “Flag” indicating the note note, “Bar line” indicating the bar line, Harmonic symbol "Cord" indicating lyrics, "Lyric" indicating lyrics, "Memo" indicating remarks, "Curve" indicating tie and slur symbols, "Forte" indicating strength symbols, "Crescendo" indicating volume change symbols, etc. Those indicating various symbols such as “Ritardando” indicating a speed change symbol and “Repeat Mark” indicating a repetition symbol are used.

  Among the values of “Type” exemplified above, for example, “score line”, “stem”, “bar line”, “curve”, and “crescendo” indicate musical score elements that require a start point and an end point, and “MBT2”. And “Y2”. On the other hand, the other musical score element data does not have the values “MBT2” and “Y2”.

  “Value” takes various values according to “Type”, and does not take a value when unnecessary. For example, when “Type” is “Clef”, the value of “Value” is any one of “G clef”, “H clef”, or “C clef”. Further, for example, when “Type” is “score line”, “Value” is unnecessary, and therefore no value is entered.

  Hereinafter, a state in which the display of the score is instructed by the logical score data 2041a will be briefly described with reference to FIGS. As already described, the data shown in FIG. 3 is data instructing the display of the first stage of the score shown in FIG. The musical score element data “1” has an MBT2 “4: 4” from the left end of the stage indicated by MBT1 “0: 0: 000” at the height in the Y-axis direction “100” indicated by “Y1” and “Y2”. : 480 "is instructed to draw a line toward the right end of the stage. That is, the musical score element data “1” indicates the display of the first line of the first row. Similarly, the musical score element data “2” to “5” indicates the display of the second line to the fifth line in the first stage.

  The musical score element data “6” to “9” are data instructing display of a treble clef, a tempo symbol, a tone symbol, and a time signature symbol, respectively. The musical score element data “10” and “11” are data for instructing display of rest symbols located at the first beat and the second beat in the first measure. The musical score element data “12” is data instructing display of the lyrics “Oh,”.

  The musical score element data “13” to “15” respectively indicate the note head, note tail, and note pad of the eighth note, and these three indicate the display of one note. “MBT1” and “Y1” of the musical score element data “13” indicate the center point of the note head, and “Value” indicates that the note head is black. “MBT1” and “Y1” of the musical score element data “14” indicate the start point of the note, and “MBT2” and “Y2” indicate the end point of the note. Since the value of “Y2” is larger than the value of “Y1” of the score element data “14”, the trainer tool 20 displays “MBT1” and “MBT2” when displaying the note according to the score element data “14”. A note tail is displayed at a position shifted to the right by half the width of the note head from the indicated position in the X-axis direction. In the score element data “15”, “MBT1” and “Y1” are the start points of the note, “MBT2” and “Y2” are the end points of the note, and “Value” is a single that indicates the eighth note. Indicates that it is a sign of Since the value of “Y2” is smaller than the value of “Y1” of the score element data “15”, the trainer tool 20 displays “MBT1” and “MBT2” when displaying the note according to the score element data “15”. A sign having a convex on the right side is displayed at a position shifted to the right side by half the width of the note head from the position in the X-axis direction shown.

  Since the mechanism for instructing the display of the musical score elements from the musical score element data “16” and thereafter is the same as the musical score element data “1” to “15”, description thereof will be omitted. Here, the score element data “46” is column-delimited data because the values of “MBT1” and “MBT2” are “4: 4: 480” and “Value” is “barline”. Therefore, the trainer tool 20 displays the bar line according to the musical score element data “46”, and displays the musical score elements indicated by the subsequent musical score element data in different stages. In this embodiment, “MBT1” and “MBT2” of the musical score element data are configured to indicate the timing based on the beginning of each stage, but indicate the timing based on the beginning of the music. You may comprise. In this case, for example, the first measure in the fourth row is the thirteenth measure in the music, so “MBT1” of the musical score element data “137” is “13: 1: 000”.

[1.3. Performance control data]
Next, the configuration of the performance control data 2042 will be described. 5 and 6 are diagrams illustrating a part of the contents of the performance control data 2042a, which correspond to the first and fourth levels of the score shown in FIG. 2, respectively. Actually, the performance control data 2042a is a series of data including the data shown in FIGS. The performance control data 2042a is a set of element data (hereinafter referred to as “performance element data”) shown as data in each row in FIGS. 5 and 6, and each performance element data instructs the pronunciation of a musical sound or lyrics. It is data indicating information related to music such as. Each performance element data includes “ID” for identifying performance element data, “Track” indicating the performance part of the music, “MBT” indicating specific timing in the music, “Type” indicating the type of performance element data, “Value” indicating various information corresponding to the type of performance element data is included. Hereinafter, when it is necessary to distinguish each piece of performance element data, an ID is attached and the performance element data is represented as “1”.

  In the present embodiment, since the musical piece “Progress of Saints” includes only one performance part, “Track” is “1” for all performance element data. The performance element data are arranged in the order of the timing indicated by “MBT”.

  The value of “Type” is “key” indicating the tone of the music, “time” indicating the time of the music, “tempo” indicating the speed of the music, “note-on” indicating the start of musical sound, "Note off" to indicate the end of pronunciation, "chord" to indicate the harmony attached to the song, "lyric" to indicate the lyrics attached to the song, and "memo" to indicate information such as remarks attached to the song One of them. However, as the value of “Type”, a value other than those described above, such as those related to timbre and acoustic effect, may be used. In the following description, “Value” of “Memo” is not normally displayed on the score unlike “Value” such as “Lyric”.

  “Value” of the performance element data whose “Type” is “key”, “time”, “chord”, “lyric”, and “memo” indicate information such as tone, time signature, harmony, lyrics, and remarks, respectively. . “Value” of the performance element data whose “Type” is “tempo” indicates the speed of the music, for example, “120” indicates that the quarter note has a speed of (1/120). “Value” of the performance element data whose “Type” is “Note On” is a combination of a character string indicating the pitch of a musical sound and a numerical value indicating the volume. For example, the value “F3 (72)” of the performance element data “4” indicates that the central sound is pronounced at a volume of 72 (medium). The numerical value indicating the volume is an integer value from 0 to 127. “Value” of the performance element data whose “Type” is “Note Off” indicates the pitch of a musical tone whose sound generation should be terminated.

  The role of the performance control data 2042a is the same as the role of general data for instructing automatic performance to an automatic performance device such as SMF (Standard MIDI File), and the description thereof will be omitted.

[1.4. Operation of trainer tool]
Next, the operation when the trainer tool 20 assists the user in acquiring music performance using the performance control data 2042a and the logical score data 2041a described above will be described. FIG. 7 is a diagram exemplifying a screen (hereinafter, referred to as “training screen”) displayed on the display unit 202 of the trainer tool 20 when the user learns music performance using the trainer tool 20.

  On the training screen, a box 2021 for the user to specify a musical piece for which performance learning is to be performed, a box 2022 for the user to specify a starting measure number of a musical piece portion that the user wants to acquire the musical performance, and a user wants to acquire the performance. A box 2023 for designating the end measure number of the musical piece portion to be considered, a box 2024 for the user to designate the performance speed of the music, and a user for selecting whether to listen to the model performance or practice the performance by himself. A check box 2025, a button 2026 for instructing the user to start the model performance or the practice by the user, and a button 2027 for the user to instruct the model performance or the user to stop the practice are displayed.

  Also, on the training screen, 8 bars starting from the bar number input in the box 2022 among the music corresponding to the music name input in the box 2021 by the user are displayed in two stages as a music score 2028a and a music score 2028b. ing. In addition, a mark 2029 indicating the current performance position is displayed above the score 2028a.

  First, the user selects the name of a song for which he / she wants to learn a performance in box 2021. Here, it is assumed that the user selects “Saint March”. The trainer tool 20 reads the performance control data 2042a from the storage unit 204 according to the user's selection, and the music “Saint March” is the first from the “MBT” of the last performance element data included in the performance control data 2042a. It is specified that there are 21 measures, and “001” indicating the first measure number of the music is displayed in the box 2022, and “021” indicating the last measure number of the music is displayed in the box 2023. Further, the trainer tool 20 searches the performance control data 2042a for the first performance element data whose “Type” is “tempo”, and the initial speed of the music “Saint March” from the “Value” of the retrieved performance element data. The value is specified as “120”, and “120” is displayed in the box 2024.

  Subsequently, the trainer tool 20 reads the logical score data 2041a from the storage unit 204, and follows the score 2028a of the score 2028a in accordance with the data from the beginning of the score element data included in the logical score data 2041a to the first column break data (see FIG. 3). The musical score 2028b is displayed in accordance with the display and the musical score element data included between the first column break data and the next column break data. Furthermore, the trainer tool 20 displays the mark 2029 at the beginning of the first measure of the score 2028a.

  When the user wants to listen to the model performance, the user checks the “model performance” in the check box 2025 and clicks the button 2026. In response to the operation, the trainer tool 20 starts the automatic performance of the musical piece “Saint's March” by generating a musical sound with the sound system 203 in accordance with the performance control data 2042a. As the automatic performance progresses, the trainer tool 20 changes the display position of the mark 2029 at a sufficiently short time interval so that the mark 2029 indicates the current performance position in the score 2028a.

  When the automatic performance reaches the end of the music portion shown in the score 2028a, the trainer tool 20 updates the display of the score 2028a with the display content displayed in the score 2028b, and moves the mark 2029 to the left end of the score 2028a. The score of the next stage is displayed at the position of the score 2028b. However, when the level displayed in the updated score 2028a is the final level, the trainer tool 20 displays nothing at the position of the score 2028b. Thereafter, the trainer tool 20 repeats the updating of the display of the training screen described above until the automatic performance of the music ends. In this way, the display of the score and the automatic performance of the model performance are performed in conjunction with the trainer tool 20.

  On the other hand, if the user wants to practice the performance by himself / herself, the “practice” check box 2025 is checked and the button 2026 is clicked. In response to the operation, the trainer tool 20 starts the metronome sound generation by the sound system 203 according to the performance element data whose “Type” is “time” and “tempo” included in the performance control data 2042a. As the metronome sound is generated, the trainer tool 20 updates the display of the musical score 2028a and the like, as in the case where the user selects “exemplary performance”. The user performs a musical piece using the keyboard 30 in accordance with the score displayed on the display unit 202. The keyboard 30 transmits data corresponding to the keystroke by the user to the trainer tool 20, and the trainer tool 20 generates a musical tone by the sound system 203 according to the data received from the keyboard 30. As a result, the user can practice performance according to the score.

  The performance control data 2042a includes only data related to one performance part. However, when using the performance control data 2042 including data related to a plurality of performance parts, for example, the user selects a performance part to perform by himself / herself. The trainer tool 20 may automatically perform only the performance parts not selected by the user.

[1.5. Error check tool configuration]
Next, the configuration of the error check tool 10 according to the present invention will be described. FIG. 8 is a diagram illustrating a hardware configuration of the error check tool 10. The error check tool 10 is a CPU (Central Processing Unit) 101 that performs various data processing and controls other components as in a general computer, and a BIOS (Basic Input Output System) that performs basic processing on the CPU 101. ) Etc., a ROM (Read Only Memory) 102 for storing data, a RAM (Random Access Memory) 103 for temporarily storing data in the data processing of the CPU 101, and an application program for realizing the functions of the error check tool 10 (hereinafter, “ Called an “error check program”) and an HD (Hard Disk) 104 for storing various data, a display 105 for displaying images in accordance with instructions from the CPU 101, a user A keyboard 106 and a mouse 107 that transmit signals according to the operation of the above, and an input / output I / F 108 that transmits and receives data to and from various external devices.

  For example, an external storage device 401 is connected to the input / output I / F 108, and the error check tool 10 can transmit / receive logical score data 2041 and performance control data 2042 to / from the external storage device 401. The input / output I / F 108 is also connected to the Internet 402, and can transmit / receive logic score data 2041 and performance control data 2042 to / from other communication devices connected to the Internet 402. In the following description, the logical score data 2041a (see FIGS. 3 and 4) and performance control data 2042a (see FIG. 3) acquired by the error check tool 10 via the input / output I / F 108 and the like are stored in the HD 104 of the error check tool 10. 5 and FIG. 6) are already stored.

  The CPU 101 implements various functions of the error check tool 10 by executing an error check program. FIG. 9 is a block diagram showing a functional configuration of the error check tool 10. However, since a communication unit and the like for the error check tool 10 to perform data communication with the external storage device 401 are the same as those in a general communication device, they are omitted in FIG.

  The error check tool 10 includes a logical score data acquisition unit 121 that acquires logic score data 2041 from the HD 104 and the like, a performance control data acquisition unit 122 that acquires performance control data 2042 from the HD 104, a logic score data acquisition unit 121, and performance control data. The storage unit 123 that temporarily stores the logical score data 2041 and the performance control data 2042 acquired by the acquisition unit 122, the determination unit 124 that determines the consistency between the logical score data 2041 and the performance control data 2042, and the logical score data 2041 And a display unit 125 that notifies the user of the contents of the performance control data 2042 and the determination result by the determination unit 124 by a screen display, a change instruction acquisition unit 126 that acquires a change instruction for the logical score data 2041 or the performance control data 2042 by the user, Judgment part 24 and the logical score data 2041 changed by the change unit 127 for changing the logical score data 2041 or the performance control data 2042 according to the change instruction of the user acquired by the change instruction acquisition unit 126. Is output to the HD 104 or the like, and a performance control data output unit 129 is provided to output the performance control data 2042 changed by the changing unit 127 to the HD 104 or the like. Note that the error check tool 10 may further include a notification unit that notifies the determination result by the determination unit 124 by sound or the like in addition to the notification by the screen display by the display unit 125.

  The storage unit 123, the display unit 125, and the change instruction acquisition unit 126 in FIG. 9 are realized by the RAM 103, the display 105, the keyboard 106, and the mouse 107, respectively, under the control of the CPU 101 in FIG. The other components in FIG. 9 are mainly realized by the CPU 101. In the following description, the error check tool 10 is realized by the apparatus shown in FIG. 8 executing the error check program. However, the functions of the components shown in FIG. 9 can be executed. It may be realized by a combination of various processors.

[1.6. Operation of error check tool]
Hereinafter, the operation of the error check tool 10 will be described. FIG. 10 is a diagram illustrating a screen (hereinafter referred to as “check screen”) displayed on the display 105 of the error check tool 10. The check screen shows the ranges of musical scores displayed in boxes 1051 and 1053 for designating file names (excluding extensions) of the logical musical score data 2041 and performance control data 2042 for which the user performs consistency check. Box 1052, score box 1053, list 1054 for displaying performance control data, check box 1055 for selecting an item that the user wants to perform consistency check, and correction of inconsistency between data by the user. A check box 1056 for selecting a method to be performed, a button 1057 for instructing the user to start an inconsistency check between data, and a button 1058 for instructing the user to save data changed by the inconsistency check. include.

  The user first selects the file names (excluding the extension) of the logical score data 2041 and performance control data 2042 to be checked for consistency in box 1051. Here, it is assumed that the user selects “Saint March”. The logical score data acquisition unit 121 and the performance control data acquisition unit 122 read the logical score data 2041a and the performance control data 2042a from the HD 104 according to the user's selection, and temporarily store them in the storage unit 123. The display unit 125 displays the box 1053 and the list 1054 according to the data newly stored in the storage unit 123. In this case, the display unit 125 displays “001 to 004” indicating the first four measures of the music as an initial value in the box 1052, and displays the first level of the score generated according to the logical score data 2041a in the box 1053. The list 1054 displays the first part of the element data included in the performance control data 2042a. Further, the display unit 125 displays the check box 1055 in a state where “all” as an initial value is selected, and displays the check box 1056 as “display correction candidates” as an initial value. Display with status. FIG. 10 shows a check screen when these processes by the display unit 125 are completed.

  When the user wants to perform data consistency check using the initial values of options shown in the check box 1055 and the check box 1056, the user clicks a button 1057. In accordance with the operation, the determination unit 124 performs the consistency check in the order of repetition, pitch and length, volume, speed, lyrics, and chord, for example. FIG. 11 is a diagram illustrating a processing flow of these determination processes performed by the determination unit 124.

  The determination unit 124 searches for musical score element data in which “Type” indicates a repetition symbol in order from the top of the logical musical score data 2041a (see FIGS. 3 and 4) stored in the storage unit 123. “Type” indicating a repetition symbol includes “repeat mark”, “ending”, “coder”, “Seño”, and the like. In this case, the determination unit 124 first obtains musical score element data “136” whose “Type” is “repeat mark” as a search result. Since the “Value” of the score element data “136” is “Start”, the determination unit 124 continues to the score element whose “Type” is “Repeat Mark” and whose “Value” is “End”. Search for data. As a result, the determination unit 124 obtains the musical score element data “174” as a search result.

  Subsequently, the determination unit 124 searches score element data whose “Type” is “ending” from the top. This is because the musical score element data in which “Type” is “ending” is musical score element data indicating the first parenthesis, the second parenthesis, and the like at the end of the music used together with the repeat mark in the musical score. In this case, the determination unit 124 obtains score element data “170” as a search result. Since the “Value” of the score element data “170” is “First”, the determination unit 124 continues to the score element data in which “Type” is “Ending” and “Value” is “Second”. Search for. As a result, the determination unit 124 obtains score element data “175” as a search result. If the determination unit 124 succeeds in searching the musical score element data “175”, the determination unit 124 ends the series of search processing (step S101).

  If the determination unit 124 succeeds in the above-described series of search processing (step S102; Yes), the values of “MBT1” and “MBT2” included in the searched score element data are values indicating the timing from the beginning of the music piece. (Hereinafter referred to as “MBT conversion”) (step S103). This is because, in the present embodiment, “MBT1” and “MBT2” included in the logical score data 2041 indicate the timing from the beginning of each stage in the score. The MBT conversion process will be described below.

  The determination unit 124 sequentially determines from the top of the logical score data 2041a whether (1) the score element data indicates a bar break or (2) the score element data is the score element data searched in step S101. (3) It is determined whether the musical score element data is related to a repetition symbol. Hereinafter, the determinations (1) to (3) are referred to as “determination for MBT conversion”.

  In the determination for MBT conversion, when the musical score element data is data indicating a bar break, the determination unit 124 adds 1 to the counter. The value of this counter indicates the total number of preceding measures. Note that the musical score element data indicating the break between bars is not only “Type” is “barline”, but also “Type” is “repeat mark” and “Value” is “end”, for example. is there.

  In the determination for MBT conversion, if the musical score element data to be determined is the one retrieved in step S101, the determination unit 124 sets the bar number included in “MBT1” and “MBT2” of the musical score element data at that time. Add the counter value. As a result, “MBT1” and “MBT2” are numerical values indicating the timing from the beginning of the music.

  Further, in the determination for MBT conversion, when the score element data is related to the repetition symbol, the determination unit 124 again performs the score element data that has already been determined for the MBT conversion according to the repetition symbol. The determination for MBT conversion and MBT conversion are performed. For example, the determination unit 124 performs determination for MBT conversion up to the musical score element data “174” indicating the end point of the repetition, and then returns to the musical score element data “136” indicating the starting point of the repetition. The determination for MBT conversion is repeated for the score element data after “136”. The above is the MBT conversion process.

As a result of the MBT conversion, “MBT1” and “MBT2” of the musical score element data searched in step S101 are converted as follows. Hereinafter, the score element data after the MBT conversion is referred to as the converted score element data. Note that the musical score element data “136” is subjected to MBT conversion twice according to the repetition symbol, resulting in two converted musical score element data. Hereinafter, they are distinguished as converted musical score element data “136-1” and converted musical score element data “136-2”.
Music score element data after conversion “136-1”... MBT1 “13: 0: 000”, MBT2 “13: 0: 000”.
The musical score element data after conversion “170”... MBT1 “16: 0: 000”, MBT2 “16: 4: 480”.
The musical score element data after conversion “174”... MBT1 “16: 4: 480”, MBT2 “16: 4: 480”.
The musical score element data after conversion “136-2”... MBT1 “17: 0: 000”, MBT2 “17: 0: 000”.
Music score element data after conversion “175”... MBT1 “20: 0: 000”, MBT2 “21: 4: 480”.

  Subsequently, the determination unit 124 specifies a bar number indicating a repetitive portion in the music based on the converted musical score element data. In this case, in accordance with the contents of “Type” and “Value” of the converted musical score element data, the determination unit 124 starts with MBT1 “13: 0: 000” of the converted musical score element data “136-1”. The part of the song that ends immediately before MBT1 “16: 0: 000” of “170”, that is, bars 13 to 15, and MBT1 “17: 0: 000” of the musical score element data “136-2” after conversion The part of the music that starts and ends immediately before MBT1 “20: 0: 000” of the musical score element data “175”, that is, the 17th bar to the 19th bar, is specified as a part to be repeatedly played.

  Subsequently, the determination unit 124 searches the performance control data 2042a for performance element data that indicates the timing after “MBT” is “13: 1: 000” and includes data that should have the same value when repeated. The data that should have the same value at the time of repetition is, for example, the portion indicating the pitch among the values of “Value” of the performance element data whose “Type” is “Note On”. On the other hand, for example, for performance element data whose “Type” is “tempo”, the value of “Value” may change during repetition, or the performance element data itself may be omitted when the tempo during repetition is the same. Therefore, the determination unit 124 does not search these performance element data. In this case, the determination unit 124 obtains performance element data “88” as a search result.

  Since “MBT” of the performance element data “88” is “13: 1: 000” and the portion to be repeatedly played is shifted by 4 bars, the determination unit 124 includes the element data included in the performance control data 2042a. , “MBT” is included in a certain range before and after “17: 1: 000”, for example, “16: 4: 466” to “17: 1: 015”, “Type” is “code”, and “ Search for “Value” that is “B ♭ m”. “MBT” is not “17: 1: 000”, but “16: 4: 466” to “17: 1: 015” is created by the performance control data 2042a based on the actual performance by the performer. This is because there is a slight timing difference between the performance element data corresponding to the repetitive portion.

  In this case, the determination unit 124 succeeds in the search and obtains performance element data “120” as a search result. If the search is successful as described above, the determination unit 124 determines that the condition regarding the consistency between the musical score element data searched in step S101 and the performance element data corresponding thereto is satisfied. In step S105, in order to make a more accurate determination, the determination unit 124 relates to a plurality of performance element data included in the thirteenth bar to the fifteenth bar, and corresponding performance elements included in the seventeenth bar to the nineteenth bar. It may be determined that the consistency between the musical score element data searched in step S101 and the performance control data 2042a is maintained only when data search is performed and all the searches are successful.

  As described above, when it is determined in step S105 that the consistency between the musical score element data and the performance element data is maintained, the determination unit 124 returns to the search process in step S101 (step S105; Yes). Since the logical score data 2041a does not include data indicating repetition symbols in addition to the score element data searched in the previous step S101, the determination unit 124 fails in the second search in step S101 (step S102; No), the consistency check process regarding the repetition symbol is terminated. On the other hand, when the musical score element data newly indicating the repetition symbol is retrieved in step S101 (step S102; Yes), the determination unit 124 repeats the processing from step S103 on for the newly retrieved musical score element data.

  If the search fails in the search in step S105, the determination unit 124 determines that the content of the performance control data 2042a is not consistent with the content of the musical score element data searched in step S101 (step S105; No). In that case, since “display correction candidates” is selected in the check box 1056 of the check screen shown in FIG. 10, the determination unit 124 selects the performance control data 2042a when the logical score data 2041a is assumed to be correct. The correction candidate data and the correction candidate data of the logical score data 2041a when the performance control data 2042a is assumed to be correct are created. More specifically, for example, the determination unit 124 replaces the contents of the seventeenth to nineteenth measures with the contents of the thirteenth to fifteenth measures in the performance control data 2042a as correction candidate data for the performance control data 2042a. create. In addition, the determination unit 124 creates data obtained by deleting the musical score element data searched in step S101 from the logical musical score data 2041a as correction candidate data of the logical musical score data 2041a. The determination unit 124 sends the correction candidate data created as described above to the display unit 125 and the change unit 127 together with data indicating the correction part (hereinafter referred to as “correction part data”) (step S106). Terminate processing once.

  In step S106, when the display unit 125 receives the correction candidate data and the correction part data from the determination unit 124, the display unit 125 updates the display of the box 1052, the box 1053, and the list 1054 so that the correction part indicated by the correction part data is included. To do. Further, the display unit 125 separately opens a display window and displays a screen (hereinafter referred to as “change instruction screen”) showing the contents of the correction candidate data as illustrated in FIG. 12 on the display window. On the change instruction screen, a box 1061 for displaying a musical score according to correction candidate data of the logical musical score data 2041a when the performance control data 2042a is correct, and performance control data when the logical musical score data 2041a is assumed to be correct are displayed. A list 1062 indicating correction candidate data 2042a, a button 1063 and a button 1064 for selecting the correction candidate data, and a button 1065 for selecting manual correction are included.

  When the user compares the contents of the current logic score data 2041a and performance control data 2042a shown on the check screen with the contents of the correction candidate data shown on the change instruction screen, and selects one of the correction candidate data A button 1063 or a button 1064 is clicked. When the button 1063 is clicked, the change instruction acquisition unit 126 sends data indicating that the correction candidate data of the logical score data 2041a has been selected to the change unit 127 according to the user's click operation. The changing unit 127 updates the logical score data 2041a stored in the storage unit 123 with the correction candidate data of the logical score data 2041a received from the determination unit 124 in step S106 according to the data received from the change instruction acquiring unit 126. . Similarly, when button 1064 is clicked by the user, change instruction acquisition unit 126 sends data indicating that correction candidate data of performance control data 2042a has been selected to change unit 127, and change unit 127 performs performance control data. The performance control data 2042a stored in the storage unit 123 is updated with the correction candidate data 2042a.

  On the other hand, when the user clicks the button 1065, the change instruction acquisition unit 126 sends data indicating that manual correction has been selected to the display unit 125. In accordance with the data, the display unit 125 allows the user to edit the box 1053 and the list 1054 on the check screen. The editing process for the data displayed in the score display or in the list display is the same as that in a normal authoring tool, and the description thereof will be omitted. The change instruction acquisition unit 126 sends data corresponding to the editing operation on the box 1053 and the list 1054 by the user to the change unit 127. The change unit 127 updates the contents of the logical score data 2041a and the performance control data 2042a stored in the storage unit 123 according to the data received from the change instruction acquisition unit 126.

  As described above, when the logical score data 2041a or the performance control data 2042a stored in the storage unit 123 is updated, the determination unit 124 uses the updated data again to end once at that time. The consistency check process (FIG. 11; step S101 to step S106) regarding the repeated symbol that has been performed is repeated. In this way, the consistency check process and the data change process are repeated until there is no inconsistency regarding the repeated symbols between the logical score data 2041a and the performance control data 2042a.

  When the determination unit 124 finishes the consistency check process for the repetitive symbols, the determination unit 124 subsequently performs similar processes for the pitch and the volume. First, the determination unit 124 searches the musical score element data 2041a for musical score element data in which “Type” is “note head”, “stem”, and “flag” and “MBT1” is equal. In this case, the determination unit 124 obtains score element data “13” to “15” as search results. Next, the determination unit 124 searches for the musical score element data preceding the musical score element data searched for “Type” that is “key” and “time”, and “MBT 1” that is closest to the searched musical score element data. . In this case, the determination unit 124 obtains score element data “8” and “9” as search results (FIG. 11; step S101). When the determination unit 124 succeeds in these searches (step S102; Yes), it performs MBT conversion on the searched score element data (step S103). As a result, the determination unit 124 obtains converted score element data. In this case, however, the musical score element data “8”, “9”, and “13” to “15” indicate the musical score elements related to the first stage, so that the musical score element data after conversion is the musical score element data before MBT conversion. Will be the same content.

  The musical score element data after the conversion shows the display of eighth notes of midrange high notes arranged at the timing of “1: 2: 240” by the combination thereof. Accordingly, the determination unit 124 is included in the performance control data 2042a within a certain range around “MBT” of “1: 2: 240”, for example, “1: 2: 225” to “1: 2: 255”. , “Type” is “Note On”, and “Value” includes “F3”. In this case, the determination unit 124 obtains performance element data “4” as a search result. Subsequently, the determination unit 124 is included in the performance control data 2042a within a certain range before and after “MBT” is “1: 2: 479”, for example, “1: 2: 464” to “1: 3: 014”. Then, the performance element data in which “Type” is “Note Off” and “Value” is “F3” is searched (step S104). Note that “1: 2: 479” indicates the timing when the length of the eighth note has elapsed from “1: 2: 240”. In this case, the determination unit 124 obtains performance control data “6” as a search result.

  As described above, the determination unit 124 searches for performance element data corresponding to the musical score element data searched in step S101 (step S104). If the search is successful, the musical score element data searched in step S101 and the musical score element data are included in them. It is determined that a condition regarding consistency with the corresponding performance element data is satisfied (step S105; Yes). In that case, the determination unit 124 returns the processing to step S101, and repeats the processing from step S101 onward for other musical score element data relating to the pitch and pitch included in the logical musical score data 2041a.

  On the other hand, if the determination in step S104 fails, the determination unit 124 determines that a condition regarding consistency is not satisfied between the musical score element data searched in step S101 and the performance element data corresponding thereto (step S104). S105; No). In this case, the determination unit 124 creates correction candidate data for the logic score data 2041a based on the performance control data 2042a and correction candidate data for the performance control data 2042a based on the logic score data 2041a.

  For example, the musical score element data “177”, “178”, and “181” are tied in a tie from the fifth bar to the sixth bar in the fourth row, and a musical score element group that indicates a mid-range haute sound that is continuously pronounced. Is shown. However, the performance control data 2042a shown in FIG. 6 includes the MBT “21: 4: 479” corresponding to the timing of the end of the sound production indicated by the musical score element data “177”, “178”, and “181”. Before and after, the performance element data of Type “Note Off” and Value “F3” cannot be found. Instead, performance element data “151” with Type “Note Off” and Value “F3” is found in MBT “20: 4: 474”. In that case, the determination unit 124 creates logical score data 2041a in which the score element data “177” and “181” are deleted as correction candidate data of the logic score data 2041a based on the performance control data 2042a. Further, the determination unit 124 creates performance control data 2042a in which “MBT” of the performance element data “151” is changed to “21: 4: 479” as correction candidate data of the performance control data 2042a based on the logical score data 2041a. To do.

  The determination unit 124 sends the correction candidate data created as described above to the display unit 125 and the change unit 127 (step S106). Since the processing of the display unit 125, the change instruction acquisition unit 126, and the change unit 127 after the correction candidate data is transmitted from the determination unit 124 is the same as that described for the consistency check regarding repetitive symbols, the description thereof is omitted. In this way, consistency with the performance element data included in the performance control data 2042a is obtained with respect to each of the score element data regarding the pitch and the sound length included in the logical score data 2041.

  By the way, depending on the consistency check regarding the pitch and the pitch, the consistency check is performed on the musical score element data included in the logical musical score data 2041a. For example, the musical score elements to be included in the logical musical score data 2041a If data is missing, the error is not detected. Therefore, the determination unit 124 continues the consistency check process regarding the pitch and the tone length by switching the relationship between the logic score data 2041a and the performance control data 2042a. As a result, the element data relating to the pitch and the pitch included in the logical score data 2041a and the performance control data 2042a have consistency with each other with higher accuracy.

  When the determination unit 124 finishes the above processing relating to the pitch and the tone length, the determination unit 124 repeats the same processing as that related to the pitch and the tone length for each of the volume, speed, lyrics, and chord. For example, the determination unit 124 searches the musical score element data “35” whose “Type” is “Forte” in the consistency check regarding sound volume (step S101), and uses the performance element data “18” as performance element data corresponding thereto. Search is performed (step S104). Since the volume indicated by “Value” of the performance element data “18” is “32” indicating a slightly lower volume, in this case, the determination unit 124 determines that the consistency condition is not satisfied (step S105; No). Then, the determination unit 124 sets the numerical value included in the “Value” of the performance element data “18” and the logical score data 2041a obtained by changing “Type” of the score element data “35” to, for example, “Meso Piano”, for example, “94”. The performance control data 2042a changed to is generated as correction candidate data, and sent to the display unit 125 and the change unit 127 (step S106).

  Further, for example, in the speed consistency check, the determination unit 124 searches for the musical score element data “139” whose “Type” is “Ritardando” (step S101), and “Type” is “ The performance element data “87” and “106” which are “tempo” are searched (step S104). “Value” of the performance element data “87” and “106” is “110” and “115”, respectively, indicating that the instruction is to gradually increase the performance speed. It is determined that the sex condition is not satisfied (step S105; No). Then, the determination unit 124 performs the logical score data 2041a in which “Type” in the score element data “139” is changed to “Accherland”, for example, and the performance in which “Value” in the performance element data “106” is changed to “105”, for example. The control data 2042a is created as correction candidate data, respectively, and sent to the display unit 125 and the change unit 127 (step S106).

  Further, for example, in the consistency check regarding lyrics, the determination unit 124 searches for musical score element data “43” whose “Type” is “lyric” (step S101), and searches for performance element data corresponding thereto (step S101). In step S104), the search fails, and instead, the performance element data “27” whose “Type” is “memo” is found. In this case, the determination unit 124 determines that the consistency condition is not satisfied (step S105; No). The determination unit 124 uses the musical score data 2041a from which the musical score element data “43” is deleted and the performance control data 2042a in which “Type” of the performance element data “27” is changed to “lyric” as correction candidate data. It is created and sent to the display unit 125 and the change unit 127 (step S106). The process of checking the consistency with respect to the code by the determination unit 124 is exactly the same as that with respect to the lyrics.

  The above is the operation of the error check tool 10 when the user of the error check tool 10 selects an initial value among the options included in the check box 1055 and the check box 1056 on the check screen. By selecting other options, it is possible to cause the error check tool 10 to perform consistency check for each item individually. In addition, when the user selects “Modify based on score” or “Modify based on performance” in the check box 1056, the error check tool 10 automatically causes the logical score data 2041 a and the performance control data 2042 a to be between. Inconsistencies can be corrected. In that case, when the determination unit 124 finds inconsistency between the data, it creates correction candidate data according to the user's selection, sends the correction candidate data only to the changing unit 127, and the display unit 202 displays the change instruction screen. Is not displayed. Then, the changing unit 127 updates the logical score data 2041a or performance control data 2042a stored in the storage unit 123 with the correction candidate data received from the determination unit 124.

  When all the various consistency checks as described above are completed, the determination unit 124 sends data indicating the completion to the display unit 125, and the display unit 125 displays a message indicating the completion of the consistency check. When the user wants to save the logical score data 2041a and the performance control data 2042a that have been changed in the consistency check process, the user clicks a button 1058 on the check screen. In response to the operation, the logic score data output unit 128 and the performance control data output unit 129 output the changed logic score data 2041a and the performance control data 2042a stored in the storage unit 123 to the HD 104, respectively. Store those data. Thereafter, the logical score data 2041a and the performance control data 2042a stored in the HD 104 are copied to the storage unit 204 of the trainer tool 20 via a storage medium or a communication network and used for learning the performance of the user of the trainer tool 20. In this case, since there is no inconsistency between the content of the score displayed on the display unit 202 of the trainer tool 20 and the performance sound generated by the sound system 203, a high-quality performance acquisition support service is provided.

[2. Second Embodiment]
The data processing apparatus (hereinafter referred to as “error check tool 50”) according to the second embodiment of the present invention is an apparatus that supports learning of music performance for the user, similar to the error check tool 10 in the first embodiment. (Hereinafter, referred to as “trainer tool 60”) is a device that checks consistency of data used and corrects necessary data when there is a mismatch between the data. However, in the trainer tool 60, instead of the performance control data 2042 in the first embodiment, moving image data that is a record of the state of the exemplary performance actually performed by the performer, and the exemplary performance along with the moving image data. Performance sound data, which is data obtained by recording a sound, is used. Further, the error check tool 50 performs a consistency check between the logic score data and the moving image data instead of the consistency check between the logic score data 2041 and the performance control data 2042 in the first embodiment. Since other points in the present embodiment are the same as those in the first embodiment, only the points in which the present embodiment is different from the first embodiment will be described below. Moreover, in the following description, the same name and code | symbol are used for what is common with the structure part etc. in 1st Embodiment.

[2.1. Movie data and performance sound data]
The storage unit 204 of the trainer tool 60 stores logical score data 2041, moving image data 501 and performance sound data 502 for each of a plurality of music pieces. FIG. 13 is a diagram schematically showing the structure of the moving image data 501. The moving image data 501 is a plurality of still image data arranged in time series. In the following description, the moving image data 501 includes still image data every 20 milliseconds, and each still image data includes data indicating timing at which the still image data is to be displayed (hereinafter, “display timing data”). Called). For example, when the display timing data indicates 0 minute 0 second 000 milliseconds, it is represented as [0m0s000ms]. Hereinafter, each of the still image data to which the display timing data is attached is referred to as “image element data”. The control unit 201 of the trainer tool 60 creates still image data that interpolates them based on adjacent image element data, and uses the still image data included in the moving image data 501 and the still image data created by interpolation as display timing data. The data are sequentially sent to the display unit 202 in accordance with the indicated timing. As a result, the still images sequentially displayed by the display unit 202 are viewed as a moving image by the user.

  By the way, as described above, the moving image data 501 is data created by recording how the performer actually performs the model performance. During recording, the performer is marked with a red circular mark (hereinafter referred to as “reference mark”) indicating the reference position in the vicinity of each key of the sound, and further, a yellow light that is lit when a key is pressed in the vicinity of all keys. A music keyboard with a lamp is used. As a result, the image indicated by the image element data included in the moving image data 501 includes an image indicating a reference mark and a lamp indicating the key that has been pressed.

  FIG. 14 is a diagram schematically showing the structure of the performance sound data 502. The performance sound data 502 includes element data including “timing” indicating the timing to output the sound, “value-R” indicating the output value of the right sound, and “value-L” indicating the output value of the left sound. (Hereinafter referred to as “sound element data”). The “timing” included in the sound element data is expressed as [0m0s000ms0], and the portion of “0m0s000ms” indicates 0 minute 0 second 000 milliseconds as in the display timing data, and the last “0” is 1 / Shows the passage of time in units of 32 milliseconds. For example, [0m0s000ms5] indicates a timing when 5/32 milliseconds have elapsed from the reference timing.

  The sound system 203 of the trainer tool 60 inputs a voltage corresponding to a value indicated by “value-R” and “value-L” included in each sound element data to the amplifier at a timing indicated by “timing”. Vibrate the diaphragm of the speaker. As a result, the user can hear the performance sound indicated by the performance sound data 502.

[2.2. Operation of trainer tool]
FIG. 15 is a diagram illustrating a training screen displayed on the display unit 202 of the trainer tool 60. The training screen in the trainer tool 60 includes a box 601 for displaying a moving image according to the moving image data 501 in addition to the components of the training screen in the trainer tool 20. When the user of the trainer tool 60 selects “exemplary performance” in the check box 2025 and clicks the button 2026, a moving image according to the moving image data 501 is displayed in the box 601, and a performance sound according to the performance sound data 502 is simultaneously displayed. The display of the musical score 2028a and the musical score 2028b according to the logical musical score data 2041 is sequentially updated as sound is generated from the sound system 203 and the moving image and performance sound progress.

  When the user clicks the button 2026 after selecting “practice” in the check box 2025, a moving image according to the moving image data 501 is displayed in the box 601, and at the same time, a metronome sound is generated from the sound system 203. As the metronome sound progresses, the display of the score 2028a and the score 2028b according to the logical score data 2041 is sequentially updated.

[2.3. Configuration and operation of error check tool]
The hardware configuration of the error check tool 50 is the same as the hardware configuration of the error check tool 10 shown in FIG. However, the moving image data 501 and the performance sound data 502 are stored in the HD 104 of the error check tool 50 instead of the performance control data 2042. The logical score data 2041, the moving image data 501 and the performance sound data 502 stored in the HD 104 of the error check tool 50 are associated with each other by assigning the same file names with different extensions only to the same music piece. It has been. The moving image data 501 and the performance sound data 502 relating to the same musical piece are obtained by recording and recording the same exemplary performance by a device that operates based on the same clock signal, so that the consistency between them is maintained. . However, since the logical score data 2041 is created separately from the moving image data 501 and the performance sound data 502, there is a possibility that there is an inconsistency between these data. The error check tool 50 checks the consistency between the logical score data 2041 and the moving image data 501, so that the logical score data 2041, the moving image data 501 and the performance sound data 502 relating to the same music are consistent. In addition, it is a tool for modifying the logic score data 2041.

  FIG. 16 is a block diagram showing a functional configuration of the error check tool 50. The error check tool 50 includes a moving image data acquisition unit 621 instead of the performance control data acquisition unit 122 as compared with the error check tool 10 in the first embodiment. Further, the error check tool 50 includes an image recognition processing unit 622 that recognizes a reference mark and an image of a lit lamp from still image data included in the moving image data 501. On the other hand, the error check tool 50 does not include the performance control data output unit 129 included in the error check tool 10.

  When the user selects a file name of a data group to be checked for consistency on the screen displayed on the display unit 125 of the error check tool 50 and instructs to start the consistency check, the logical score data acquisition unit 121 and the moving image The data acquisition unit 621 reads the logical score data 2041 and the moving image data 501 having the specified file names from the HD 104 and temporarily stores them in the storage unit 123. Hereinafter, it is assumed that the user selects “Saint March” as the file name, and the storage unit 123 stores logical score data 2041a and moving image data 501a related to the song “Saint March”.

  When the logical score data 2041a and the moving image data 501a are stored in the storage unit 123, the image recognition processing unit 622 performs recognition processing of the image of the reference mark and the lit lamp for each of the image element data included in the moving image data 501a. . First, the image recognition processing unit 622 extracts red and the similar color of the reference mark from RGB (Red, Green, Blue) data of each pixel included in the image element data. For example, when the RGB data of the reference mark is [255, 0, 0], the image recognition processing unit 622 has [245 to 225, 0 to 50, 0 to 50] among the pixels included in the image element data. ] Are extracted.

  Subsequently, the image recognition processing unit 622 groups adjacent pixels among the extracted pixels, and calculates coordinate data of the center of gravity of the graphic indicated by the pixels of each group. Further, the image recognition processing unit 622 calculates a distance from the center of gravity for each pixel located on the outer edge of the figure. If the calculated distance from the center of gravity is included before or after the value indicating the radius of the reference mark for any pixel located at the outer edge of the figure, the image recognition processing unit 622 displays the figure as an image indicating the reference mark. It is recognized that. The image recognition processing unit 622 stores the coordinate data of the center of gravity of the graphic recognized as the image indicating the reference mark in the storage unit 123 together with the display timing data included in the image element data. Hereinafter, the center of gravity of a figure recognized as an image indicating a reference mark is referred to as a “reference point”.

  Subsequently, the image recognition processing unit 622 performs recognition processing of the image of the lit lamp included in the image element data. The process of recognizing the image of the lamp with the image recognition processing unit 622 turned on is the same as the case of recognizing the image of the reference mark except that the range of the RBG data of the pixels extracted from the image element data is different. The image recognition processing unit 622 stores the coordinate data of the center of gravity of the figure recognized as an image showing the lit lamp together with the display timing data included in the image element data in the storage unit 123. Hereinafter, the center of gravity of a figure recognized as an image showing a lit lamp is referred to as a “lighting point”.

  FIG. 17 is a diagram illustrating a structure of data stored in the storage unit 123 by the image recognition processing unit 622. However, when the image recognition processing unit 622 finishes the image recognition process, the “pitch” field is blank. The data shown in FIG. 17 is data for specifying which pitch key is pressed at which timing based on the coordinate data obtained from the image element data included in the moving image data 501a. Accordingly, the data shown in FIG. 17 is hereinafter referred to as “keystroke data”. The keystroke data is a collection of element data (hereinafter referred to as “keystroke element data”) that is data obtained from each image element data. Each keying element data includes “ID” for identifying keying element data, “timing” indicating display timing data included in the corresponding image element data, “reference point” indicating coordinate data of the reference point, and coordinate data of the lighting point “Lighting point” indicating “pitch” and “pitch” indicating the pitch of the key being pressed. Each keystroke element data can include a plurality of coordinate data at “reference point” and “lighting point”. Hereinafter, in order to distinguish the keying element data, “ID” is added and expressed as keying element data “1”.

  The image recognition processing unit 622 performs processing for specifying “pitch” based on the “reference point” and the “lighting point” for each keying element data as follows. First, the image recognition processing unit 622 uses the coordinate data of “reference point” to calculate coordinate data of a point that internally divides two adjacent reference points into 12 equal parts (hereinafter referred to as “internal dividing points”). To do. Subsequently, the image recognition processing unit 622 associates a pitch with each of the reference point and the internal dividing point. Specifically, the image recognition processing unit 622 associates pitches in order from the smallest X coordinate to the reference point, such as C3, C4,..., For example, the reference point of C3 and the reference point of C4 The pitches are associated with the inner dividing points in the order of C # 3, D3,.

  Subsequently, the image recognition processing unit 622 specifies, for each piece of coordinate data included in the “lighting point”, a reference point or internal division point closest to the lighting point indicated by the coordinate data. For example, in the case of the keystroke element data “40” illustrated in FIG. 17, the coordinate data (102, 163) included in the “lighting point” is the coordinate data (53, 185) and (174) included in the “reference point”. , 183), the fifth inner dividing point (103.4, 184.2) is specified as the closest one. The image recognition processing unit 622 specifies the pitch associated with the reference point or internal division point closest to the lighting point as the pitch corresponding to the lighting point. In the case of the keystroke element data “40”, the lighting point (102, 163) is the closest to the interior dividing point located fifth from the reference point (53, 185) indicating C3, and therefore the corresponding pitch is “F3”. Is identified. When the pitch of the lighting point is specified as described above, the image recognition processing unit 622 stores data indicating the pitch in the “pitch” of the keystroke element data.

  By the way, in identifying the reference point and the lighting point, a graphic recognition technique other than the method described above may be used. Further, the “pitch” in the keystroke data may be specified by directly recognizing the shape when the key is pressed without using a reference mark or a lamp.

  When the image recognition processing unit 622 finishes the above-described pitch specification processing for all keying element data, the determination unit 124 indicates the musical score indicating the notes corresponding to the logical score data 2041a for each keying element data included in the keying data. Whether or not element data is included is determined as follows. First, the determination unit 124 converts the “timing” value of the keystroke element data into the MBT format used in “MBT1” and “MBT2” of the logical score data 2041a. Specifically, the determination unit 124 extracts score element data whose “Type” is “tempo” included in the logical score data 2041a, determines the time length of the tick at each timing, and then the elapsed time from the leading timing. MBT corresponding to is calculated. For example, when the tempo is “120”, the time length of one tick is a time length obtained by dividing 60 seconds by 120 × 480, that is, about 1.04 milliseconds. Therefore, when the tempo does not change at “120”, the timing [0m0s800ms] of the keying element data “40” is the timing when 768 ticks have elapsed from the head timing, and is expressed as “1: 2: 288” in the MBT format.

  Subsequently, the determination unit 124 converts the “pitch” value of the keystroke element data into the Y coordinate format used in “Y1” of the logical score data 2041a. For example, the pitch “F3” of the keystroke element data “40” is converted to “105”. Using the converted keying element data, the determination unit 124 includes “MBT1” of the score element data included in the logical score data 2041a before a timing of the keying element data or after a timing of the keying element data. Then, “Type” is searched for “note head” and “Value” is “pitch” of the keystroke element data. Here, “MBT1” included in a certain period after the timing of the keying element data is also searched for because “timing” included in the keying element data is timing information in actual performance. Because it has. For example, the determination unit 124 obtains score element data “13” as a search result for the keystroke element data “40”.

  If the above search is successful, the determination unit 124 then searches for score element data in which the value of “MBT1” is equal to that of the searched score element data and “Type” is “stem” and “flag”. For example, when the musical score element data “13” is searched in the above search processing, the determination unit 124 obtains the musical score element data “14” and “15” as search results. Further, the determination unit 124 searches for musical score element data including “MBT1” indicating the timing before “MBT1” of the musical score element data searched as described above, and “Type” being “key”. In this case, the musical score element data “8” is searched.

  The determination unit 124 determines whether or not the musical score element data searched as described above is consistent with the keystroke element data. For example, the musical score element data “8” and “13” to “15” indicate the eighth notes that instruct the pronunciation of the middle tone of the middle tone arranged at the timing of “1: 2: 240” as a whole. That is, it is shown that the middle tone should be pronounced during the period of “1: 2: 240” to “1: 2: 479”. On the other hand, the keying element data “40” indicates that F3 (medium tone) is keyed at the timing of “1: 2: 288”, so the determination unit 124 matches the consistency of these data. Is determined to be maintained. If the determination unit 124 determines that the logical score data 2041a is consistent with the keying element data, the determination unit 124 repeats the above-described processing related to the consistency check regarding the next keying element data included in the keying data.

  On the other hand, although the determination unit 124 has failed to retrieve the musical score element data corresponding to the keying element data, or has succeeded in retrieving the corresponding musical score element data, the musical note indicated by the musical score element data for which the timing of the keying element data has been retrieved. Is not included before and after the sound generation period indicated by, the determination unit determines that the logical score data 2041a does not maintain consistency with the keystroke element data. In that case, the determination unit 124 creates correction candidate data of the logical score data 2041a based on the keystroke element data, and sends the created correction candidate data to the display unit 125 and the change unit 127. Thereafter, the display unit 125 displays a screen prompting the user to make a correction instruction, and the changing unit 127 updates the logical score data 2041a in accordance with the user's correction instruction corresponding to the screen. When the logical score data 2041a is updated, the determination unit 124 repeats the above-described processing relating to the consistency check using the updated logic score data 2041a.

  When the determination unit 124 completes the consistency check as described above for all the keying element data included in the keying data, the determination unit 124 performs the same consistency check by exchanging the moving image data 501a and the logical score data 2041a. As a result, for example, an inconsistency such that the keystroke corresponding to the note included in the logical score data 2041a is not made according to the moving image data 501 is also detected.

  When all the consistency checks as described above are completed, the display unit 125 displays a message indicating the completion of the consistency check. When the user clicks a button for instructing data storage in response to the message, the logic score data output unit 128 outputs the changed logic score data 2041a to the HD 104, and the HD 104 stores the data. When the logical score data 2041a stored in the HD 104 in this way is used in the trainer tool 60 together with the moving image data 501a and the performance sound data 502a, the provided performance acquisition support service provides the content, video and pronunciation of the displayed score. It will be of high quality with no discrepancy between the playing sound.

[3. Third Embodiment]
A data processing apparatus according to the third embodiment of the present invention (hereinafter referred to as “error check tool 70”) includes data for instructing display and coloring of lyrics in karaoke (hereinafter referred to as “lyric data”), and a model. It is a device that checks the consistency with singing voice data and corrects the necessary data if there is a mismatch between the data. Since the configuration and operation of the error check tool 70 in this embodiment are the same as the error check tool 50 in the second embodiment in many respects, only the points that this embodiment is different from the second embodiment will be described below. Will be explained. Moreover, in the following description, the same name and code | symbol are used for what is common with the structure part etc. in 2nd Embodiment.

[3.1. Error check tool configuration]
The hardware configuration of the error check tool 70 is the same as the hardware configuration of the error check tool 10 shown in FIG. However, the HD 104 of the error check tool 70 stores audio data 701 and lyrics data 702 instead of the logical score data 2041 and performance control data 2042. These pieces of data relating to the same musical piece are associated with each other by being given the same file name with different extensions only. Furthermore, a segment database 703 used in a speech recognition process described later is stored in the HD 104 of the error check tool 70.

  The structure of the sound data 701 is the same as the structure of the performance sound data 502 (see FIG. 14) in the second embodiment. FIG. 18 is a diagram showing the structure of the lyrics data 702. The lyric data 702 is a set of element data (hereinafter referred to as “lyric element data”) indicating which character is displayed at which timing on the display unit of the karaoke device and which timing is colored. Each lyric element data includes “ID”, “lyric”, “display start timing”, “display end timing”, “coloring timing”, and “display position”. Hereinafter, when it is necessary to distinguish the lyric element data, “ID” is added and expressed as lyric element data “1”.

  “Lyrics” indicates each character included in the lyrics. “Display start timing” and “display end timing” are data in the same format as the display timing data and the like in the second embodiment. “Coloring timing” is data indicating the timing to change the display color of the displayed lyrics characters in order to indicate the timing of singing to the karaoke singer, and the format is “display start timing” and This is the same as “display end timing”. “Display position” is a numerical value indicating a horizontal position with reference to the left end position of the display unit of the karaoke apparatus.

  The segment database 703 is a database that stores data indicating a spectrum (hereinafter referred to as “spectrum data”) of a speech waveform corresponding to a segment, that is, a phonetic feature. For example, when the lyrics “Sakura” are pronounced, the sound is “#s” “s” “sa” “a” “ak” “k” “ku” “u” “ur”. ”,“ R ”,“ r−a ”, and“ a # ”. However, “#s” indicates the rising portion of the sound of “s”, for example, “s-a” indicates the portion that changes from “s” to “a”, and “a #” indicates the sound of “a”. The attenuation part is shown. The segment database 703 stores symbols indicating these segments (hereinafter referred to as “segment symbols”) and spectrum data of the speech waveform of each segment in association with each other.

  FIG. 19 is a block diagram showing a functional configuration of the error check tool 70. The error check tool 70 includes an audio data acquisition unit 721 and a lyrics data acquisition unit 722 instead of the logical score data acquisition unit 121 and the moving image data acquisition unit 621 as compared with the error check tool 50 in the second embodiment. Further, the error check tool 70 includes a voice recognition processing unit 723 that recognizes which segment is pronounced from the voice data 701 instead of the image recognition processing unit 622. Further, the error check tool 70 includes a lyrics data output unit 724 instead of the logic score data output unit 128. Further, the error check tool 70 includes a segment database acquisition unit 725.

[3.2. Operation of error check tool]
The operation of the error check tool 70 in this embodiment is different from the operation of the error check tool 50 in the second embodiment only in the operations of the speech recognition processing unit 723 and the determination unit 124. Therefore, only those different operations will be described below.

  In response to the user's start instruction for consistency check, the audio data acquisition unit 721, the lyrics data acquisition unit 722, and the segment database acquisition unit 725 read the audio data 701, the lyrics data 702, and the segment database 703 from the HD 104, respectively. Store in the storage unit 123. Subsequently, the speech recognition processing unit 723 extracts, for example, the element data included in the first 100 milliseconds from the element data included in the audio data 701 (hereinafter referred to as “audio element data”), and extracts the extracted audio element The spectrum data of the speech waveform indicated by the data group is calculated. The speech recognition processing unit 723 specifies the spectrum data of the nearest segment to the calculated spectrum data by a method such as obtaining a correlation coefficient between the calculated spectrum data and each spectrum data included in the segment database 703. To do. The speech recognition processing unit 723 stores, in the storage unit 123, the unit symbol associated with the specified unit spectrum data together with the data [0m0s000ms0] indicating the timing of the speech element data group. The data indicating this timing is the “timing” value of the head of the extracted audio element data group.

  Subsequently, the speech recognition processing unit 723 extracts speech element data included in the next 100 millisecond period from the speech data 701 and specifies the corresponding unit symbol in the same manner as described above. If the identified unit symbol is the same as the last one stored in the storage unit 123, the speech recognition processing unit 723 does not store the newly specified unit symbol in the storage unit 123, but in the next 100 milliseconds. With respect to the included speech element data, the process of specifying the corresponding unit symbol is repeated. On the other hand, when the newly identified segment symbol is different from the last one stored in the storage unit 123, the speech recognition processing unit 723 displays the newly identified segment symbol together with data indicating the timing of the extracted speech element data group. After the data is stored in the storage unit 123, the process of specifying the corresponding unit symbol is repeated for the speech element data included in the next 100 millisecond period.

  FIG. 20 is a diagram illustrating data stored in the storage unit 123 by the voice recognition processing unit 723 (hereinafter referred to as “unit timing data”). The element timing data is a set of a plurality of element data (hereinafter referred to as “element element data”), and each element element data includes “ID”, “element symbol”, and “timing”. . Hereinafter, when distinguishing each piece element data, “ID” is added and represented as piece element data “1”.

  When the speech recognition processing unit 723 completes the generation of the segment timing data for all of the speech data 701, the determination unit 124 sequentially handles the segment element data included in the segment timing data from the top in response to the pronunciation of one character. Group into segment element data groups. For example, “#s”, “s”, “s-a”, and “a” indicate a series of segments that constitute the pronunciation of “sa”, so that the determination unit 124 stores the segment element data “1” to “4”. A single element element data group is assumed.

  Subsequently, for each segment element data group, the determination unit 124 indicates, from the lyrics data 702, “Lyrics” indicates the character indicated by the segment element data group, and “Coloring timing” is the segment element data group. The lyrics element data included before and after the timing indicated by the “timing” of the first element element data included is searched. For example, regarding the element element data “1” to “4”, the determination unit 124 includes “lyric” as “sa” and “coloring timing” included in the range of [0m41s300ms] to [0m41s700ms], for example. And lyric element data “1” is obtained as a search result.

  If the lyric element data is successfully searched, the determination unit 124 determines that there is consistency between the speech data 701 and the segment element data group, and the above-described series of segment element data groups is related to the above-described series. Repeat the consistency check. On the other hand, if the search for the lyric element data fails, the determining unit 124 determines that there is no consistency between the speech data 701 and the segment element data group, and the lyric data 702 based on the segment element data group. Correction candidate data is generated, and the generated correction candidate data is sent to the display unit 125 and the change unit 127. Thereafter, when the changing unit 127 updates the lyrics data 702 in accordance with the change instruction from the user, the determination unit 124 repeats the above-described series of consistency checks using the changed lyrics data 702.

  When the consistency check as described above is completed for all the element element data included in the element timing data, the display unit 125 displays a message indicating the completion of the consistency check. When the user clicks a button for instructing data storage in response to the message, the lyrics data output unit 724 outputs the changed lyrics data 702 to the HD 104, and the HD 104 stores the data. Since the lyrics data 702 stored in the HD 104 in this way is consistent with the contents of the model song, when the lyrics are displayed by the lyrics data 702 in the karaoke apparatus, the lyrics characters 702 are displayed at an accurate timing. The display color is changed.

It is the figure which showed the structure of the trainer tool concerning 1st Embodiment of this invention. It is the figure which showed the score displayed on the display part of the trainer tool concerning 1st Embodiment of this invention. It is the figure which illustrated the content of the logic score data concerning 1st Embodiment of this invention. It is the figure which illustrated the content of the logic score data concerning 1st Embodiment of this invention. It is the figure which illustrated the content of the performance control data concerning 1st Embodiment of this invention. It is the figure which illustrated the content of the performance control data concerning 1st Embodiment of this invention. It is the figure which illustrated the screen displayed on the display part of the trainer tool concerning a 1st embodiment of the present invention. It is the figure which showed the hardware constitutions of the error check tool concerning 1st Embodiment of this invention. It is the block diagram which showed the function structure of the error check tool concerning 1st Embodiment of this invention. It is the figure which illustrated the screen displayed on the display of the error check tool concerning a 1st embodiment of the present invention. It is the figure which showed the processing flow of the consistency check performed by the determination part of the error check tool concerning 1st Embodiment of this invention. It is the figure which illustrated the screen displayed on the display part of the error check tool concerning 1st Embodiment of this invention. It is the figure which showed typically the structure of the moving image data concerning 2nd Embodiment of this invention. It is the figure which showed typically the structure of the performance sound data concerning 2nd Embodiment of this invention. It is the figure which illustrated the screen displayed on the display part of the trainer tool concerning 2nd Embodiment of this invention. It is the block diagram which showed the function structure of the error check tool concerning 2nd Embodiment of this invention. It is the figure which showed the structure of the keystroke data concerning 2nd Embodiment of this invention. It is the figure which showed the structure of the lyric data concerning 2nd Embodiment of this invention. It is the block diagram which showed the function structure of the error check tool concerning 2nd Embodiment of this invention. It is the figure which showed the structure of the fragment | piece timing data concerning 2nd Embodiment of this invention.

Explanation of symbols

  10.50.70 ... Error check tool, 20.60 ... Trainer tool, 30 ... Keyboard, 101 ... CPU, 102 ... ROM, 103 ... RAM, 104 ... HD, 105 ... Display, 106 ... Keyboard, 107 ... Mouse, 108 205 ... Input / output I / F, 121 ... Logical score data acquisition unit, 122 ... Performance control data acquisition unit, 123/204 ... Storage unit, 124 ... Determination unit, 125/202 ... Display unit, 126 ... Change instruction acquisition unit DESCRIPTION OF SYMBOLS 127 ... Change part, 128 ... Logic score data output part, 129 ... Performance control data output part, 201 ... Control part, 203 ... Sound system, 206 ... Oscillator, 401 ... External storage device, 402 ... Internet, 621 ... Movie data Acquisition unit, 622 ... Image recognition processing unit, 721 ... Audio data acquisition unit, 722 ... Lyric data acquisition unit 723 ... voice recognition processing unit, 724 ... lyric data output unit, 725 ... unit database acquisition unit.

Claims (5)

First acquisition means for acquiring first aggregate data including at least one element data that is a combination of timing data indicating timing and content data that is data related to timing indicated by the timing data;
Aggregated data including one or more element data that is a combination of timing data indicating timing and content data relating to the timing indicated by the timing data, and content in a format different from that of the first aggregated data Second acquisition means for acquiring second set data including data;
Of the element data included in the first aggregate data, the first content indicated by the content data included in the element data including the timing data indicating a specific timing, and the element data included in the second aggregate data Determination means for determining whether or not the second content indicated by the content data included in the element data including the timing data indicating the specific timing satisfies a predetermined relationship. A data processing device. If the determination means determines that the first content and the second content do not satisfy a predetermined relationship, element data including content data indicating the first content and the second content The data processing apparatus according to claim 1, further comprising a notifying unit that notifies information specifying at least one of element data including content data indicating content. When the determination means determines that the first content and the second content do not satisfy a predetermined relationship, content data indicating the first content and content indicating the second content The data processing apparatus according to claim 1, further comprising: a changing unit that changes at least one of the data according to a predetermined condition. Each of the first aggregate data and the second aggregate data is:
Performance control data for instructing the automatic performance device to control the tone of music contained in the music,
Logical score data that is a combination of data indicating the constituent elements of the musical score of the music and timing data indicating the position of the constituent elements on the time axis in the music;
Multiple character data arranged in time series,
Sound data indicating the sound or sound of a song,
The data processing apparatus according to any one of claims 1 to 3, wherein the data processing apparatus includes at least one of a plurality of still image data and moving image data arranged in time series. Indicates a specific timing among the element data included in the first set data including one or more element data that is a combination of the timing data indicating the timing and the content data that is the data related to the timing indicated by the timing data. First content indicated by content data included in element data including timing data;
Aggregated data including one or more element data that is a combination of timing data indicating timing and content data relating to the timing indicated by the timing data, and content in a format different from that of the first aggregated data Whether the second content indicated by the content data included in the element data including the timing data indicating the specific timing among the element data included in the second set data including the data satisfies a predetermined relationship A program for causing a computer to execute a process for determining whether or not.

Images