CN113963601B - Rhythm singing visualization method, system and device - Google Patents
Rhythm singing visualization method, system and device Download PDFInfo
- Publication number
- CN113963601B CN113963601B CN202111230129.8A CN202111230129A CN113963601B CN 113963601 B CN113963601 B CN 113963601B CN 202111230129 A CN202111230129 A CN 202111230129A CN 113963601 B CN113963601 B CN 113963601B
- Authority
- CN
- China
- Prior art keywords
- note
- notes
- beat
- module
- card
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000033764 rhythmic process Effects 0.000 title claims abstract description 78
- 238000007794 visualization technique Methods 0.000 title claims abstract description 16
- 238000012800 visualization Methods 0.000 claims abstract description 21
- 238000004519 manufacturing process Methods 0.000 claims abstract description 11
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims description 55
- 230000006698 induction Effects 0.000 claims description 27
- 230000000007 visual effect Effects 0.000 claims description 22
- 230000001020 rhythmical effect Effects 0.000 claims description 20
- 230000033001 locomotion Effects 0.000 claims description 19
- 238000004891 communication Methods 0.000 claims description 13
- 239000011295 pitch Substances 0.000 claims description 6
- 239000003086 colorant Substances 0.000 claims 1
- 230000015572 biosynthetic process Effects 0.000 abstract description 5
- 238000003786 synthesis reaction Methods 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000000875 corresponding effect Effects 0.000 description 74
- 238000000034 method Methods 0.000 description 25
- 238000010586 diagram Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 9
- 238000012549 training Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B15/00—Teaching music
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/04—Electrically-operated educational appliances with audible presentation of the material to be studied
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Electrophonic Musical Instruments (AREA)
Abstract
The invention relates to the technical field of music aided teaching, and discloses a rhythm singing visualization method, a system and a device, which comprise the following steps: s1, sequentially inputting a beat number and a plurality of notes, and presetting a beat time value and a reference loudness; s2, sequentially acquiring a signature and a plurality of notes according to the sequence; s3, determining the beat intensity according to the beat number, determining the rhythm intensity according to the note sequence, and synthesizing or calling corresponding audio according to the beat intensity, the rhythm intensity and the reference loudness; s4, determining information such as the position of the indication module according to the sign and note information; s5, the control indication module moves according to the track line, and the sound production module is controlled to synchronize the audio sound production. The beneficial effects are as follows: according to the invention, through setting the note and the number-clapping unit module capable of being freely combined and combining with the acoustic model synthesis technology, rhythm content capable of performing autonomous vision and singing exercise is enriched, and personalized requirements of users are met; through setting up visualization module, reduced the degree of difficulty of singing rhythm exercise.
Description
Technical Field
The invention relates to the technical field of music aided teaching, in particular to a rhythm singing visualization method, a system and a device.
Background
Rhythms are the sum of the length and intensity of sound, and rhythms and singing are the basic functions of learning and reading a spectrum. The reading spectrum has two main contents, namely, the sound level and the rhythm. The purpose of rhythmic singing is to master the note combination of various values and the intensity and speed of different beats.
In music, time is divided into equal basic units, each unit being called a "beat" or simply a "beat". The time value of the "beat" is expressed by the time value of the note, and different notes correspond to different time values, and the time value of one beat can be quarter note (namely one beat of quarter note), half note (one beat of half note) or eighth note (one beat of eighth note). The beat time is a relative time concept, such as when the prescribed tempo of the music is 60 beats per minute, the time taken by each beat is one second, and the half beat is one half second; when the prescribed speed is 120 beats per minute, the time per beat is half a second, and so on. After the basic value of the tempo is determined, notes of various values are associated with the tempo. For example, when a quarter note is taken as one beat, one full note corresponds to four beats, one half note corresponds to two beats, an eighth note corresponds to half beat, and a sixteen note corresponds to one quarter beat; if an octave is taken as one beat, a whole note corresponds to an octave, a half note is a four beat, a quarter note is a two beat, and a sixteen note is a half beat.
In rhythmic singing, exercise is often performed by a beating method, namely, a beat is defined by the actions of hands or feet at one time and the action of feet at one time, each note pressing value is played as a beat to say "the sound" or a certain sound, and the sound and the action of the "the sound" are required to be kept uniform and stable.
In the prior rhythm singing training, taking an explanatory diagram shown in fig. 1 as an example, the first row of fig. 1 represents a note number by Arabic numerals, the second row represents a rhythm, the rhythm is composed of a beat number and notes, the musical notes are divided by small interval lines, the second row of 2/4 represents a beat number, which characterizes different musical beats, emphasis is given to a combination rule of strong beats and weak beats, specifically, the total length of notes of each bar in a music score is 2/4, the quarter notes are taken as one beat, the bars are provided with two beats, and commonly, 1/4 beats, 2/4 beats, 3/8 beats, 6/8 beats, 7/8 beats, 9/8 beats, 12/8 beats and the like are provided. The third row represents the beat rule of the corresponding note, and the fourth row represents the pronunciation intensity relationship of the note. In fig. 1, notes represent the length of a sound, and the length and intensity of a corresponding rhythm sound can be clearly determined in combination with a sign. However, as can be seen from fig. 1, even though the notes with the same pronunciation time length have different signs or positions in the bars, the pronunciation strengths corresponding to the notes are different. The common pronunciation intensity rules are shown in table 1.
When the student performs the teaching exercise of the rhythm of the visual singing, the student mainly controls the pronunciation length and the intensity of each note in the continuous rhythm visual singing, familiarizes and determines the intensity and the pronunciation length of the notes according to the sign and the notes, and makes a sound of 'Da, da … …' or '1, 2, 3 … …' or the like to represent each note. For beginners, the accurate control of the pronunciation time and intensity of each note in the rhythm is difficult, and a teacher can perform visual singing pronunciation demonstration, and the pronunciation position of each note in the time domain is indicated by the space position of hands by moving up and down, so that the beginners can learn the music rhythm more easily through visual and auditory synchronous demonstration.
For a rhythmic singer, particularly a child or an infant, when performing a singing rhythm exercise, the rhythmic singer is typically demonstrated by a teacher, or a video is watched, or exercise is performed through an auxiliary metronome. The metronome in the market is characterized in that related parameters are set through a plurality of buttons, so that the metronome is complex to set for children and is not easy to use; on the two hand, the metronome cannot fully demonstrate the rhythm according to the note combination set by itself, and is often in a fixed several modes; in three aspects, a metronome is generally a segment in which the beat is fixed and then pronounces, and its display portion is also segmented and discrete, and cannot indicate the pronouncing position of each note in the time domain by a continuously changing spatial position by hand like a teacher.
Disclosure of Invention
As known from the background art, for a rhythmic singer, especially a child or an infant, the existing metronome is complex to operate, has a single mode, and cannot show the relationship between the intensity of a note and the continuous change of the note in real time. In order to overcome the technical problems in the prior art, the invention provides a rhythm singing visualization method, a system and a device, and the specific technical scheme is as follows:
a rhythmic singing visualization method, comprising the steps of: s1, sequentially inputting a beat number and a plurality of notes, and presetting a beat time value and a reference loudness; s2, sequentially acquiring a signature and a plurality of notes according to the sequence; s3, determining the beat intensity according to the beat number, determining the rhythm intensity according to the note sequence, and synthesizing or calling corresponding audio according to the beat intensity, the rhythm intensity and the reference loudness; s4, determining the position information of the corresponding time point indication module according to the number of the notes, the sequentially input beat time values; determining a track line for indicating the movement of the module according to the position information; the trajectory line corresponds to the duration of the sequentially input notes; s5, the control indication module moves according to the track line, and the sound production module is controlled to synchronize the audio sound production.
Further, in the step S3, the step of retrieving the corresponding audio refers to directly retrieving the original audio of the stored corresponding note and beat combination; synthesizing corresponding audio refers to determining an acoustic model according to beat intensity, the rhythm intensity and reference loudness, and calling original audio corresponding to a single note according to the acoustic model to synthesize continuous and smooth audio.
Further, in the step S3, it is determined whether the current rhythm combination corresponding to the sign and the note belongs to a high-frequency combination or a low-frequency combination; for high-frequency combination, directly calling corresponding audio; for low frequency combinations, the corresponding audio is synthesized.
Further, the method comprises the following steps: s1, a user inputs a beat number and a plurality of notes in sequence at a terminal, and presets a beat time value and a reference loudness; s2, the central processing unit sequentially acquires a signature and a plurality of notes according to the sequence; s3, the central processing unit determines the pronunciation intensity of each note according to the sign, the notes and the reference loudness, and calls or synthesizes corresponding audio; s4, the central processing unit determines the position information of the corresponding time point indication module according to the number of the beats, the plurality of notes sequentially input and the beat time value; determining a track line for indicating the movement of the module according to the position information; the trajectory line corresponds to the duration of the sequentially input notes; s5, the central processing unit controls the indication module to continuously move at a constant speed according to the track line, and controls the sounding module to sound according to the audio frequency.
Further, the method comprises the following steps: s1, a user sequentially splices a first note card with sign information and a plurality of second note cards with notes or basic combination note information, puts the spliced first note card and second note card into a note card induction area, and presets a beat time value and a reference loudness; s2, the note card induction area sequentially acquires the number and the note according to the sequence, converts and transmits the number and the note to the central processing unit, and the central processing unit sequentially acquires the number and the note according to the sequence; s3, the central processing unit determines the pronunciation intensity of each note according to the sign, the notes and the reference loudness, and calls or synthesizes corresponding audio; s4, the central processing unit determines the position information of the corresponding time point indication module according to the number of the beats, the plurality of notes sequentially input and the beat time value; determining a track line for indicating the movement of the module according to the position information; the trajectory line corresponds to the duration of the sequentially input notes; s5, the central processing unit controls the indication module to continuously move at a constant speed according to the track line, and controls the sounding module to sound according to the audio frequency.
Further, in S1, the first note card and the second note card have the same size; the note card induction area is provided with a clamping groove, a groove or a magnet, and the first note card and the second note card are in clamping connection, concave-convex connection or magnetic attraction connection with the note card induction area; the mode of acquiring the information of the first note card and the second note card by the note card sensing area is radio frequency chip sensing identification, image coding identification, code hole identification, image content identification or multi-touch identification.
Further, in S1, the user sequentially concatenates the first note card with the sign information, the plurality of second note cards with the note duration values, and the plurality of third note cards with the note pitches, places the concatenated first note card, second note card, and third note card into the note card sensing area, and presets the beat duration values and the reference loudness.
Further, the size of the second note card is correspondingly proportional to the note duration on the second note card; the note card induction zone is provided with a plurality of note card induction zones, and the tone color of each note card induction zone can be set according to the needs.
Further, between S1 and S2, S11 is included, whether the inputted notes contain the bar corresponding to the sign is judged, if so, S2 is entered, and if not, S1 is returned and error reporting is prompted.
A rhythmic singing visualization system includes a terminal for inputting a beat number and notes; the communication module is used for being respectively in communication connection with the terminal and the central processing unit; a plurality of note cards, wherein the note cards are provided with a signature and notes; the memory stores audio data, note card codes, acoustic models, music rules and other information corresponding to each note card; the note card identification module is provided with a note card induction zone and is used for identifying note cards placed in the note card induction zone; an indication module for moving along the track line, and representing the time domain position of the notes in real time; the sounding module is used for playing audio; the central processing unit determines the pronunciation intensity of each note according to the sign, the notes and the reference loudness, and calls or synthesizes corresponding audio; determining position information of a corresponding time point indication module according to the number of the notes and the beat time values which are sequentially input; determining a track line for indicating the movement of the module according to the position information; the trajectory line corresponds to the duration of the sequentially input notes; and the control indication module continuously moves at a constant speed according to the track line, and the sounding module is controlled to sound according to the audio frequency.
The utility model provides a rhythm visual singing visualization device, includes foretell system, still includes click module or sound pickup module, click module record user click operation and relevant time point, sound pickup module record user's pronunciation and relevant time point, central processing unit compares user click information and user's pronunciation information with the pronunciation content of corresponding note to calculate the deviation, output result after handling.
The beneficial effects of the invention are as follows: according to the invention, through setting the note and the number-clapping unit module capable of being freely combined and combining with the acoustic model synthesis technology, the content capable of performing the singing exercise is greatly enriched, and the personalized requirements of users are met; by setting the note card capable of being automatically identified, the convenience of using the note combination rhythm is improved; through the inscription of the macroscopic note proportion line corresponding to the equal proportion of the pronunciation time length of the note, then the control indication module continuously moves along the note proportion line at a constant speed, and simultaneously the sounding device is controlled to sound synchronously, the abstract music sounding process is simplified into visual linear motion, and the entrance difficulty of rhythmic vision beginners is greatly reduced.
Drawings
Fig. 1 is a schematic diagram of a rhythmic singing training element.
Fig. 2 is a flow chart of the method of the present invention.
FIG. 3 is a schematic diagram of a first note card.
FIG. 4 is a schematic diagram of a second note card.
FIG. 5 is a schematic diagram of a third note card.
FIG. 6 is a schematic diagram of a combination of a first note card and a second note card.
Fig. 7 is a system block diagram of the present invention.
Fig. 8 is a schematic structural view of a first embodiment of the device of the present invention.
Fig. 9 is a schematic structural view of a second embodiment of the device of the present invention.
Fig. 10 is a schematic structural view of a third embodiment of the device of the present invention.
Fig. 11 is a schematic structural view of a fourth embodiment of the device of the present invention.
Fig. 12 is a block diagram of a fifth embodiment of the apparatus of the present invention.
Fig. 13 is a schematic structural view of a sixth embodiment of the device of the present invention.
Fig. 14 (a) is a schematic structural diagram of a note card reading apparatus according to a seventh embodiment of the present invention.
Fig. 14 (B) is a schematic view of a first state of a tempo visualization device of a seventh embodiment of the invention.
Fig. 14 (C) is a schematic view of a second state of a rhythm visualization device of a seventh embodiment of the present invention.
Fig. 15 is a schematic diagram of the movement process of the indication module of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order to overcome the defects that a metronome in the prior art is complex in operation, single in mode and incapable of reflecting the strong and weak relation of notes and the continuous change of the notes in real time, the invention designs a rhythm visual singing visualization method, a system and a device.
A visual singing method of rhythm is shown in figure 2, S1, input the number of beats and a plurality of notes sequentially, and preset beat time value and reference loudness; s2, sequentially acquiring a signature and a plurality of notes according to the sequence; s3, determining the beat intensity according to the beat number, determining the intensity of each note according to the note sequence, and synthesizing corresponding audio according to the beat intensity, the note intensity and the reference loudness; s4, determining the position information of the corresponding time point indication module according to the number of the notes, the sequentially input beat time values; determining a track line for indicating the movement of the module according to the position information, wherein the track line corresponds to the beat lines of the notes which are sequentially input; the position of the indication module on the track line corresponds to the relative position of a single note on the beat line; s5, controlling the indication module to move according to the track line, and controlling the sounding module to sound according to the audio frequency.
Aiming at the characteristic of single mode of the existing metronome, the rhythm singing visualization method mainly comprises the following two different methods according to different transmission modes of note information and pat information.
The first method is as follows: 1. directly inputting a beat number and a plurality of notes through intelligent terminals such as a mobile phone, a computer and the like, and presetting a beat time value and a reference loudness; beat time value refers to the corresponding time of one beat; the reference loudness refers to the decibel size of the audio, and because in the rhythm visual singing exercise, beat accents and rhythm accents need to be distinguished, the intensity of a single note is determined by the intensity of the beat where the single note is positioned and the position in the music bar, and therefore the reference loudness needs to be preset; after the terminal inputs the related data, the data is directly acquired by the central processing unit in a communication mode; note that the beat value and the reference loudness are input autonomously by the user, and when the user does not select input, a system default value may be used.
2. The central processing unit synthesizes corresponding audio; the central processing unit firstly determines the beat intensity according to the beat number, and when the beat number is determined, the beat intensity in the bar is fixed and can directly determine the intensity relation of single beat according to the beat number; then determining the rhythm strength of each note according to the input note sequence, wherein the rhythm strength refers to the regular strength relationship presented by different numbers and positions of notes in a single beat, and the strength of each note in the single beat can be determined at the moment; and finally, determining the intensity relation of each input note according to the preset reference loudness, and then synthesizing the audio correspondingly. There are two processing modes when synthesizing audio; in the first mode, the system stores the original audio corresponding to the corresponding note and beat combination, and the next processing is carried out after direct calling; and in a second mode, single original audio corresponding to a single note card is stored in the system, and corresponding continuous smooth audio is synthesized according to the corresponding pronunciation intensity law, the tone, the beat time value and the acoustic model determined by the reference loudness information, and the next processing is carried out after the synthesis.
3. An indication module trajectory is determined. In the invention, the abstract music sounding process is mainly expressed as a visual concrete indication module motion process. In determining the trajectory line, reference is made to the procedure of the original beat. The central processing unit projects the sounding time of the notes on a horizontal line or a circumferential line correspondingly according to the mathematical model, and determines a mark point corresponding to each note according to a preset value, wherein the mark point is the position of the time indication module, and finally the trajectory line is determined specifically through other forms such as a curve, a broken line, a straight line and the like. The specific form of the track line is shown in the drawings in the embodiments.
4. And finally, the central processing unit controls the indication module to move according to the track line and controls the sounding module to sound synchronously according to the audio frequency. The synchronous sounding refers to that when the indication module correspondingly passes through the mark point of the note, the sounding module sounds the audio corresponding to the note and continuously sounds within the range of the duration value of the note.
The second method is as follows: the clapping information and the note information are carried in a physical form, and the clapping and the corresponding notes can be manually adjusted based on training purposes before the clapping information and the note information are input. The identification and conversion process between the note card and the sensing device is added in the transmission process of the sign information and the phonetic symbol information. A rhythmic singing visualization method comprises the following steps.
1. The user splices a first note card with the sign information and a plurality of second note cards with the note information in sequence, puts the spliced first note card and second note card into a note card induction zone, and presets a beat time value and a reference loudness.
The note card comprises a first note card and a second note card, wherein the first note card is printed with the sign information, and the second note card is printed with the note information. Note cards include, but are not limited to, specific splice objects such as cards, puzzles, building blocks, and the like. In a specific embodiment, the structure of the first note card is shown in fig. 3, on which the sign information is printed, and the sign may be, but is not limited to, a score or a pattern. The sign of the music expresses different beats, and refers to the regular combination of strong beat and weak beat, in particular to the total length of notes of each bar in the music score. In a specific embodiment, as shown in fig. 4, the second note card is printed with visual note information, where the note information may include, but is not limited to, numbered musical notes or staff notes, one note card may correspond to one note or may correspond to a plurality of combined notes, and the note information mainly records the duration of the note. Specifically, the note cards representing the values in fig. 4 are, in order from top to bottom, a full note card, a half note card, a quarter note card, an eighth note card, and a sixteen note Fu Yinfu card, and also a corresponding rest card, and a combination note card, which are not illustrated here. It should be noted that, although the second note card indicating the duration in fig. 4 is a second note card having a length proportional to the corresponding note duration, a pattern having a length not proportional to the note duration may be used.
A third note card characterizing the pitch of music is also provided in the present invention, as shown in fig. 5, preferably sized not to exceed the size of the second note card representing the minimum value for ease of use in combination. When in use, the third note card can be spliced with the second note card for use together; the third note card may also be configured to be used alone as a card containing both pitch and note duration information.
Since at least one integral musical bar is needed in the rhythm singing to show the relationship between the intensity of the beats, the plurality of note cards should contain at least one integral musical bar composed of a first note card and a corresponding number of second note cards as shown in fig. 6 when the note cards are spliced. In the splicing process, in order to accurately and completely carry out visual expression on music information, the note card splicing can be sequentially spliced according to a specific sequence, and can be but is not limited to a horizontal sequence from left to right, a vertical sequence from top to bottom, a clockwise sequence or the like. In order to facilitate information reading of the note card, preferably, the spliced shape of the note card is consistent with the shape of the note card sensing area.
2. And the note card induction area sequentially acquires a signature and a plurality of notes according to the sequence, and converts and transmits the signature and the notes to the central processing unit.
After the note cards are spliced, the note cards are placed in a note card induction area, and the note card identification module can identify the arrangement sequence, the signature information and the note information on the note cards according to the spliced shape or the distance relation. In a specific embodiment, a radio frequency module, an NFC module, a barcode, an identification code hole, or image identification information may be disposed on the first note card, the second note card, and the third note card, and the note card sensing area identifies the signature information and the note information on the notes by identifying the corresponding radio frequency information, barcode, identification code hole, or image identification information. More specifically, the note card identification module records the position number and the code of the note card in the identification process, and transmits the position number and the code to the central processing unit, and the central processing unit retrieves the relevant data of the note card from the storage according to the code of the note card. For example, taking the note card in fig. 5 as an example, the note card recognition module can use data packets to transmit after recognizing the note card, and the corresponding data packets are (1, 2/4), (2, 16), (3, 4), (4, 8), (5, 16), namely, represented in the form of (X, Y) data packets, wherein X represents the position number of the note card, Y represents the number of the note card or the note information, the encoding rule of the (X, Y) data packets can be set according to the requirement, and different encodings represent different note information and number information, which can be suitable for combination and superposition of the note card. The CPU sequentially retrieves the relevant data in the memory according to the (X, Y) data packets.
Because the complete musical section comprises the sign and the corresponding section notes, the note card sensing area can judge whether the spliced note card is effective after identifying the corresponding sign information and the note character information, if and only if the spliced note card at least comprises a first note card and the complete musical section formed by a plurality of second note cards, the note card is effective, otherwise, the error reporting is prompted.
It should be noted that, since the second note card itself carries the associated note information, in order to facilitate the user to more intuitively observe the duration relationship of the notes on each note card, the second note card may be preferably sized to be proportional to the duration correspondence of the carried note, for example, when the second note card is rectangular, the length of the second note card may be corresponding to the duration of the note, so that the ratio of the lengths of the note cards is equal to the ratio of the duration of the notes thereon. For example, when the second note card is in an arc shape, the number of circle center angles of the second note card can be corresponding to the note time value, so that the ratio of the number of circle center angles of each note card is equal to the ratio of note time periods on the second note card. The first note card and the second note card may also be manufactured with a standard size, in which case the size of the second note card does not correspond to the note duration thereon.
3. Determining the beat intensity according to the beat number, determining the rhythm intensity according to the note sequence, and determining corresponding audio according to the beat intensity, the rhythm intensity and the reference loudness; the step is the same as the first method, and it is to be noted that the system classifies the high frequency occurrence or the low frequency occurrence according to the combination of the sign and the tone, and acquires corresponding audio by adopting different schemes respectively; the high-frequency corresponding combination can be obtained according to the statistics of the existing music learning materials, or can be obtained according to the automatic statistics of the actual use cases, and is comprehensively determined. Because of the traditional card recognition pronunciation technology, most of the methods are to inquire corresponding audio files in a system according to the codes of corresponding playing audios and then play the corresponding audio files, namely, the first mode in the first method; the system uses this scheme for high frequency tempo combination, which can guarantee continuous smoothness of audio. However, due to the addition of the combined cards, the memory of the system can be greatly increased by combining the cards, for example, 12 cards representing different notes or combined notes, and the combination modes have different strengths and weaknesses due to different note positions, so that 12 factorial species schemes are combined; as the number of cards increases, it is obviously not feasible to exhaust these combinations and store the corresponding audio files. Therefore, the audio file synthesis technology is adopted, namely, in the second mode in the first method, only the audio corresponding to a single card is needed to be prepared, then the real-time synthesis is carried out through an acoustic module according to the pronunciation intensity rule of notes, and the system combines the rhythms of long tail and low frequency, so that the practical application of the product is greatly expanded. Through flexible application of the two methods in the system, the audio quality of the high-frequency note combination rhythm exercise is guaranteed, and the requirement of the free combination note exercise rhythm is met.
4. Determining position information of a corresponding time point indication module according to the number of the notes and the beat time values which are sequentially input; determining a track line for indicating the movement of the module according to the position information, wherein the track line corresponds to the beat lines of the notes which are sequentially input; the position of the indication module on the track line corresponds to the relative position of the single note on the beat line.
5. And the control indication module moves according to the track line, and the sounding module is controlled to sound according to the audio frequency. The second method has the following three characteristics, firstly, the demonstration rhythm can be set in a card combination and splicing mode, the method is very suitable for children, and the interaction mode is relatively simple; secondly, the note cards can be freely spliced and combined into various rhythms instead of a plurality of set rhythms, and the rhythms which can be trained are more abundant in combination; thirdly, the pronunciation position of each note in the time domain is indicated through the continuous motion of the indication module in the space, the initial position of pronunciation of each note is clearly displayed, rhythm singing is carried out for the sufficient pre-judgment time of a user, and the situation that a beginner always slows down a little when the indication mode is used for prompting is effectively avoided.
A rhythmic singing visualization system, as shown in fig. 7, a terminal for inputting a beat number and notes; the communication module is used for being respectively in communication connection with the terminal and the central processing unit; a plurality of note cards, wherein the note cards are provided with a signature and notes; a memory storing information data corresponding to each note card; the note card identification module is provided with a note card induction zone, and the note card identification module is used for identifying note cards placed in the note card induction zone; the indication module is used for moving along the note proportional line and representing the time domain position of the sounding note in real time; the sounding module is used for playing audio; the central processing unit controls the movement of the indication module according to the total length of the note proportional line and the total time value of a plurality of notes; determining the pronunciation intensity of each note according to the sign, the notes and the reference loudness, and calling or synthesizing corresponding audio; determining position information of a corresponding time point indication module according to the number of the notes and the beat time values which are sequentially input; determining a track line for indicating the movement of the module according to the position information and displaying a visual track line through a display, wherein the track line corresponds to the beat lines of the notes sequentially input; the position of the indication module on the track line corresponds to the relative position of a single note on the beat line; and the control indication module continuously moves at a constant speed according to the track line, and the sounding module is controlled to sound according to the audio frequency.
Specifically, the note card can be, but is not limited to, a paper card, a paper jigsaw, a plastic card or a building block or a card or a building block made of other materials, wherein a sign or a note is printed on the note card, and the first note card with sign information can be of a uniform standard size. The shape of the note card may be rectangular, arcuate, or other irregular shape. Preferably, the length of the second note card of the rectangle is positively correlated with the note duration, and the central angle corresponding to the second note card of the arc is positively correlated with the note duration. Meanwhile, the note card is provided with a radio frequency module, an NFC module, a bar code, an identification code or image identification information which can be identified and corresponds to the sign or the note, and multi-point touch information.
A visual singing device for rhythm comprises a plurality of note cards, a note card identification module with a note card induction zone, a storage, an indication module, a sounding module and a central processing unit. The central processing unit is respectively in communication connection with the note card identification module, the storage, the indication module and the sounding module.
In the first embodiment, as shown in fig. 8, a rhythm visual singing visualization device includes a rectangular base plate 11, a rectangular note card 12 and a circular tower-shaped sound generating device 13, a slot 14 is arranged on the base plate, and a plurality of note cards can be spliced and embedded in the slot 14. A note card identification module, a memory and a central processing unit are arranged in the bottom. After the note card is embedded into the clamping groove, the device performs analysis processing, the note card identification module acquires corresponding card information according to the bar code at the bottom of the note card, and the central processing unit controls the indication block to move from left to right through data processing, so that the sounding device can sound synchronously. The sounding device 13 is in communication connection with the central processing unit, and the appearance of the sounding device does not influence the implementation effect of the invention. In this embodiment, the trace line 10 is a broken line with an arrow, and the indication module 20 is a circular cursor. Both the trace 10 and the indication module 20 are displayed on a display screen, with the arrow on the trace 10 being determined by the note of the minimum value entered, and then marked on the trace 10 in sequence. In the present embodiment, the horizontal projection length of the trajectory line 10 corresponds to the note duration.
In the second embodiment, as shown in fig. 9, a clamping groove and an indication block are added on the basis of the first embodiment, and at this time, different clamping grooves can be formed, so that the sound generating device can generate sounds with different timbres. The first embodiment is applicable to basic rhythm exercises and the second embodiment is applicable to combined rhythm familiarity. In the single slot of the first embodiment, music bar note cards with different beats can be inserted, but the arrangement of more note cards can be satisfied in the form of multiple slots because the length of the second note card limits the length of the base plate. While the corresponding indication blocks of different clamping grooves can synchronously or asynchronously move. In this embodiment, the trace line 10 is a horizontal line with an arrow, and the indication module is an arrow-shaped cursor, which is displayed by a display screen. The length of the trace line 10 at this point corresponds to the duration of the note.
In the third embodiment, as shown in fig. 10, in this embodiment, a circular base plate is used as the base plate, and an annular clamping groove and a concentrically arranged pointer are formed on the base plate. The note card is correspondingly designed into an arc shape, and can be embedded in the annular clamping groove after being spliced. The note card identification module, the memory, the sounding module, the central processing unit are all built-in the bottom plate, note card identification module passes through the radio frequency module discernment note card information that note card is built-in. Compared with a rectangular bottom plate, the embodiment displays the rhythm as circular motion, and can more intuitively display the music rhythm when the number of spliced single music bars or note cards is small. In this embodiment, the track line 10 is a circular arc segment, and the indication module is an external pointer concentric with the track line 10, where the central angle of the track line corresponds to the note duration.
In the fourth embodiment, as shown in fig. 11, the device comprises a housing with a certain modeling design, a slot for inserting a note card is arranged on the housing, the note card identification module identifies information corresponding to the card through a coding hole at the bottom of the note card, a region for placing a corresponding note track line pattern is arranged at the lower end of the slot, and a row of LED lamp strips are arranged below the region; the different musical notes are taken as rhythms of one beat, the printed corresponding track line reference pattern blocks are arranged in the area, and different reference pattern blocks can be replaced according to different beat settings, for example, a quarter musical note is taken as one beat, and a user can place the track line pattern blocks corresponding to the quarter musical note as one beat in the area as a reference. In this embodiment, a slot structure is adopted to replace the original groove design, and an LED lamp strip is used as an indication module. Specifically, when the note card is inserted, the LED lamp strip controls the LED lamps to light up from left to right successively, and the user combines the track line reference pattern as a reference. The central processing unit controls the sounding device and the LED lamp to sound synchronously. It should be noted that, in order to ensure visual continuity, the light belt is a flexible LED light belt, and meanwhile, the arrangement of LEDs needs to be compact, so as to ensure the visual effect of continuous movement. In this embodiment, a curve with peaks or valleys is used as the trace line, and an LED strip is used as the indication module, where the horizontal projection of the trace line corresponds to the note duration.
In the fifth embodiment, as shown in fig. 12, a clicking module or a sound pickup module is added to the original system structure. The clicking module can be buttons, keys and the like, and performs clicking operation according to the music rhythm by a user; the sound input module may be a sound pickup device such as a microphone. Clicking or sounding by the user according to the rhythm set by the note card; through the input and identification processing of click data or sound data of a user, the central processing unit compares the standard pronunciation data of the corresponding notes, gives deviation from the standard pronunciation, and outputs a result after processing.
A sixth embodiment, the structure of which is shown in fig. 13, wherein the present embodiment adds a locking slot of the third note card 15 on the basis of the first embodiment, and adds training learning of pitch on the basis of the original training rhythm; meanwhile, the top of the round tower-shaped sound production device 13 is provided with a camera 30 as a recognition device of the note card, the camera collects image information of the first note card 12, the second note card 14 and the third note card 15 on the rectangular bottom plate 11, corresponding beat numbers, note time values and tone Fu Yingao are obtained after recognition analysis processing, and after data processing, the central processing unit controls the indication block to move from left to right and simultaneously controls the sound production device to synchronously produce sound.
The seventh embodiment, the structure is as shown in fig. 14, is formed by two split parts of the note card reading device part and the rhythm visualization device part, which are connected in communication; wherein the note card reading device section, as shown in fig. 14 (a), is mainly composed of a communication module, a note card identification module, and a memory; the rhythm visualization device section, as shown in fig. 14 (B), is mainly composed of an indication module, a sound-producing module, a memory, and a central processing unit. Compared with the third embodiment, in the present embodiment, the note card reading device obtains the information corresponding to the card by multi-point touch, each card has a plurality of touch points at the bottom, different relative positional relationships between the touch points correspond to different codes, the different codes correspond to different card information contents, as shown in fig. 14 (a), the bottom plate on which the note card is placed is a multi-point touch screen, and the cards with different information can be placed on the screen to identify the corresponding codes according to the positional relative relationships of the contacts at the bottom of the card, so as to determine the beat or note information corresponding to the card; by placing the note card, the rhythm information corresponding to a plurality of circles of the visualization device can be set at one time, and each circle can correspond to different contents; after the note card reading device acquires the content corresponding to the card, related information is transmitted to the rhythm visualization device through the communication module. The rhythm visualization device can also directly receive the information such as the number-patting notes and the like input by the mobile phone or other terminals.
The rhythm visualization device part adopts a round bottom plate, a display screen is arranged on the bottom plate, and a sounding module and a central processing unit module are arranged below the bottom plate. The rhythm visualization device determines the displayed and played content through calculation and analysis, and displays and plays the content. In the use process, the rhythm visualization device displays the content corresponding to the first circle of rhythm, including related information such as a note card image, a flickering indication block, a track line and the like, and as shown in fig. 14 (B), a display screen on the rhythm visualization device displays corresponding note marks according to the proportion of note values, and the number of central angles occupied by the note marks is proportional to the correspondence of the note values. The indication module moves along the ring shape under the note mark by adopting a flashing icon, and the figure is a visual schematic diagram of the device for displaying the first circle of rhythm; after the first row of note cards are displayed, the display screen displays relevant visual information corresponding to the second row of note cards according to the note card content corresponding to the second circle of rhythms, as shown in fig. 14 (C), which is a visual schematic diagram of the device displaying the second circle of rhythms; according to the transferred information such as notes, the display screen can sequentially display the visual contents of a plurality of circles. It can be seen that, in this embodiment, the note card reading device may adopt a multi-touch scheme, and the note card reading device may set a plurality of rows of note cards and send identification information to the rhythm visualization device portion through communication, or may send information such as a note to be signed to the rhythm visualization device portion through a mobile phone or other terminals, so as to further satisfy the combination requirements of different music measures, and the rhythm visualization device may sequentially display the rhythm visualization processes of the plurality of music measures according to the reading sequence, corresponding to a plurality of circles of different visualized contents.
It should be noted that, although in the above embodiments 1-7, the indication module adopts a pointer or LED strip or display to directly display the digital image, other forms such as a cursor, a caliper, or a moving slider may also be adopted. The note card induction area can be provided with a clamping groove, a groove or a magnet, and the first note card, the second note card and the third note card induction area can be connected with the note card induction area through clamping, concave-convex matching or magnetic attraction. In the invention, the constant-speed continuous motion of the indication module is realized in the following two ways, but is not limited to, as shown in fig. 15, the time from A to B of the indication module is one beat; the indication module can continuously and uninterruptedly move from A to B at a constant speed; it is also possible to jump from the a position to the 1 position, then pause for a while, then to the 2 position, pause for a while, and so on, until the B position. In the present invention, the central processor may be, but is not limited to, a computing control unit such as CPU, MCU, MPU, DSP and necessary accessories.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (9)
1. A rhythmic singing visualization method, comprising the steps of:
s1, sequentially inputting a beat number and a plurality of notes, and presetting a beat time value and a reference loudness;
s2, sequentially acquiring a signature and a plurality of notes according to the sequence;
s3, determining the beat intensity according to the beat number and determining the rhythm intensity according to the note sequence; determining the intensity of each input note according to the beat intensity, the note rhythm intensity and the reference loudness, and calling the original audio corresponding to a single note according to a corresponding acoustic model to synthesize continuous and smooth audio;
s4, determining the position information of the corresponding time point indication module according to the number of the notes, the sequentially input beat time values; determining a track line for indicating the movement of the module according to the position information; the trajectory line corresponds to the duration of the sequentially input notes;
s5, the control indication module moves according to the track line, and the sound production module is controlled to synchronize the audio sound production.
2. The rhythmic singing visualization method according to claim 1, characterized by comprising the steps of:
s1, a user inputs a beat number and a plurality of notes in sequence at a terminal, and presets a beat time value and a reference loudness;
S2, the central processing unit sequentially acquires a signature and a plurality of notes according to the sequence;
s3, the central processing unit determines the sounding intensity of each note according to the number of beats, the position relation of a plurality of notes and the reference loudness, and synthesizes corresponding continuous smooth audio according to the sounding intensity law, the tone color, the beat time value and the acoustic model determined by the reference loudness information;
s4, the central processing unit determines the position information of the corresponding time point indication module according to the number of the beats, the plurality of notes sequentially input and the beat time value; determining a track line for indicating the movement of the module according to the position information; the trajectory line corresponds to the duration of the sequentially input notes;
s5, the central processing unit controls the indication module to continuously move at a constant speed according to the track line, and controls the sounding module to sound according to the audio frequency.
3. The rhythmic singing visualization method according to claim 1, characterized by comprising the steps of:
s1, a user sequentially splices a first note card with sign information and a plurality of second note cards with note information
The note card is used for placing the spliced first note card and second note card into a note card induction area, and presetting a beat time value and a base
A quasi-loudness;
s2, the note card induction area sequentially acquires a number and a plurality of notes according to the sequence, and the number and the notes are converted and transmitted to the central processing unit, and the central processing unit sequentially acquires the number and the notes according to the sequence;
s3, the central processing unit determines the pronunciation intensity of each note according to the sign, the notes and the reference loudness, and synthesizes corresponding audio;
s4, the central processing unit determines the position of the corresponding time point indicating module according to the number of the beats, the plurality of notes sequentially input and the beat time value
Information; determining a track line for indicating the movement of the module according to the position information; the trajectory line corresponds to the duration of the sequentially input notes;
s5, the central processing unit controls the indication module to continuously move at a constant speed according to the track line, and controls the sounding module to sound according to the audio frequency.
4. A rhythmic singing visualization method according to claim 3, characterized in that,
in S1, the first note card and the second note card have the same size;
the note card induction area is provided with a clamping groove, a groove or a magnet, and the first note card and the second note card are in clamping connection, concave-convex connection or magnetic attraction connection with the note card induction area;
the mode of acquiring the information of the first note card and the second note card by the note card sensing area is radio frequency chip sensing identification, image coding identification, code hole identification, image content identification or multi-touch identification.
5. A rhythmic singing visualization method according to claim 3, characterized in that,
in S1, a user sequentially splices a first note card with a sign information, a plurality of second note cards with note time values and a plurality of third note cards with note pitches, puts the spliced first note card, second note card and third note card into a note card induction area, and presets a beat time value and a reference loudness;
and in S3, synthesizing corresponding audio with pitch information according to the first, second and third note card information and the corresponding music rules.
6. A rhythmic singing visualization method according to claim 3, characterized in that,
the size of the second note card is correspondingly proportional to the note duration value on the second note card, and the width of each second note card corresponds to the horizontal projection length of the track line of the corresponding note; the note card induction areas are provided with a plurality of, and each note card induction area corresponds to different tone colors.
7. The rhythmic singing visualization method of any of claims 1-6, characterized in that,
s11, judging whether the inputted notes contain bars corresponding to the signs or not, and entering S2 if the notes contain bars; if not, returning to S1 and prompting error reporting.
8. A rhythmic singing visual system is characterized in that,
the terminal is used for inputting the mark and the note; the communication module is used for being respectively in communication connection with the terminal and the central processing unit;
a plurality of note cards, wherein the note cards are provided with a signature and notes;
the memory stores audio data corresponding to each note card or audio data corresponding to different sign and note combinations;
the note card identification module is provided with a note card induction zone and is used for identifying note cards placed in the note card induction zone;
the indication module is used for moving along the track line and representing the time domain position of the sounding note in real time;
the sounding module is used for playing audio;
the central processing unit determines the beat intensity according to the beat number and determines the rhythm intensity according to the note sequence; determining the intensity of each input note according to the beat intensity, the note rhythm intensity and the reference loudness, and calling the original audio corresponding to a single note according to a corresponding acoustic model to synthesize continuous and smooth audio;
determining position information of a corresponding time point indication module according to the number of the notes and the beat time values which are sequentially input;
Determining a track line for indicating the movement of the module according to the position information;
the trajectory line corresponds to the duration of the sequentially input notes;
and the control indication module continuously moves at a constant speed according to the track line, and the sounding module is controlled to sound according to the audio frequency.
9. A rhythmic singing visualization device is characterized in that,
the system of claim 8, further comprising a click module or a sound pickup module, wherein the click module records the click operation of the user, the sound pickup module records the sound production of the user, the central processing unit compares the click information of the user or the sound production information of the user with the sound production content of the corresponding notes, calculates deviation, and outputs the result after processing.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310754993.0A CN116631264A (en) | 2021-10-22 | 2021-10-22 | Method, system and device for assisting music teaching |
| CN202111230129.8A CN113963601B (en) | 2021-10-22 | 2021-10-22 | Rhythm singing visualization method, system and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111230129.8A CN113963601B (en) | 2021-10-22 | 2021-10-22 | Rhythm singing visualization method, system and device |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310754993.0A Division CN116631264A (en) | 2021-10-22 | 2021-10-22 | Method, system and device for assisting music teaching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113963601A CN113963601A (en) | 2022-01-21 |
| CN113963601B true CN113963601B (en) | 2023-08-01 |
Family
ID=79465965
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310754993.0A Pending CN116631264A (en) | 2021-10-22 | 2021-10-22 | Method, system and device for assisting music teaching |
| CN202111230129.8A Active CN113963601B (en) | 2021-10-22 | 2021-10-22 | Rhythm singing visualization method, system and device |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310754993.0A Pending CN116631264A (en) | 2021-10-22 | 2021-10-22 | Method, system and device for assisting music teaching |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN116631264A (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115662215B (en) * | 2022-12-09 | 2023-03-28 | 普赞加信息科技南京有限公司 | Intelligent music teaching method and system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101513690B1 (en) * | 2012-03-29 | 2015-04-21 | 주식회사 휴먼리그 | melody block toy for music education |
| CN105453168A (en) * | 2014-01-30 | 2016-03-30 | 施政 | System and method for learning,composing,and playing music with physical objects |
| CN106327949A (en) * | 2015-06-14 | 2017-01-11 | 刘英山 | Method and device for training music rhythm |
| CN106205279B (en) * | 2016-08-29 | 2020-10-27 | 得理电子(上海)有限公司 | Note duration prompting method and system and electronic equipment thereof |
| CN109272808B (en) * | 2018-11-13 | 2021-01-29 | 上海艺瓣文化传播有限公司 | Interactive system for music beat rhythm teaching and science popularization |
-
2021
- 2021-10-22 CN CN202310754993.0A patent/CN116631264A/en active Pending
- 2021-10-22 CN CN202111230129.8A patent/CN113963601B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN116631264A (en) | 2023-08-22 |
| CN113963601A (en) | 2022-01-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10964298B2 (en) | Network musical instrument | |
| CN106205280B (en) | Interactive dynamic color form and meaning music notation method and musical instrument playing trainer | |
| US7893337B2 (en) | System and method for learning music in a computer game | |
| US7271329B2 (en) | Computer-aided learning system employing a pitch tracking line | |
| US7390958B2 (en) | Method for teaching music | |
| CN106991995B (en) | Constant-name keyboard digital video-song musical instrument with stepless tone changing and key kneading and tone changing functions | |
| US20030167902A1 (en) | Machine and method for teaching music and piano | |
| CN108140369A (en) | Music score, system and method | |
| CN203311671U (en) | Musical instrument learning-assisting device with lamplight display function | |
| CN101719366A (en) | Method for editing and displaying musical notes and music marks and accompanying video system | |
| US5275082A (en) | Visual music conducting device | |
| CN109272808B (en) | Interactive system for music beat rhythm teaching and science popularization | |
| CN106981282A (en) | A kind of piano performance key reminding method and system | |
| JP2018146718A (en) | Training device, training program, and training method | |
| CN106952510B (en) | Pitch calibrator | |
| US20190325776A1 (en) | Lights-guided piano learning and teaching apparatus, and method | |
| JP7157377B2 (en) | Performance information display device, performance information display method, performance information display program, and electronic musical instrument | |
| CN104200801A (en) | Violin instrument | |
| Brown | Scratch music projects | |
| CN113963601B (en) | Rhythm singing visualization method, system and device | |
| CN2655363Y (en) | Digital music teaching device | |
| JP2015060207A (en) | Music data display device, music data presentation method, and program | |
| CN106205279B (en) | Note duration prompting method and system and electronic equipment thereof | |
| Weinberg | Interconnected musical networks: bringing expression and thoughtfulness to collaborative group playing | |
| CN204066715U (en) | Violin |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |