CN106601217A - Interactive-type musical instrument performing method and device - Google Patents
Interactive-type musical instrument performing method and device Download PDFInfo
- Publication number
- CN106601217A CN106601217A CN201611108803.4A CN201611108803A CN106601217A CN 106601217 A CN106601217 A CN 106601217A CN 201611108803 A CN201611108803 A CN 201611108803A CN 106601217 A CN106601217 A CN 106601217A
- Authority
- CN
- China
- Prior art keywords
- target
- pattern
- musical instrument
- action
- targeted customer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/0008—Associated control or indicating means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Electrophonic Musical Instruments (AREA)
Abstract
The embodiment of the invention provides an interactive-type musical instrument performing method and device, and the method comprises the steps: obtaining a preset number of pieces of first data of a target user in a preset time duration; determining the type of motions of the target user according to all first data, wherein the motions comprise static postures or dynamic postures; determining a target posture of the target user according to each piece of first data when the motion of the target user is static posture, and determining a target musical instrument type corresponding to the target posture according to the target posture and a preset musical instrument type template library; displaying a pattern corresponding to the target musical instrument type according to the target musical instrument type and a preset pattern library; determining the target posture of the target user according to each piece of first data when the motion of the target user is the dynamic posture, and outputting an audio corresponding to a current display pattern according to the target posture, the current display pattern, and a preset audio library. According to the embodiment of the invention, the method achieves the interactive-type musical instrument performing.
Description
Technical field
The present invention relates to human-computer interaction technique field, more particularly to a kind of interactive instrument playing method and device.
Background technology
Human-computer interaction technology referred to by computer input, outut device, realizes that people hands over computer in an efficient way
Mutual technology.With the development of human-computer interaction technology, the interface that people interacts with a computer constantly changes.From the life of most original
Row user interface (carrying out Text Command input by keyboard, display carries out text message output) is made to develop into graphical user circle
Face (by input through keyboard text, mouse control cursor, display shows figure), the innovation of user interface greatly improves user
With the interactive efficiency of computer.As the three-dimensional motions such as the design of the invention and computerized algorithm for updating hardware, depth camera are caught
The equipment of obtaining is widely used, and natural user interface also arises at the historic moment, and has obtained the development advanced by leaps and bounds.Naturally use
Family interface captures the athletic posture data of human body by three-dimensional motion capture device, and is sent to computer, and computer is to human body
Attitude data is analyzed process, such as the motion such as wave, jump of identifying user, the fortune for further identifying these
It is dynamic to be converted into order, so as to realize man-machine interaction.
Interactive instrument playing refers to by capturing human action, carries out motion analyses using computer and musical performance divides
Analysis, then carries out the process of pattern displaying and audio output.At present, natural user interface is captured using three-dimensional motion capture device
The athletic posture data of human body, lack using the athletic posture data of three-dimensional motion capture device capture human body to realize interactive mode
The technology of instrument playing.Accordingly, it would be desirable to a kind of capture human action to realize interactive instrument playing using 3 D captured equipment
Technology.
The content of the invention
The purpose of the embodiment of the present invention is to provide a kind of interactive instrument playing method and device, to realize interactive pleasure
Device is played.
To reach above-mentioned purpose, the embodiment of the invention discloses a kind of interactive instrument playing method, method includes：
Obtain first data of predetermined number of targeted customer in preset duration；
According to each first data, the action classification of the targeted customer is determined, wherein, the action classification includes static appearance
Gesture or dynamic action；
When the action classification of the targeted customer is static posture, according to each first data the targeted customer is determined
Target pose, and according to the target pose and default musical instrument species template base, determine that the target pose is corresponding
Target musical instrument species；According to the target musical instrument species and predetermined pattern storehouse, show corresponding with the target musical instrument species
Pattern；
When the action classification of the targeted customer is dynamic action, according to each first data the targeted customer is determined
Target action, and according to the target action, currently displaying pattern and preset audio storehouse, output and the currently displaying figure
The corresponding audio frequency of case.
Optionally, it is described to determine the action classification of the targeted customer according to each first data, including：
Judge whether each first data meet condition of similarity；
If it is, the action classification for determining the targeted customer is static posture；
If not, the action classification for determining the targeted customer is dynamic action.
Optionally, the corresponding relation of each musical instrument species and static posture is preserved in the musical instrument species template base, it is described
According to the target pose and default musical instrument species template base, the corresponding target musical instrument species of the target pose is determined
Step, including：
In the default musical instrument species template base, search and the target pose identical static posture；
According to the musical instrument species template base, it is determined that the corresponding musical instrument species of the static posture for being found, and this is found pleasure in
Device species is defined as the corresponding target musical instrument species of the target pose.
Optionally, it is described according to the target musical instrument species and predetermined pattern storehouse, show and the target musical instrument species
The step of corresponding pattern, including：
According to the target musical instrument species and predetermined pattern storehouse, it is determined that target figure corresponding with the target musical instrument species
Case；
Judge currently whether there is the pattern of display；
If currently without the pattern for showing, showing the target pattern；
If currently there is the pattern of display, judge whether the target pattern is consistent with currently displaying pattern；If it is,
Then keep the currently displaying pattern；If it is not, then showing the target pattern.
Optionally, it is described according to the target action, currently displaying pattern and preset audio storehouse, export current with described
The step of display pattern corresponding audio frequency, including：
According to the corresponding relation between default pattern and action, it is determined that target pattern corresponding with the target action；
Judge whether the target pattern is consistent with currently displaying pattern；
If it is, according to the preset audio storehouse, exporting audio frequency corresponding with the currently displaying pattern.
The embodiment of the invention also discloses a kind of interactive instrument playing device, described device includes：
Module is obtained, for obtaining first data of predetermined number of targeted customer in preset duration；
Determining module, for according to each first data, determining the action classification of the targeted customer, wherein, the action
Classification includes static posture or dynamic action；
Display module, for determine when the determining module targeted customer action classification be static posture when, according to every
Individual first data determine the target pose of the targeted customer, and according to the target pose and default musical instrument kind class template
Storehouse, determines the corresponding target musical instrument species of the target pose；According to the target musical instrument species and predetermined pattern storehouse, show
Pattern corresponding with the target musical instrument species；
Output module, for determine when the determining module targeted customer action classification be dynamic action when, root
Determine the target action of the targeted customer according to each first data, and according to the target action, currently displaying pattern and
Preset audio storehouse, exports audio frequency corresponding with the currently displaying pattern.
Optionally, the determining module, including：
First judging submodule, for judging whether each first data meet condition of similarity；
First determination sub-module, for when the first judging submodule judged result is to be, determining that the target is used
The action classification at family is static posture；
Second determination sub-module, for when the first judging submodule judged result is no, determining that the target is used
The action classification at family is dynamic action.
Optionally, the corresponding relation of each musical instrument species and static posture is preserved in the musical instrument species template base, it is described
Display module, including：
Submodule is searched, in the default musical instrument species template base, searching and the target pose identical
Static posture；
3rd determination sub-module, for according to the musical instrument species template base, it is determined that the static posture correspondence for being found
Musical instrument species, and the musical instrument species is defined as into the corresponding target musical instrument species of the target pose.
Optionally, the display module, including：
4th determination sub-module, for according to the target musical instrument species and predetermined pattern storehouse, it is determined that with the target
The corresponding target pattern of musical instrument species；
Second judging submodule, for judging currently whether there is display pattern；
First display sub-module, for when the second judging submodule judged result is no, showing the target figure
Case；
Second display sub-module, for when the second judging submodule judged result is to be, judging the target figure
Whether case is consistent with currently displaying pattern；If it is, keeping the currently displaying pattern；If not, showing the target
Pattern.
Optionally, the output module, including：
5th determination sub-module, for according to the corresponding relation between default pattern and action, it is determined that with the target
The corresponding target pattern of action；
3rd judging submodule, for judging whether the target pattern is consistent with currently displaying pattern；
Output sub-module, for when the 3rd judging submodule judged result is to be, according to the preset audio storehouse,
Output audio frequency corresponding with the currently displaying pattern.
As seen from the above technical solutions, interactive instrument playing method and device provided in an embodiment of the present invention, can be with
Obtain first data of predetermined number of targeted customer in preset duration；According to each first data, the action of targeted customer is determined
Classification, wherein, action classification includes static posture or dynamic action；When the action classification of targeted customer is static posture, root
Determine the target pose of targeted customer according to each first data, and according to target pose and default musical instrument species template base,
Determine the corresponding target musical instrument species of target pose；According to target musical instrument species and predetermined pattern storehouse, show and target musical instrument
The corresponding pattern of species；When the action classification of targeted customer is dynamic action, according to each first data targeted customer is determined
Target action, and according to target action, currently displaying pattern and preset audio storehouse, export corresponding with currently displaying pattern
Audio frequency, it is achieved thereby that interactive instrument playing.
Certainly, implementing arbitrary product or method of the present invention must be not necessarily required to while reaching all the above excellent
Point.
Description of the drawings
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is a kind of schematic flow sheet of interactive instrument playing method provided in an embodiment of the present invention；
Fig. 2 is a kind of structural representation of interactive instrument playing device provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than the embodiment of whole.It is based on
Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made
Embodiment, belongs to the scope of protection of the invention.
The embodiment of the invention discloses a kind of interactive instrument playing method and device, is described in detail individually below.
Referring to Fig. 1, Fig. 1 is a kind of schematic flow sheet of interactive instrument playing method provided in an embodiment of the present invention, bag
Include following steps：
S101, obtains first data of predetermined number of targeted customer in preset duration.
Method provided in an embodiment of the present invention can apply to electronic equipment, for example：Computer.The embodiment of the present invention is in terms of
Illustrate as a example by calculation machine, in actual applications, user can according to demand select the type of electronic equipment, the present invention to this not
Limit.
It should be noted that the embodiment of the present invention can utilize the three-dimensional motion capture device of external connection of computer in three-dimensional space
Between in the posture of targeted customer and motion are captured, so as to computer can obtain first data of targeted customer.Wherein,
Three-dimensional motion capture device can be depth camera (for example, Kinect), inertia motion capture device (for example, gyroscope, acceleration
Degree meter etc.), changing sensor (for example, data fabric) etc., the present invention is not construed as limiting to this.Computer sets with three-dimensional motion capture
It is standby to adopt wired connection mode or radio connection, the company between specific computer and three-dimensional motion capture device
Connect and belong to prior art, the present invention will not be described here.
First data can at least include three-dimensional position, three-dimensional motion speed, the three-dimensional motion acceleration of targeted customer's limbs
Degree etc..Specifically using three-dimensional motion capture device the posture and motion of targeted customer are captured in three dimensions, from
And the process of first data of acquisition targeted customer, belonging to prior art, the embodiment of the present invention will not be described here.
Exemplary, preset duration can be 3 seconds, and three-dimensional motion capture device can be gathering 30 first numbers each second
According to 90 the first data of targeted customer then can be obtained in preset duration.
S102, according to each first data, determines the action classification of targeted customer.
Specifically, action classification includes static posture or dynamic action.Computer determines that target is used according to each first data
The action classification at family, Ke Yiwei：Judge whether each first data meet condition of similarity；If it is, determining the action of targeted customer
Classification is static posture；If not, the action classification for determining targeted customer is dynamic action.
It should be noted that computer judges the step of whether each first data meet condition of similarity, Ke Yiwei：When each
When one data are identical, then judge that each first data meet condition of similarity；Other are different from when there are at least one first data extremely
During few first data, then judge that each first data are unsatisfactory for condition of similarity.
When the first data include multinomial content, the such as three-dimensional position of targeted customer's limbs, three-dimensional motion speed, three maintenance and operations
Dynamic acceleration etc., can successively be compared the corresponding content of each first data, to determine the action classification of targeted customer.
It is exemplary, when the first data for obtaining are 2, the three-dimensional position of first the first data, three-dimensional motion speed,
Three-dimensional motion acceleration is respectively：21cm、5cm、7cm、0cm/s、0cm/s、0cm/s、0m/s2、0m/s2、0m/s2, second
The three-dimensional position of one data, three-dimensional motion speed, three-dimensional motion acceleration are respectively：21cm、5cm、7cm、0cm/s、0cm/s、
0cm/s、0m/s2、0m/s2、0m/s2, it can be determined that go out three-dimensional position, three-dimensional motion speed, three maintenance and operations of first the first data
Dynamic acceleration three-dimensional position respectively with second the first data, three-dimensional motion speed, three-dimensional motion acceleration are corresponding identical, then
Judge that each first data meet condition of similarity, the action classification for determining targeted customer is static posture.
Exemplary, the first data of acquisition have 2, the three-dimensional position of first the first data, three-dimensional motion speed,
Three-dimensional motion acceleration is respectively：21cm、5cm、7cm、0cm/s、0cm/s、0cm/s、0m/s2、0m/s2、0m/s2, second
The three-dimensional position of one data, three-dimensional motion speed, three-dimensional motion acceleration are respectively：21cm、5cm、7cm、0cm/s、15cm/s、
2cm/s、0m/s2、2m/s2、1m/s2, it can be determined that go out the three-dimensional motion speed and second the first data of first the first data
Three-dimensional motion speed it is different, three-dimensional motion acceleration and the three-dimensional motion of second the first data of first the first data add
Speed is different, then judge that each first data are unsatisfactory for condition of similarity, and the action classification for determining targeted customer is dynamic action.
Computer judges the step of whether each first data meet condition of similarity, can also be：Obtained according to each first data
The skeleton point change in location value of targeted customer is obtained, if skeleton point change in location value is not above the first predetermined threshold value, is judged
Each first data meet condition of similarity；If skeleton point change in location value is more than the first predetermined threshold value, each first data are judged
It is unsatisfactory for condition of similarity.
Exemplary, when the first predetermined threshold value is 2cm, if skeleton point change in location is 1cm, judge each first number
According to condition of similarity is met, the action classification for determining targeted customer is static posture；If skeleton point change in location is 5cm, sentence
Fixed each first data are unsatisfactory for condition of similarity, and the action classification for determining targeted customer is dynamic action.
Similar, computer judges the step of whether each first data meet condition of similarity, can also be：According to each first
Data obtain the skeleton point velocity variations value of targeted customer, if skeleton point velocity variations value is not above the second predetermined threshold value,
Then judge that each first data meet condition of similarity；If skeleton point velocity variations value is more than the second predetermined threshold value, each is judged
One data are unsatisfactory for condition of similarity.Or, the skeleton point acceleration change value of targeted customer is obtained according to each first data, if
Skeleton point acceleration change value is not above the 3rd predetermined threshold value, then judge that each first data meet condition of similarity；If skeleton
Point acceleration change value then judges that each first data are unsatisfactory for condition of similarity more than the 3rd predetermined threshold value.
Exemplary, when the second predetermined threshold value is 0.5cm/s, if skeleton point velocity variations are 0cm/s, judge each
First data meet condition of similarity, and the action classification for determining targeted customer is static posture；If skeleton point change in location is
5cm/s, then judge that each first data are unsatisfactory for condition of similarity, and the action classification for determining targeted customer is dynamic action.
Wherein, the size of the first predetermined threshold value, the second predetermined threshold value and the 3rd predetermined threshold value can be according to user's request
Setting, the present invention is not limited this.
S103, when the action classification of targeted customer is static posture, determines targeted customer's according to each first data
Target pose, and according to target pose and default musical instrument species template base, determine the corresponding target musical instrument kind of target pose
Class；According to target musical instrument species and predetermined pattern storehouse, pattern corresponding with target musical instrument species is shown.
It should be noted that computer can be using signal processing, machine learning, location recognition, change in location tracking etc.
Each first data is analyzed process by mode, so that it is determined that the target pose of targeted customer.In actual applications, it is determined that
The mode of the target pose of targeted customer can set according to user's request, and the present invention is not construed as limiting to this, specifically determines mesh
The method of the target pose of mark user is prior art, and the present invention will not be described here.
In embodiments of the present invention, each musical instrument species can be preserved in musical instrument species template base corresponding with static posture
Relation.For example, each musical instrument species for preserving in musical instrument species template base can be as shown in table 1 with the corresponding relation of static posture：
Table 1
| Musical instrument species | Static posture |
| Drum | Put one's palms together devoutly and be put in front |
| Violin | Put one's palms together devoutly and lift in the crown |
| Trombone | Both hands are separately lifted in the crown |
Computer determines the corresponding target musical instrument of target pose according to target pose and default musical instrument species template base
The step of species, Ke Yiwei：In default musical instrument species template base, search and target pose identical static posture；According to
Musical instrument species template base, it is determined that the corresponding musical instrument species of the static posture for being found, and the musical instrument species is defined as into target
The corresponding target musical instrument species of posture.
Exemplary, using machine learning method, process is analyzed to each first data, so that it is determined that targeted customer
Target pose lift in the crown to put one's palms together devoutly, the posture correspondence lifted in the crown of putting one's palms together devoutly in default musical instrument species template base
Musical instrument species be violin, it is determined that target musical instrument species be violin.
Specifically, computer shows corresponding with target musical instrument species according to target musical instrument species and predetermined pattern storehouse
The step of pattern, Ke Yiwei：According to target musical instrument species and predetermined pattern storehouse, it is determined that target corresponding with target musical instrument species
Pattern；Judge currently whether there is the pattern of display；If currently without the pattern for showing, showing the target pattern；If worked as
Before have the pattern of display, judge whether the target pattern consistent with currently displaying pattern；If it is, keeping described current
The pattern of display；If it is not, then showing the target pattern.
It should be noted that the output of pattern is completed by images outputting framework.Graphical interfaces can export cartoon animals figure
Case and cartoon scene, can represent a kind of musical instrument species, for example with each cartoon animals：Lion correspondence is roused；Fox correspondence is little
Violin；Rabbit correspondence trombone.By the musical instrument species that pre-builds and the one-to-one relationship of cartoon animals pattern, show and mesh
The corresponding target pattern of mark musical instrument species.In the case of the pattern being not currently displayed, target pattern is directly displayed；Current
In the case of having the pattern of display, then need to judge whether target pattern consistent with currently displaying pattern, when target pattern with
When currently displaying pattern is consistent, images outputting framework is sent a signal to, keep currently displaying pattern；When target pattern with work as
When the pattern of front display is inconsistent, images outputting framework is sent a signal to, currently displaying pattern is updated, with display target figure
Case.
Exemplary, when target musical instrument species is violin, pattern corresponding with violin is fox in predetermined pattern storehouse
When, if the pattern being not currently displayed, show fox pattern；If currently displaying pattern is fox, keep showing
Fox pattern；If currently displaying pattern is tiger, fox pattern is shown.
S104, when the action classification of targeted customer is dynamic action, determines targeted customer's according to each first data
Target action, and according to target action, currently displaying pattern and preset audio storehouse, export sound corresponding with currently displaying pattern
Frequently.
It should be noted that computer can be right in modes such as Land use models identification, machine learning, finite-state automatas
Each first data is analyzed process, so that it is determined that the target action of targeted customer.In actual applications, targeted customer is determined
The mode of the corresponding target musical instrument species of target action can be set according to user's request, the present invention is not construed as limiting to this.Tool
The set the goal really method of target action of user of body belongs to prior art, and the present invention will not be described here.
Exemplary, Land use models identification is analyzed process, so that it is determined that the mesh of targeted customer to each first data
Mark action is：One downward action of hand.
Specifically, computer is exported and currently displaying figure according to target action, currently displaying pattern and preset audio storehouse
The step of case corresponding audio frequency, Ke Yiwei：According to the corresponding relation between default pattern and action, it is determined that with target action pair
The target pattern answered；Judge whether target pattern is consistent with currently displaying pattern；If it is, according to preset audio storehouse, output
Audio frequency corresponding with currently displaying pattern.
It should be noted that when targeted customer plays different musical instruments, its action is different.In the embodiment of the present invention
In, the action data of various musical instruments can be gathered in advance, and be marked, formation can be used for the action template base of musical instrument identification,
The corresponding relation of each musical instrument species and action can be preserved in action template base.For example, each pleasure for preserving in action template base
Device species can be as shown in table 2 with the corresponding relation of action：
Table 2
| Musical instrument species | Action |
| Drum | Hand is moved downward |
| Violin | The side-to-side movement of hand |
| Trombone | Hand is moved forward and backward |
Exemplary, the action template base of the embodiment of the present invention is saved and correspond to respectively three kinds of drum, violin and trombone
The action of musical instrument, is described as follows：
The action of correspondence drum：In practical application, the action for tapping drum is mainly completed by moving downward for hand, therefore, inspection
Survey is based on the data of targeted customer's hand.Target action needs to meet two criterion：
First, molar behavior judges, judges hand whether moving downward, and sets an action duration interval, it is stipulated that
The action that hand completes to move from top to bottom in this interval is only standard compliant；Interval lower bound is that ordinary people most can complete soon
The time of corresponding hammer action, for example, can be 5 milliseconds, 8 milliseconds etc., and the upper bound is that the ordinary people of setting most completes slowly hammer action
Time, for example can be 1 second, 1.5 seconds etc..
Often receive first data, storage and as the data of benchmark first, before finding out successively with the benchmark the
Each first data of the time difference of one data in action duration interval, the order of duration length is short to successively to each according to duration
Individual first data are traveled through, and whether the data of judgment standard first are more than molar behavior with the position difference of the first data of traversal
Threshold value, more than threshold value action triggers are then illustrated.And the data of benchmark first are shorter with the duration of the first data break of traversal, can
To think that it is faster that targeted customer taps bulging speed of action, it is possible to which, by the audio frequency of this feedback of the information to output, action is fast
Degree is faster, and the sound for exporting audio frequency is bigger.
2nd, ending action judgement, after molar behavior judgement is met, judges the move distance of the data of newest two first
Whether it is more than ending action threshold value, so as to ensure that this action triggers is the first time triggering of one-off, rather than taps it
After be parked in original place.This judgement can effectively prevent the false touch that a beat triggers multiple action.Ending action threshold value can
To be set according to user's request, here of the present invention is not limited.
The action of correspondence violin：In practical application, the action for pulling violin is mainly completed by the side-to-side movement of hand,
Therefore, detection is based on the data of targeted customer's hand.The embodiment of the present invention is right so that targeted customer draws qin using the right hand as an example
Handss state in which builds a finite-state automata, and meeting automat after corresponding statess carries out state transfer, is simultaneously emitted by
Control signal (note start/note terminate).
Five kinds of states are defined as follows：The S1- right hands right side preparation area prepare start, the S2- right hands from the right side toward left movement,
The S3- right hands prepare to start in left side preparation area, and during the S4- right hands are moved from left to right, the S5- right hands are in other regions.State
That what is shifted it is critical only that：1) only S2 states could be entered in S1 states, into after S2 states, sends note commencing signal,
Equally, only S4 states could be entered in S3 states, into after S4 states, sends note commencing signal.In other words, only exist
Right side prepares region, could trigger to Zola's qin event, and this processing method can filter out the mistake produced after release
Triggering；2) under S2 states, it is necessary to which persistent movement can just be maintained at S2 states, equally, under S4 states, it is necessary to which persistent movement is
S4 states can be maintained at, be otherwise considered as event and be terminated, send note end signal.S1, S3, S5 state is arranged side by side three shape
State, three regions in difference corresponding flat, automat can be according in the first data for obtaining in the case of not trigger action
The position data of the right hand, the automatic conversion between tri- states of S1, S3 and S5.
The action of correspondence trombone：In practical application, the action for pulling trombone is mainly completed by moving forward and backward for hand, is similar to
In the action of the side-to-side movement of violin, therefore, finite-state automata is equally taken, simply the corresponding area of S1, S3, S5 state
Domain by depth data X-Y plane data conversion into X-Z plane data, concrete grammar referring to violin part description,
This is repeated no more.
Computer is according to action template base, it may be determined that the corresponding target musical instrument species of target action, according to target musical instrument
Species can determine target pattern corresponding with target action.So as to judge whether are the target pattern and currently displaying pattern
Unanimously, and will determine that result sends a signal to audio output framework, so that audio output framework determines whether to export audio frequency.When this
When target pattern is consistent with currently displaying pattern, audio output framework is sent a signal to, exported corresponding with currently displaying pattern
Audio frequency；When the target pattern is inconsistent with currently displaying pattern, audio output framework is sent a signal to, do not export sound
Frequently.
Wherein, audio output framework can use MIDI (the Musical Instrument Digital of Windows systems
Interface, musical instrument digital interface) framework.The specific method for carrying out audio output using MIDI frameworks belongs to prior art,
The embodiment of the present invention will not be described here.
Exemplary, according to above-mentioned action template base, the corresponding musical instrument species of target action for determining targeted customer is little
Violin, currently displaying pattern is fox, and the corresponding action of fox pattern is the action for playing the violin, it is thus determined that target is dynamic
It is fox to make corresponding target pattern, consistent with currently displaying pattern, so as to export violin sound corresponding with fox pattern
Frequently.
It can be seen that, using interactive instrument playing method provided in an embodiment of the present invention, realize interactive instrument playing.
Corresponding with above-mentioned embodiment of the method, the embodiment of the present invention also provides a kind of interactive instrument playing device.
Referring to Fig. 2, the structural representation of a kind of interactive instrument playing device that Fig. 2 is provided by the embodiment of the present invention,
Including：
Module 201 is obtained, for obtaining first data of predetermined number of targeted customer in preset duration；
Determining module 202, for according to each first data, determining the action classification of the targeted customer, wherein, it is described dynamic
Make classification including static posture or dynamic action；
Display module 203, for determine when the determining module targeted customer action classification be static posture when, according to
Each the first data determines the target pose of the targeted customer, and according to the target pose and default musical instrument species mould
Plate storehouse, determines the corresponding target musical instrument species of the target pose；According to the target musical instrument species and predetermined pattern storehouse, show
Show pattern corresponding with the target musical instrument species；
Output module 204, for determine when the determining module targeted customer action classification be dynamic action when,
Determine the target action of the targeted customer according to each first data, and according to the target action, currently displaying pattern with
And preset audio storehouse, export audio frequency corresponding with the currently displaying pattern.
Wherein it is determined that module 202, including：
First judging submodule (not shown), for judging whether each first data meet condition of similarity；
First determination sub-module (not shown), for when the first judging submodule judged result for be when, really
The action classification of the fixed targeted customer is static posture；
Second determination sub-module (not shown), for when the first judging submodule judged result for it is no when, really
The action classification of the fixed targeted customer is dynamic action.
Wherein, the corresponding relation of each musical instrument species and static posture is preserved in the musical instrument species template base, it is described aobvious
Show module 203, can include：
Submodule (not shown) is searched, in the default musical instrument species template base, searching and the mesh
Mark posture identical static posture；
3rd determination sub-module (not shown), for according to the musical instrument species template base, it is determined that found
The corresponding musical instrument species of static posture, and the musical instrument species is defined as into the corresponding target musical instrument species of the target pose.
Wherein, display module 203, can also include：
4th determination sub-module (not shown), for according to the target musical instrument species and predetermined pattern storehouse, really
Fixed target pattern corresponding with the target musical instrument species；
Second judging submodule (not shown), for the pattern for judging currently whether to have display；
First display sub-module (not shown), for when the second judging submodule judged result is no, showing
Show the target pattern；
Second display sub-module (not shown), for when the second judging submodule judged result is to be, sentencing
Whether the target pattern that breaks is consistent with currently displaying pattern；If it is, keeping the currently displaying pattern；If not,
Show the target pattern.
Wherein, output module 204, including：
5th determination sub-module (not shown), for according to the corresponding relation between default pattern and action, really
Fixed target pattern corresponding with the target action；
3rd judging submodule (not shown), for judging the target pattern and currently displaying pattern whether
Cause；
Output sub-module (not shown), for when the 3rd judging submodule judged result is to be, according to institute
Preset audio storehouse is stated, audio frequency corresponding with the currently displaying pattern is exported.
It can be seen that, using interactive instrument playing device provided in an embodiment of the present invention, realize interactive instrument playing.
For device embodiment, because it is substantially similar to embodiment of the method, so description is fairly simple, it is related
Part is illustrated referring to the part of embodiment of the method.
It should be noted that herein, such as first and second or the like relational terms are used merely to a reality
Body or operation make a distinction with another entity or operation, and not necessarily require or imply these entities or deposit between operating
In any this actual relation or order.And, term " including ", "comprising" or its any other variant are intended to
Nonexcludability is included, so that a series of process, method, article or equipment including key elements not only will including those
Element, but also including other key elements being not expressly set out, or also include for this process, method, article or equipment
Intrinsic key element.In the absence of more restrictions, the key element for being limited by sentence "including a ...", it is not excluded that
Also there is other identical element in process, method, article or equipment including the key element.
Each embodiment in this specification is described by the way of correlation, identical similar portion between each embodiment
Divide mutually referring to what each embodiment was stressed is the difference with other embodiment.Especially for system reality
For applying example, because it is substantially similar to embodiment of the method, so description is fairly simple, related part is referring to embodiment of the method
Part explanation.
Presently preferred embodiments of the present invention is the foregoing is only, protection scope of the present invention is not intended to limit.It is all
Any modification, equivalent substitution and improvements made within the spirit and principles in the present invention etc., are all contained in protection scope of the present invention
It is interior.
Claims (10)
1. a kind of interactive instrument playing method, it is characterised in that methods described includes：
Obtain first data of predetermined number of targeted customer in preset duration；
According to each first data, determine the action classification of the targeted customer, wherein, the action classification include static posture or
Dynamic action；
When the action classification of the targeted customer is static posture, according to each first data the mesh of the targeted customer is determined
Mark posture, and according to the target pose and default musical instrument species template base, determine the corresponding target of the target pose
Musical instrument species；According to the target musical instrument species and predetermined pattern storehouse, pattern corresponding with the target musical instrument species is shown；
When the action classification of the targeted customer is dynamic action, according to each first data the mesh of the targeted customer is determined
Mark action, and according to the target action, currently displaying pattern and preset audio storehouse, export and the currently displaying pattern pair
The audio frequency answered.
2. method according to claim 1, it is characterised in that described according to each first data, determines the targeted customer
Action classification, including：
Judge whether each first data meet condition of similarity；
If it is, the action classification for determining the targeted customer is static posture；
If not, the action classification for determining the targeted customer is dynamic action.
3. method according to claim 1, it is characterised in that each musical instrument species is preserved in the musical instrument species template base
With the corresponding relation of static posture, it is described according to the target pose and default musical instrument species template base, determine the mesh
The step of mark posture corresponding target musical instrument species, including：
In the default musical instrument species template base, search and the target pose identical static posture；
According to the musical instrument species template base, it is determined that the corresponding musical instrument species of the static posture for being found, and by the musical instrument kind
Class is defined as the corresponding target musical instrument species of the target pose.
4. method according to claim 1, it is characterised in that described according to the target musical instrument species and predetermined pattern
The step of storehouse, display pattern corresponding with the target musical instrument species, including：
According to the target musical instrument species and predetermined pattern storehouse, it is determined that target pattern corresponding with the target musical instrument species；
Judge currently whether there is the pattern of display；
If currently without the pattern for showing, showing the target pattern；
If currently there is the pattern of display, judge whether the target pattern is consistent with currently displaying pattern；If it is, protecting
Hold the currently displaying pattern；If it is not, then showing the target pattern.
5. method according to claim 1, it is characterised in that it is described according to the target action, currently displaying pattern with
And preset audio storehouse, the step of export audio frequency corresponding with the currently displaying pattern, including：
According to the corresponding relation between default pattern and action, it is determined that target pattern corresponding with the target action；
Judge whether the target pattern is consistent with currently displaying pattern；
If it is, according to the preset audio storehouse, exporting audio frequency corresponding with the currently displaying pattern.
6. a kind of interactive instrument playing device, it is characterised in that described device includes：
Module is obtained, for obtaining first data of predetermined number of targeted customer in preset duration；
Determining module, for according to each first data, determining the action classification of the targeted customer, wherein, the action classification
Including static posture or dynamic action；
When display module for determining the action classification of targeted customer when the determining module is static posture, according to each the
One data determine the target pose of the targeted customer, and according to the target pose and default musical instrument species template base,
Determine the corresponding target musical instrument species of the target pose；According to the target musical instrument species and predetermined pattern storehouse, show with
The corresponding pattern of the target musical instrument species；
Output module, for determine when the determining module targeted customer action classification be dynamic action when, according to every
Individual first data determine the target action of the targeted customer, and according to the target action, currently displaying pattern and preset
Audio repository, exports audio frequency corresponding with the currently displaying pattern.
7. device according to claim 6, it is characterised in that the determining module, including：
First judging submodule, for judging whether each first data meet condition of similarity；
First determination sub-module, for when the first judging submodule judged result is to be, determining the targeted customer's
Action classification is static posture；
Second determination sub-module, for when the first judging submodule judged result is no, determining the targeted customer's
Action classification is dynamic action.
8. device according to claim 6, it is characterised in that each musical instrument species is preserved in the musical instrument species template base
With the corresponding relation of static posture, the display module, including：
Submodule is searched, it is static with the target pose identical in the default musical instrument species template base, searching
Posture；
3rd determination sub-module, for according to the musical instrument species template base, it is determined that the corresponding pleasure of the static posture for being found
Device species, and the musical instrument species is defined as into the corresponding target musical instrument species of the target pose.
9. device according to claim 6, it is characterised in that the display module, including：
4th determination sub-module, for according to the target musical instrument species and predetermined pattern storehouse, it is determined that with the target musical instrument
The corresponding target pattern of species；
Second judging submodule, for judging currently whether there is display pattern；
First display sub-module, for when the second judging submodule judged result is no, showing the target pattern；
Second display sub-module, for when the second judging submodule judged result for be when, judge the target pattern and
Whether currently displaying pattern is consistent；If it is, keeping the currently displaying pattern；If not, showing the target pattern.
10. device according to claim 6, it is characterised in that the output module, including：
5th determination sub-module, for according to the corresponding relation between default pattern and action, it is determined that with the target action
Corresponding target pattern；
3rd judging submodule, for judging whether the target pattern is consistent with currently displaying pattern；
Output sub-module, for when the 3rd judging submodule judged result is to be, according to the preset audio storehouse, output
Audio frequency corresponding with the currently displaying pattern.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611108803.4A CN106601217B (en) | 2016-12-06 | 2016-12-06 | Interactive musical instrument playing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611108803.4A CN106601217B (en) | 2016-12-06 | 2016-12-06 | Interactive musical instrument playing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106601217A true CN106601217A (en) | 2017-04-26 |
| CN106601217B CN106601217B (en) | 2021-03-02 |
Family
ID=58596507
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611108803.4A Active CN106601217B (en) | 2016-12-06 | 2016-12-06 | Interactive musical instrument playing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106601217B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111103983A (en) * | 2019-12-31 | 2020-05-05 | 维沃移动通信有限公司 | AR musical instrument playing method and electronic equipment |
| CN111199455A (en) * | 2019-12-19 | 2020-05-26 | 南京酷派软件技术有限公司 | Method, apparatus, electronic device and medium for selecting musical instrument |
| CN112752149A (en) * | 2020-12-29 | 2021-05-04 | 广州繁星互娱信息科技有限公司 | Live broadcast method, device, terminal and storage medium |
| CN114115528A (en) * | 2021-11-02 | 2022-03-01 | 深圳市雷鸟网络传媒有限公司 | Virtual object control method and device, computer equipment and storage medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020036617A1 (en) * | 1998-08-21 | 2002-03-28 | Timothy R. Pryor | Novel man machine interfaces and applications |
| US20040046736A1 (en) * | 1997-08-22 | 2004-03-11 | Pryor Timothy R. | Novel man machine interfaces and applications |
| CN101504832A (en) * | 2009-03-24 | 2009-08-12 | 北京理工大学 | Virtual performance system based on hand motion sensing |
| CN102117117A (en) * | 2010-01-06 | 2011-07-06 | 致伸科技股份有限公司 | System and method for control through identifying user posture by image extraction device |
| CN102478959A (en) * | 2010-11-28 | 2012-05-30 | 蒋霞 | Control system and method for electronic device |
| CN102609093A (en) * | 2012-02-16 | 2012-07-25 | 中国农业大学 | Method and device for controlling video playing by using gestures |
| CN102915112A (en) * | 2011-06-23 | 2013-02-06 | 奥美可互动有限责任公司 | System and method for close-range movement tracking |
| CN102947777A (en) * | 2010-06-22 | 2013-02-27 | 微软公司 | User tracking feedback |
| CN103235641A (en) * | 2013-03-17 | 2013-08-07 | 浙江大学 | 6-dimensional sensory-interactive virtual keyboard instrument system and realization method thereof |
| CN103986825A (en) * | 2014-05-09 | 2014-08-13 | 苏州乐聚一堂电子科技有限公司 | Image special effect system of mobile personalized karaoke |
| CN104007844A (en) * | 2014-06-18 | 2014-08-27 | 原硕朋 | Electronic instrument and wearable type input device for same |
| CN105915987A (en) * | 2016-04-15 | 2016-08-31 | 济南大学 | Implicit interaction method facing smart television set |
| CN205680441U (en) * | 2016-06-06 | 2016-11-09 | 北京千音互联科技有限公司 | A kind of virtual performance system |
| CN106157363A (en) * | 2016-06-28 | 2016-11-23 | 广东欧珀移动通信有限公司 | A kind of photographic method based on augmented reality, device and mobile terminal |
-
2016
- 2016-12-06 CN CN201611108803.4A patent/CN106601217B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040046736A1 (en) * | 1997-08-22 | 2004-03-11 | Pryor Timothy R. | Novel man machine interfaces and applications |
| US20120040755A1 (en) * | 1997-08-22 | 2012-02-16 | Motion Games, Llc | Interactive video based games using objects sensed by tv cameras |
| US20020036617A1 (en) * | 1998-08-21 | 2002-03-28 | Timothy R. Pryor | Novel man machine interfaces and applications |
| CN101504832A (en) * | 2009-03-24 | 2009-08-12 | 北京理工大学 | Virtual performance system based on hand motion sensing |
| CN102117117A (en) * | 2010-01-06 | 2011-07-06 | 致伸科技股份有限公司 | System and method for control through identifying user posture by image extraction device |
| CN102947777A (en) * | 2010-06-22 | 2013-02-27 | 微软公司 | User tracking feedback |
| CN102478959A (en) * | 2010-11-28 | 2012-05-30 | 蒋霞 | Control system and method for electronic device |
| CN102915112A (en) * | 2011-06-23 | 2013-02-06 | 奥美可互动有限责任公司 | System and method for close-range movement tracking |
| CN102609093A (en) * | 2012-02-16 | 2012-07-25 | 中国农业大学 | Method and device for controlling video playing by using gestures |
| CN103235641A (en) * | 2013-03-17 | 2013-08-07 | 浙江大学 | 6-dimensional sensory-interactive virtual keyboard instrument system and realization method thereof |
| CN103986825A (en) * | 2014-05-09 | 2014-08-13 | 苏州乐聚一堂电子科技有限公司 | Image special effect system of mobile personalized karaoke |
| CN104007844A (en) * | 2014-06-18 | 2014-08-27 | 原硕朋 | Electronic instrument and wearable type input device for same |
| CN105915987A (en) * | 2016-04-15 | 2016-08-31 | 济南大学 | Implicit interaction method facing smart television set |
| CN205680441U (en) * | 2016-06-06 | 2016-11-09 | 北京千音互联科技有限公司 | A kind of virtual performance system |
| CN106157363A (en) * | 2016-06-28 | 2016-11-23 | 广东欧珀移动通信有限公司 | A kind of photographic method based on augmented reality, device and mobile terminal |
Non-Patent Citations (3)
| Title |
|---|
| HONGMO JE ET AL.: ""Hand Gesture Recognition To Understand Musical Conducting Action"", 《16TH IEEE INTERNATIONAL CONFERENCE ON ROBOT & HUMAN INTERACTIVE COMMUNICATION》 * |
| MINORU FUJIMOTO ET AL.: ""A Motion Recognition Method for a Wearable Dancing Musical Instrument"", 《2009 INTERNATIONAL SYMPOSIUM ON WEARABLE COMPUTERS》 * |
| 童号等: ""交互式乐器演奏的六自由度力觉渲染方法"", 《中国图象图形学报》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111199455A (en) * | 2019-12-19 | 2020-05-26 | 南京酷派软件技术有限公司 | Method, apparatus, electronic device and medium for selecting musical instrument |
| CN111103983A (en) * | 2019-12-31 | 2020-05-05 | 维沃移动通信有限公司 | AR musical instrument playing method and electronic equipment |
| CN112752149A (en) * | 2020-12-29 | 2021-05-04 | 广州繁星互娱信息科技有限公司 | Live broadcast method, device, terminal and storage medium |
| CN114115528A (en) * | 2021-11-02 | 2022-03-01 | 深圳市雷鸟网络传媒有限公司 | Virtual object control method and device, computer equipment and storage medium |
| CN114115528B (en) * | 2021-11-02 | 2024-01-19 | 深圳市雷鸟网络传媒有限公司 | Virtual object control method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106601217B (en) | 2021-03-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10074186B2 (en) | Image search system, image search apparatus, and image search method | |
| US9690982B2 (en) | Identifying gestures or movements using a feature matrix that was compressed/collapsed using principal joint variable analysis and thresholds | |
| CN103809733B (en) | Man-machine interactive system and method | |
| EP2391972B1 (en) | System and method for object recognition and tracking in a video stream | |
| CN101142617B (en) | Method and apparatus for data entry input | |
| EP2940557A1 (en) | Method and device used for providing input candidate item corresponding to input character string | |
| US20140132515A1 (en) | System and method for inputting user commands to a processor | |
| CN106601217A (en) | Interactive-type musical instrument performing method and device | |
| US20130044053A1 (en) | Combining Explicit Select Gestures And Timeclick In A Non-Tactile Three Dimensional User Interface | |
| CN110517685A (en) | Audio recognition method, device, electronic equipment and storage medium | |
| WO2015103693A1 (en) | Systems and methods of monitoring activities at a gaming venue | |
| CN104090652A (en) | Voice input method and device | |
| CN105556438A (en) | Systems and methods for providing response to user input using information about state changes predicting future user input | |
| CN110888532A (en) | Man-machine interaction method and device, mobile terminal and computer readable storage medium | |
| CN111259751A (en) | Video-based human behavior recognition method, device, equipment and storage medium | |
| CN103823554A (en) | Digital virtual-real interaction system and digital virtual-real interaction method | |
| CN110534109A (en) | Audio recognition method, device, electronic equipment and storage medium | |
| CN103777748A (en) | Motion sensing input method and device | |
| EP3379482A1 (en) | Information processing device calculating statistical information | |
| CN111491123A (en) | Video background processing method and device and electronic equipment | |
| CN105138949A (en) | Gesture control method based on flexible mapping between multiple gestures and semantics | |
| CN103336967A (en) | Hand motion trail detection method and apparatus | |
| CN111103982A (en) | Data processing method, device and system based on somatosensory interaction | |
| CN108073267A (en) | Three dimensions control method and device based on movement locus | |
| CN106951258A (en) | The operating method and mobile terminal of a kind of mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |