CN109166565A - Virtual musical instrument processing method, device, virtual musical instrument equipment and storage medium - Google Patents
Virtual musical instrument processing method, device, virtual musical instrument equipment and storage medium Download PDFInfo
- Publication number
- CN109166565A CN109166565A CN201810965724.8A CN201810965724A CN109166565A CN 109166565 A CN109166565 A CN 109166565A CN 201810965724 A CN201810965724 A CN 201810965724A CN 109166565 A CN109166565 A CN 109166565A
- Authority
- CN
- China
- Prior art keywords
- musical instrument
- virtual
- mark
- finger
- key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/0008—Associated control or indicating means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/32—Constructional details
- G10H1/34—Switch arrangements, e.g. keyboards or mechanical switches specially adapted for electrophonic musical instruments
- G10H1/344—Structural association with individual keys
- G10H1/346—Keys with an arrangement for simulating the feeling of a piano key, e.g. using counterweights, springs, cams
Abstract
The embodiment of the present invention provides a kind of virtual musical instrument processing method, device, virtual musical instrument equipment and storage medium.The method of the embodiment of the present invention, by obtaining gesture information of each finger of user relative to the virtual musical instrument keyboard in real time during user plays virtual musical instrument keyboard；Gesture information according to each finger relative to the virtual musical instrument keyboard determines the mark of at least one virtual key of finger down；According to the mark of at least one virtual key of the finger down, play the corresponding musical instrument simulated sound of mark of at least one virtual key, without carrying the keyboard instruments such as entity piano, the performance of virtual musical instrument can be achieved whenever and wherever possible by terminal device and virtual musical instrument keyboard, it is easy to carry, and reduce the cost of purchase physical instrument.
Description
Technical field
The present embodiments relate to augmented reality field more particularly to a kind of virtual musical instrument processing methods, device, void
Quasi- instrument devices and storage medium.
Background technique
Music is a part of human soul, and piano is known as " king of musical instrument ", by vast music as common musical instrument
The welcome of fan.With the development of social progress and epoch, people start to focus on spiritual pursuit and personal mastery,
A large amount of piano study person is all emerged in the people of each age stratum.
The user of hobby piano is frequently desired to piano, and still, the volume of entity piano is larger, heavier-weight, generally
It is even more heavy between 200~500 kilograms, it is not easy to move, it is even more impossible to easily carry, be caused very to numerous piano users
Big puzzlement.
Summary of the invention
The embodiment of the present invention provides a kind of virtual musical instrument processing method, device, virtual musical instrument equipment and storage medium, to
Entity piano biggish musical instrument, heavier-weight in equal volume are solved, is not easy to move, it is even more impossible to easily carry.
The one aspect of the embodiment of the present invention is to provide a kind of virtual musical instrument processing method, comprising:
During user plays virtual musical instrument keyboard, each finger of user is obtained in real time relative to described virtual
The gesture information of keyboards for musical instruments；
Gesture information according to each finger relative to the virtual musical instrument keyboard determines at least the one of finger down
The mark of a virtual key；
According to the mark of at least one virtual key of the finger down, the mark of at least one virtual key is played
Know corresponding musical instrument simulated sound.
The other side of the embodiment of the present invention is to provide a kind of virtual musical instrument processing unit, comprising:
Gesture recognition module, for obtaining each of user in real time during user plays virtual musical instrument keyboard
Gesture information of the finger relative to the virtual musical instrument keyboard；
Processing module determines hand for the gesture information according to each finger relative to the virtual musical instrument keyboard
Refer to the mark at least one the virtual key pressed；
Playing module plays described at least one for the mark according at least one virtual key of the finger down
The corresponding musical instrument simulated sound of the mark of a virtual key.
The other side of the embodiment of the present invention is to provide a kind of virtual musical instrument equipment, comprising:
Memory, processor, and it is stored in the computer journey that can be run on the memory and on the processor
Sequence,
The processor realizes virtual musical instrument processing method described above when running the computer program.
The other side of the embodiment of the present invention is to provide a kind of virtual musical instrument equipment, comprising:
Photographic device, each finger for acquiring user in real time play the image information of the virtual musical instrument keyboard, institute
The top that photographic device is set to the virtual musical instrument keyboard is stated, the coverage of the photographic device covers the virtual musical instrument
Keyboard；
Memory,
Processor,
And it is stored in the computer program that can be run on the memory and on the processor,
The processor realizes virtual musical instrument processing method described above when running the computer program.
The other side of the embodiment of the present invention is to provide a kind of computer readable storage medium, is stored with computer journey
Sequence,
The computer program realizes virtual musical instrument processing method described above when being executed by processor.
Virtual musical instrument processing method, device, virtual musical instrument equipment and storage medium provided in an embodiment of the present invention, by
During user plays virtual musical instrument keyboard, each finger of user is obtained in real time relative to the virtual musical instrument keyboard
Gesture information；Gesture information according to each finger relative to the virtual musical instrument keyboard, determines finger down at least
The mark of one virtual key；According to the mark of at least one virtual key of the finger down, play it is described at least one
The corresponding musical instrument simulated sound of mark of virtual key passes through terminal device and void without carrying the keyboard instruments such as entity piano
The performance of virtual musical instrument can be achieved in quasi- keyboards for musical instruments whenever and wherever possible, easy to carry, and reduces the cost of purchase physical instrument.
Detailed description of the invention
Fig. 1 is the virtual musical instrument processing method flow chart that the embodiment of the present invention one provides；
Fig. 2 is virtual musical instrument processing method flow chart provided by Embodiment 2 of the present invention；
Fig. 3 is the structural schematic diagram for the virtual musical instrument processing unit that the embodiment of the present invention three provides；
Fig. 4 is the structural schematic diagram for the virtual musical instrument processing unit that the embodiment of the present invention four provides；
Fig. 5 is the structural schematic diagram for the virtual musical instrument equipment that the embodiment of the present invention five provides；
Fig. 6 is the structural schematic diagram for the virtual musical instrument equipment that the embodiment of the present invention six provides.
Through the above attached drawings, it has been shown that the specific embodiment of the present invention will be hereinafter described in more detail.These attached drawings
It is not intended to limit the range of design of the embodiment of the present invention in any manner with verbal description, but by reference to specific reality
Applying example is that those skilled in the art illustrate idea of the invention.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistent with the embodiment of the present invention.On the contrary, they be only with
The example of the consistent device and method of as detailed in the attached claim, the embodiment of the present invention some aspects.
Term " first " involved in the embodiment of the present invention, " second " etc. are used for description purposes only, and should not be understood as referring to
Show or imply relative importance or implicitly indicates the quantity of indicated technical characteristic.In the description of following embodiment,
The meaning of " plurality " is two or more, unless otherwise specifically defined.
These specific embodiments can be combined with each other below, may be at certain for the same or similar concept or process
It is repeated no more in a little embodiments.Below in conjunction with attached drawing, the embodiment of the present invention is described.
Embodiment one
Fig. 1 is the virtual musical instrument processing method flow chart that the embodiment of the present invention one provides.The embodiment of the present invention is directed to entity
Piano biggish musical instrument, heavier-weight in equal volume, are not easy to move, it is even more impossible to easily carry, provide virtual pleasure
Device processing method.
Method in the present embodiment is applied to terminal device, which can be smart phone, plate, AR equipment etc.
Mobile terminal is also possible to PC etc., and in other embodiments, this method applies also for other equipment, the present embodiment
It is schematically illustrated by taking terminal device as an example.
In the embodiment of the present invention, virtual musical instrument can be the keyboard instruments such as virtual piano, virtual electronic organ, including virtually find pleasure in
Device keyboard, each virtual key correspond to fixed pitch, can be to play any melody met in its range scope.
As shown in Figure 1, specific step is as follows for this method:
Step S101, during user plays virtual musical instrument keyboard, each finger for obtaining user in real time is opposite
In the gesture information of virtual musical instrument keyboard.
In the present embodiment, virtual musical instrument includes the virtual musical instrument keyboard being set on horizontal platform.Virtual musical instrument keyboard
It can be the keyboards for musical instruments drawn in platform surface or be located in the keyboards for musical instruments drawn on platform cardboard or be
The virtual musical instrument key of platform surface is projected in the pad pasting or can also be with keyboards for musical instruments drawing that platform surface is pasted
Disk etc., the present embodiment are not specifically limited herein for the implementation of virtual musical instrument keyboard.
During user plays virtual musical instrument keyboard, terminal device can acquire each finger phase of user in real time
For the gesture information of virtual musical instrument keyboard.
Wherein, each finger includes the gesture information of each finger relative to the gesture information of virtual musical instrument keyboard, and
The relative position information of gesture and virtual key that each finger is made.
Optionally, terminal device can also receive the process that other data acquisition devices play virtual musical instrument keyboard in user
In the gesture information of each finger for acquiring and send in real time relative to virtual musical instrument keyboard.
Step S102, the gesture information according to each finger relative to virtual musical instrument keyboard determines finger down at least
The mark of one virtual key.
In the present embodiment, gesture information according to each finger relative to virtual musical instrument keyboard is determining at least one hand
When finger makes the gesture for pressing key, according to the positional relationship for making the finger and each virtual key for pressing key gesture, really
Surely the mark of the virtual key of at least one being pressed.
Step S103, according to the mark of at least one virtual key of finger down, at least one virtual key is played
Identify corresponding musical instrument simulated sound.
After identifying the mark of at least one virtual key of finger down, the mark pair of these virtual keys is played
The sound answered plays sound so as to realize virtual musical instrument.
Optionally, the corresponding audio of mark of each virtual key can be stored in advance, determining that user's finger presses
Virtual key mark after, play the corresponding pre-stored audio of mark of the virtual key of finger down.
Optionally, the elementary audio of the musical instrument can be stored in advance, and each virtually the mark of key is corresponding
Music parameter, after the mark for determining virtual key that user's finger is pressed, according to the mark for the virtual key that user's finger is pressed
Know corresponding music parameter, pre-stored elementary audio is handled, obtains the corresponding audio of the music parameter, and play
Audio that treated.
In the present embodiment, during user plays virtual musical instrument keyboard, user switches every time when playing gesture, terminal
Equipment can collect gesture information of each finger of user relative to virtual musical instrument keyboard in real time, and determine finger down
Virtual key mark, the corresponding musical instrument simulated sound of mark of virtual key pressed is played, so as to realize void
The playing procedure of quasi- musical instrument.In the present embodiment once to switch to user and play gesture after, the gesture information of user is carried out
It identifies and simulates and issue the process of corresponding musical instrument simulated sound and be illustrated.
The embodiment of the present invention by obtaining each hand of user during user plays virtual musical instrument keyboard in real time
Refer to the gesture information relative to virtual musical instrument keyboard；Gesture information according to each finger relative to virtual musical instrument keyboard determines
The mark of the virtual key of at least one of finger down；According to the mark of at least one virtual key of finger down, play extremely
The corresponding musical instrument simulated sound of mark of a few virtual key is set without carrying the keyboard instruments such as entity piano by terminal
The performance of virtual musical instrument can be achieved in standby and virtual musical instrument keyboard whenever and wherever possible, easy to carry, and reduces purchase physical instrument
Cost.
Embodiment two
Fig. 2 is virtual musical instrument processing method flow chart provided by Embodiment 2 of the present invention.On the basis of above-described embodiment one
On, in the present embodiment, virtual musical instrument further include be set to above virtual musical instrument keyboard, coverage can cover it is entire virtual happy
The filming apparatus of device keyboard.During user plays virtual musical instrument keyboard, each finger for obtaining user in real time is opposite
In the gesture information of virtual musical instrument keyboard, comprising: each finger for receiving the user that photographic device is acquired and sent in real time is played
The image information of virtual musical instrument keyboard, photographic device are set to the top of virtual musical instrument keyboard, and the coverage of photographic device is covered
Lid virtual musical instrument keyboard；The image information for playing virtual musical instrument keyboard to each finger carries out image procossing, identifies each hand
Refer to the gesture relative to virtual musical instrument keyboard.As shown in Fig. 2, specific step is as follows for this method:
Step S201, each finger for receiving the user that photographic device is acquired and sent in real time plays virtual musical instrument keyboard
Image information, photographic device are set to the top of virtual musical instrument keyboard, and the coverage of photographic device covers virtual musical instrument keyboard.
In the embodiment of the present invention, virtual musical instrument can be the keyboard instruments such as virtual piano, virtual electronic organ, including virtually find pleasure in
Device keyboard, each virtual key, which corresponds to, fixed pitch, can be to play any melody met in its range scope.
Virtual musical instrument includes the virtual musical instrument keyboard being set on horizontal platform.Virtual musical instrument keyboard can be in platform
It the keyboards for musical instruments of surface rendering or is located in the keyboards for musical instruments drawn on platform cardboard or is viscous in platform surface
The pad pasting with keyboards for musical instruments drawing of patch or it can also be and project to virtual musical instrument keyboard of platform surface etc., this reality
The implementation that example is applied herein for virtual musical instrument keyboard is not specifically limited.
Optionally, virtual musical instrument keyboard can be the projection keyboards for musical instruments projected in the predeterminable area on platform.For example,
Virtual musical instrument keyboard can be projected in the predeterminable area on horizontally disposed platform by terminal device.
Optionally, the projection arrangement for projected virtual keyboards for musical instruments can be specially set, will virtually be found pleasure in by projection arrangement
Device keyboard projects in the predeterminable area on horizontally disposed platform.
In addition, photographic device is set to the top of virtual musical instrument keyboard, the coverage of photographic device is covered on platform
Predeterminable area, so that photographic device can cover entire virtual musical instrument keyboard, user on virtual musical instrument keyboard when playing, camera shooting
Device can take the image information that user plays all fingers used.
During user plays virtual musical instrument keyboard, each finger that photographic device acquires user in real time is played virtually
The image information of keyboards for musical instruments, and acquired image information is sent to terminal device in real time.Terminal device connects in real time
The each finger for receiving the user that photographic device is acquired and sent in real time plays the image information of virtual musical instrument keyboard.
Optionally, photographic device can be depth camera, play void according to depth camera each finger collected
The image information including depth information of quasi- keyboards for musical instruments.
Step S202, the image information for playing virtual musical instrument keyboard to each finger carries out image procossing, identifies each
Gesture of the finger relative to virtual musical instrument keyboard.
Terminal device plays the image letter of virtual musical instrument keyboard in each finger for the user for receiving photographic device transmission
After breath, Gesture Recognition can be used, the image information for playing virtual musical instrument keyboard to each finger carries out image procossing,
Gesture of each finger relative to virtual musical instrument keyboard is identified, so as to obtain playing the mistake of virtual musical instrument keyboard in user
Cheng Zhong, gesture information of each finger of user relative to virtual musical instrument keyboard.
Wherein, each finger includes the gesture information of each finger relative to the gesture information of virtual musical instrument keyboard, and
The relative position information of gesture and virtual key that each finger is made.
Optionally, the image information for playing virtual musical instrument keyboard to each finger carries out image procossing, identifies each hand
Refer to gesture relative to virtual musical instrument keyboard, specifically can using in the prior art any one according to the image information of finger into
The method of row gesture identification realizes that details are not described herein again for the present embodiment.
Above-mentioned steps S201-S202 is terminal device during user plays virtual musical instrument keyboard, is obtained in real time
A kind of feasible embodiment of each finger of user relative to the gesture information of virtual musical instrument keyboard.
Step S203, the gesture information according to each finger relative to virtual musical instrument keyboard determines finger down at least
The mark of one virtual key.
In practical applications, pitch corresponding to each key is different in keyboards for musical instruments.In the present embodiment, virtual musical instrument key
It is provided with unique mark on the virtual key of each of disk, for distinguishing different virtual keys.
Optionally, the mark of virtual key can be two dimensional code, character string, key number etc..
For example, being drawn on the pad pasting if virtual musical instrument keyboard is the pad pasting with fingerboard drawing that platform surface is pasted
Drawing on each key of system has the character string for being used to indicate its corresponding musical alphabet, the mark as key.
Optionally, the mark of virtual key can be set on corresponding virtual key, or setting is corresponding virtual
In default identified areas corresponding to key.
In addition, the mark of virtual key should be arranged in if the mark of virtual key is arranged on corresponding virtual key
The position that the users such as virtual key top will not block when playing.
In the present embodiment, gesture information according to each finger relative to virtual musical instrument keyboard determines finger down extremely
The mark of a few virtual key, can specifically realize in the following way:
Gesture information according to each finger relative to virtual musical instrument keyboard, determine make press the finger of key gesture by
Under position；According to the position for pressing the finger down of key gesture is made, the finger down for pressing key gesture is made in determination
Virtual key mark.
Step S204, according to the mark of at least one virtual key of finger down, at least one virtual key is played
Identify corresponding musical instrument simulated sound.
In the present embodiment, according to the mark of at least one virtual key of finger down, at least one virtual key is played
The corresponding musical instrument simulated sound of mark, can specifically realize in the following way:
According to the mark of at least one virtual key of finger down, the mark for obtaining at least one virtual key is corresponding
Music parameter；According to the corresponding music parameter of the mark of at least one virtual key, musical instrument simulated sound is played.
Wherein, music parameter can be used for describing the sound issued when the corresponding true key of a virtual key is pressed
Feature, may include the information such as pitch, scale, sound level, musical alphabet.
Optionally, music parameter corresponding to the mark of each virtual key can be stored in advance, according to finger down
The mark of virtual key can both inquire corresponding music parameter.
Optionally, virtual key is identified as two-dimensional barcode information, and two-dimensional barcode information includes the corresponding music ginseng of virtual key
Number.According to the two-dimensional barcode information of at least one virtual key of finger down, processing is decoded to two-dimensional barcode information, can be obtained
The corresponding music parameter of the virtual key of at least one of finger down into two-dimensional barcode information.
In the present embodiment, terminal device can be the movement that smart phone, plate, AR equipment etc. have audio playing function
Terminal or intelligent sound box, terminal device can play pleasure according to the corresponding music parameter of mark of at least one virtual key
Device simulated sound.
Optionally, the corresponding audio of mark of each virtual key can be stored in advance, determining that user's finger presses
Virtual key mark after, play the corresponding pre-stored audio of mark of the virtual key of finger down.
Optionally, the elementary audio of the musical instrument can be stored in advance, and each virtually the mark of key is corresponding
Music parameter, after the mark for determining virtual key that user's finger is pressed, according to the mark for the virtual key that user's finger is pressed
Know corresponding music parameter, pre-stored elementary audio is handled, obtains the corresponding audio of the music parameter, and play
Audio that treated.
Optionally, terminal device can also send play instruction to playback terminal, and play instruction includes that at least one is virtual
The corresponding music parameter of the mark of key, so that mark corresponding music parameter of the playback terminal according at least one virtual key
Play musical instrument simulated sound.
In the present embodiment, during user plays virtual musical instrument keyboard, user switches every time when playing gesture, terminal
Equipment can collect gesture information of each finger of user relative to virtual musical instrument keyboard in real time, and determine finger down
Virtual key mark, the corresponding musical instrument simulated sound of mark of virtual key pressed is played, so as to realize void
The playing procedure of quasi- musical instrument.In the present embodiment once to switch to user and play gesture after, the gesture information of user is carried out
It identifies and simulates and issue the process of corresponding musical instrument simulated sound and be illustrated.
The embodiment of the present invention by obtaining each hand of user during user plays virtual musical instrument keyboard in real time
Refer to the gesture information relative to virtual musical instrument keyboard；Gesture information according to each finger relative to virtual musical instrument keyboard determines
The mark of the virtual key of at least one of finger down；According to the mark of at least one virtual key of finger down, play extremely
The corresponding musical instrument simulated sound of mark of a few virtual key is set without carrying the keyboard instruments such as entity piano by terminal
The performance of virtual musical instrument can be achieved in standby and virtual musical instrument keyboard whenever and wherever possible, easy to carry, and reduces purchase physical instrument
Cost.
Embodiment three
Fig. 3 is the structural schematic diagram for the virtual musical instrument processing unit that the embodiment of the present invention three provides.The embodiment of the present invention mentions
The virtual musical instrument processing unit of confession can execute the process flow of virtual musical instrument processing method embodiment offer.As shown in figure 3, should
Virtual musical instrument processing unit 30 includes: gesture recognition module 301, processing module 302 and playing module 303.
Specifically, gesture recognition module 301 is used for during user plays virtual musical instrument keyboard, obtains use in real time
Gesture information of each finger at family relative to virtual musical instrument keyboard.
Processing module 302 determines finger down for the gesture information according to each finger relative to virtual musical instrument keyboard
At least one virtual key mark.
Playing module 303 is used for the mark of at least one virtual key according to finger down, and it is virtual to play at least one
The corresponding musical instrument simulated sound of the mark of key.
Device provided in an embodiment of the present invention can be specifically used for executing embodiment of the method provided by above-described embodiment one,
Details are not described herein again for concrete function.
The embodiment of the present invention by obtaining each hand of user during user plays virtual musical instrument keyboard in real time
Refer to the gesture information relative to virtual musical instrument keyboard；Gesture information according to each finger relative to virtual musical instrument keyboard determines
The mark of the virtual key of at least one of finger down；According to the mark of at least one virtual key of finger down, play extremely
The corresponding musical instrument simulated sound of mark of a few virtual key is set without carrying the keyboard instruments such as entity piano by terminal
The performance of virtual musical instrument can be achieved in standby and virtual musical instrument keyboard whenever and wherever possible, easy to carry, and reduces purchase physical instrument
Cost.
Example IV
Fig. 4 is the structural schematic diagram for the virtual musical instrument processing unit that the embodiment of the present invention four provides.In above-described embodiment three
On the basis of, in the present embodiment, gesture recognition module 301 is also used to:
The each finger for receiving the user that photographic device is acquired and sent in real time plays the image information of virtual musical instrument keyboard,
Photographic device is set to the top of virtual musical instrument keyboard, and the coverage of photographic device covers virtual musical instrument keyboard；To each hand
The image information that thrum plays virtual musical instrument keyboard carries out image procossing, identifies hand of each finger relative to virtual musical instrument keyboard
Gesture.
Optionally, as shown in figure 4, the virtual musical instrument processing unit 30 further include: virtual musical instrument keyboard projection modules 304.
Virtual musical instrument keyboard projection modules 304 are used for:
Virtual musical instrument keyboard is projected in the predeterminable area on horizontally disposed platform, photographic device is set to virtual pleasure
The top of device keyboard, the coverage of photographic device cover the predeterminable area on platform.
Optionally, unique mark is provided on each virtual key of virtual musical instrument keyboard.Processing module 302 is also used
In:
Gesture information according to each finger relative to virtual musical instrument keyboard, determine make press the finger of key gesture by
Under position；According to the position for pressing the finger down of key gesture is made, the finger down for pressing key gesture is made in determination
Virtual key mark.
Optionally, playing module 303 is also used to:
According to the mark of at least one virtual key of finger down, the mark for obtaining at least one virtual key is corresponding
Music parameter；According to the corresponding music parameter of the mark of at least one virtual key, musical instrument simulated sound is played.
Optionally, virtual key is identified as two-dimensional barcode information, and two-dimensional barcode information includes the corresponding music ginseng of virtual key
Number.
Optionally, playing module 303 is also used to:
Play instruction is sent to playback terminal, play instruction includes the corresponding music ginseng of mark of at least one virtual key
Number, so that playback terminal plays musical instrument simulated sound according to the corresponding music parameter of mark of at least one virtual key.
Device provided in an embodiment of the present invention can be specifically used for executing embodiment of the method provided by above-described embodiment two,
Details are not described herein again for concrete function.
The embodiment of the present invention by obtaining each hand of user during user plays virtual musical instrument keyboard in real time
Refer to the gesture information relative to virtual musical instrument keyboard；Gesture information according to each finger relative to virtual musical instrument keyboard determines
The mark of the virtual key of at least one of finger down；According to the mark of at least one virtual key of finger down, play extremely
The corresponding musical instrument simulated sound of mark of a few virtual key is set without carrying the keyboard instruments such as entity piano by terminal
The performance of virtual musical instrument can be achieved in standby and virtual musical instrument keyboard whenever and wherever possible, easy to carry, and reduces purchase physical instrument
Cost.
Embodiment five
Fig. 5 is the structural schematic diagram for the virtual musical instrument equipment that the embodiment of the present invention five provides.As shown in figure 5, the equipment 50
It include: processor 501, memory 502, and it is stored in the computer journey that can be executed on memory 502 and by processor 501
Sequence.
Processor 501 realizes any of the above-described embodiment of the method when executing and storing in the computer program on memory 502
The virtual musical instrument processing method of offer.
The embodiment of the present invention by obtaining each hand of user during user plays virtual musical instrument keyboard in real time
Refer to the gesture information relative to virtual musical instrument keyboard；Gesture information according to each finger relative to virtual musical instrument keyboard determines
The mark of the virtual key of at least one of finger down；According to the mark of at least one virtual key of finger down, play extremely
The corresponding musical instrument simulated sound of mark of a few virtual key is set without carrying the keyboard instruments such as entity piano by terminal
The performance of virtual musical instrument can be achieved in standby and virtual musical instrument keyboard whenever and wherever possible, easy to carry, and reduces purchase physical instrument
Cost.
Embodiment six
Fig. 6 is the structural schematic diagram for the virtual musical instrument equipment that the embodiment of the present invention six provides.As shown in fig. 6, the equipment 60
It include: photographic device 600, processor 601, memory 602, and be stored on memory 602 and can be executed by processor 601
Computer program.
Photographic device 600 plays the image information of virtual musical instrument keyboard for acquiring each finger of user in real time.Camera shooting
Device is set to the top of virtual musical instrument keyboard, and the coverage of photographic device covers virtual musical instrument keyboard.
Processor 601 realizes any of the above-described embodiment of the method when executing and storing in the computer program on memory 602
The virtual musical instrument processing method of offer.
In the embodiment of the present invention, virtual musical instrument equipment can be the keyboard instruments such as virtual piano, virtual electronic organ.
Optionally, virtual musical instrument equipment may include virtual musical instrument keyboard, and each virtual key, which corresponds to, fixed sound
Height, can be to play any melody met in its range scope.
Virtual musical instrument keyboard can be set on horizontal platform.Virtual musical instrument keyboard can be in platform surface drafting
Keyboards for musical instruments or be located in the keyboards for musical instruments drawn on platform cardboard or be platform surface paste with pleasure
The pad pasting of device keyboard drawing or it can also be and project to virtual musical instrument keyboard of platform surface etc., the present embodiment is right herein
It is not specifically limited in the implementation of virtual musical instrument keyboard.
Optionally, virtual musical instrument keyboard can be the projection keyboards for musical instruments projected in the predeterminable area on platform.For example,
Virtual musical instrument keyboard can be projected in the predeterminable area on horizontally disposed platform by terminal device.
Optionally, virtual musical instrument equipment can also include that the projection for projected virtual keyboards for musical instruments being specially arranged fills
It sets, is projected to virtual musical instrument keyboard in the predeterminable area on horizontally disposed platform by projection arrangement.
The embodiment of the present invention by obtaining each hand of user during user plays virtual musical instrument keyboard in real time
Refer to the gesture information relative to virtual musical instrument keyboard；Gesture information according to each finger relative to virtual musical instrument keyboard determines
The mark of the virtual key of at least one of finger down；According to the mark of at least one virtual key of finger down, play extremely
The corresponding musical instrument simulated sound of mark of a few virtual key is set without carrying the keyboard instruments such as entity piano by terminal
The performance of virtual musical instrument can be achieved in standby and virtual musical instrument keyboard whenever and wherever possible, easy to carry, and reduces purchase physical instrument
Cost.
In addition, the embodiment of the present invention also provides a kind of computer readable storage medium, it is stored with computer program, the meter
Calculation machine program realizes the virtual musical instrument processing method that any of the above-described embodiment of the method provides when being executed by processor.
In several embodiments provided by the present invention, it should be understood that disclosed device and method can pass through it
Its mode is realized.For example, the apparatus embodiments described above are merely exemplary, for example, the division of the unit, only
Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can be tied
Another system is closed or is desirably integrated into, or some features can be ignored or not executed.Another point, it is shown or discussed
Mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or logical of device or unit
Letter connection can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of hardware adds SFU software functional unit.
The above-mentioned integrated unit being realized in the form of SFU software functional unit can store and computer-readable deposit at one
In storage media.Above-mentioned SFU software functional unit is stored in a storage medium, including some instructions are used so that a computer
It is each that equipment (can be personal computer, server or the network equipment etc.) or processor (processor) execute the present invention
The part steps of embodiment the method.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (Read-
Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic or disk etc. it is various
It can store the medium of program code.
Those skilled in the art can be understood that, for convenience and simplicity of description, only with above-mentioned each functional module
Division progress for example, in practical application, can according to need and above-mentioned function distribution is complete by different functional modules
At the internal structure of device being divided into different functional modules, to complete all or part of the functions described above.On
The specific work process for stating the device of description, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to of the invention its
Its embodiment.The present invention is directed to cover any variations, uses, or adaptations of the invention, these modifications, purposes or
Person's adaptive change follows general principle of the invention and including the undocumented common knowledge in the art of the present invention
Or conventional techniques.The description and examples are only to be considered as illustrative, and true scope and spirit of the invention are by following
Claims are pointed out.
It should be understood that the present invention is not limited to the precise structure already described above and shown in the accompanying drawings, and
And various modifications and changes may be made without departing from the scope thereof.The scope of the present invention is only limited by appended claims
System.
Claims (17)
1. a kind of virtual musical instrument processing method characterized by comprising
During user plays virtual musical instrument keyboard, each finger of user is obtained in real time relative to the virtual musical instrument
The gesture information of keyboard；
Gesture information according to each finger relative to the virtual musical instrument keyboard determines at least one void of finger down
The mark of quasi- key；
According to the mark of at least one virtual key of the finger down, the mark pair of at least one virtual key is played
The musical instrument simulated sound answered.
2. the method according to claim 1, wherein described during user plays virtual musical instrument keyboard,
Gesture information of each finger of user relative to the virtual musical instrument keyboard is obtained in real time, comprising:
The each finger for receiving the user that photographic device is acquired and sent in real time plays the image information of the virtual musical instrument keyboard,
The photographic device is set to the top of the virtual musical instrument keyboard, and the coverage of the photographic device covers the virtual pleasure
Device keyboard；
The image information for playing the virtual musical instrument keyboard to each finger carries out image procossing, identifies each finger phase
For the gesture of the virtual musical instrument keyboard.
3. according to the method described in claim 2, it is characterized in that, the method also includes:
The virtual musical instrument keyboard is projected in the predeterminable area on horizontally disposed platform, the photographic device is set to institute
The top of virtual musical instrument keyboard is stated, the coverage of the photographic device covers the predeterminable area on the platform.
4. according to the method described in claim 2, it is characterized in that, being set on each virtual key of the virtual musical instrument keyboard
It is equipped with unique mark,
The gesture information according to each finger relative to the virtual musical instrument keyboard, determines at least the one of finger down
The mark of a virtual key, comprising:
The hand for pressing key gesture is made in gesture information according to each finger relative to the virtual musical instrument keyboard, determination
Refer to the position pressed；
Press the position of the finger down of key gesture according to described make, determine described in make press the finger of key gesture by
Under virtual key mark.
5. the method according to claim 1, wherein described at least one virtual qin according to the finger down
The mark of key plays the corresponding musical instrument simulated sound of mark of at least one virtual key, comprising:
According to the mark of at least one virtual key of the finger down, the mark pair of at least one virtual key is obtained
The music parameter answered；
According to the corresponding music parameter of mark of at least one virtual key, musical instrument simulated sound is played.
6. according to the method described in claim 5, it is characterized in that,
The virtual key is identified as two-dimensional barcode information, and the two-dimensional barcode information includes the corresponding music ginseng of the virtual key
Number.
7. according to the method described in claim 5, it is characterized in that, described at least one virtual qin according to the finger down
The mark of key, after the corresponding music parameter of mark for obtaining at least one virtual key, further includes:
Play instruction is sent to playback terminal, the play instruction includes the corresponding sound of mark of at least one virtual key
Happy parameter, so that the playback terminal plays musical instrument mould according to the corresponding music parameter of mark of at least one virtual key
Onomatopoeia sound.
8. a kind of virtual musical instrument processing unit characterized by comprising
Gesture recognition module, for obtaining each finger of user in real time during user plays virtual musical instrument keyboard
Gesture information relative to the virtual musical instrument keyboard；
Processing module, for the gesture information according to each finger relative to the virtual musical instrument keyboard, determine finger by
Under at least one virtual key mark；
Playing module plays that described at least one is empty for the mark according at least one virtual key of the finger down
The corresponding musical instrument simulated sound of mark of quasi- key.
9. device according to claim 8, which is characterized in that the gesture recognition module is also used to:
The each finger for receiving the user that photographic device is acquired and sent in real time plays the image information of the virtual musical instrument keyboard,
The photographic device is set to the top of the virtual musical instrument keyboard, and the coverage of the photographic device covers the virtual pleasure
Device keyboard；
The image information for playing the virtual musical instrument keyboard to each finger carries out image procossing, identifies each finger phase
For the gesture of the virtual musical instrument keyboard.
10. device according to claim 9, which is characterized in that described device further include: virtual musical instrument keyboard projective module
Block, the virtual musical instrument keyboard projection modules are used for:
The virtual musical instrument keyboard is projected in the predeterminable area on horizontally disposed platform, the photographic device is set to institute
The top of virtual musical instrument keyboard is stated, the coverage of the photographic device covers the predeterminable area on the platform.
11. device according to claim 9, which is characterized in that on each virtual key of the virtual musical instrument keyboard
It is provided with unique mark, the processing module is also used to:
The hand for pressing key gesture is made in gesture information according to each finger relative to the virtual musical instrument keyboard, determination
Refer to the position pressed；
Press the position of the finger down of key gesture according to described make, determine described in make press the finger of key gesture by
Under virtual key mark.
12. device according to claim 8, which is characterized in that the playing module is also used to:
According to the mark of at least one virtual key of the finger down, the mark pair of at least one virtual key is obtained
The music parameter answered；
According to the corresponding music parameter of mark of at least one virtual key, musical instrument simulated sound is played.
13. device according to claim 12, which is characterized in that
The virtual key is identified as two-dimensional barcode information, and the two-dimensional barcode information includes the corresponding music ginseng of the virtual key
Number.
14. device according to claim 12, which is characterized in that the playing module is also used to:
Play instruction is sent to playback terminal, the play instruction includes the corresponding sound of mark of at least one virtual key
Happy parameter, so that the playback terminal plays musical instrument mould according to the corresponding music parameter of mark of at least one virtual key
Onomatopoeia sound.
15. a kind of virtual musical instrument equipment characterized by comprising
Memory, processor, and it is stored in the computer program that can be run on the memory and on the processor,
The processor realizes such as method of any of claims 1-7 when running the computer program.
16. a kind of roadside device characterized by comprising
Photographic device, each finger for acquiring user in real time plays the image information of the virtual musical instrument keyboard, described to take the photograph
As device is set to the top of the virtual musical instrument keyboard, the coverage covering virtual musical instrument key of the photographic device
Disk；
Memory,
Processor,
And it is stored in the computer program that can be run on the memory and on the processor,
The processor realizes such as method of any of claims 1-7 when running the computer program.
17. a kind of computer readable storage medium, which is characterized in that it is stored with computer program,
Such as method of any of claims 1-7 is realized when the computer program is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810965724.8A CN109166565A (en) | 2018-08-23 | 2018-08-23 | Virtual musical instrument processing method, device, virtual musical instrument equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810965724.8A CN109166565A (en) | 2018-08-23 | 2018-08-23 | Virtual musical instrument processing method, device, virtual musical instrument equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109166565A true CN109166565A (en) | 2019-01-08 |
Family
ID=64896384
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810965724.8A Pending CN109166565A (en) | 2018-08-23 | 2018-08-23 | Virtual musical instrument processing method, device, virtual musical instrument equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109166565A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110223664A (en) * | 2019-05-22 | 2019-09-10 | 上海墩庐生物医学科技有限公司 | The methods and applications of opera gong and drum are played in a kind of mobile phone and plate mobile terminal |
| CN111103983A (en) * | 2019-12-31 | 2020-05-05 | 维沃移动通信有限公司 | AR musical instrument playing method and electronic equipment |
| WO2020155915A1 (en) * | 2019-01-29 | 2020-08-06 | 北京字节跳动网络技术有限公司 | Method and apparatus for playing back audio |
| CN111679806A (en) * | 2020-06-10 | 2020-09-18 | 浙江商汤科技开发有限公司 | Play control method and device, electronic equipment and storage medium |
| CN111862910A (en) * | 2019-04-25 | 2020-10-30 | 逢甲大学 | Virtual electronic musical instrument system and operation method thereof |
| CN112203114A (en) * | 2020-09-07 | 2021-01-08 | 佛山创视嘉科技有限公司 | Collaborative playing method, system, terminal device and storage medium |
| CN112289288A (en) * | 2020-11-03 | 2021-01-29 | 上海音乐学院 | Music creation system based on intelligent mobile terminal |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120007884A1 (en) * | 2010-07-06 | 2012-01-12 | Samsung Electronics Co., Ltd. | Apparatus and method for playing musical instrument using augmented reality technique in mobile terminal |
| CN103235641A (en) * | 2013-03-17 | 2013-08-07 | 浙江大学 | 6-dimensional sensory-interactive virtual keyboard instrument system and realization method thereof |
| CN105513580A (en) * | 2014-09-26 | 2016-04-20 | 上海渐华科技发展有限公司 | Keyboard musical instrument playing note identification system based on auxiliary camera |
| CN107644631A (en) * | 2017-10-13 | 2018-01-30 | 深圳市明德智慧教育科技有限公司 | Method, system and the virtual reality device of music input based on virtual reality |
| CN107924278A (en) * | 2015-08-12 | 2018-04-17 | 三星电子株式会社 | For playing the method for virtual musical instrument and for supporting its electronic equipment |
-
2018
- 2018-08-23 CN CN201810965724.8A patent/CN109166565A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120007884A1 (en) * | 2010-07-06 | 2012-01-12 | Samsung Electronics Co., Ltd. | Apparatus and method for playing musical instrument using augmented reality technique in mobile terminal |
| CN103235641A (en) * | 2013-03-17 | 2013-08-07 | 浙江大学 | 6-dimensional sensory-interactive virtual keyboard instrument system and realization method thereof |
| CN105513580A (en) * | 2014-09-26 | 2016-04-20 | 上海渐华科技发展有限公司 | Keyboard musical instrument playing note identification system based on auxiliary camera |
| CN107924278A (en) * | 2015-08-12 | 2018-04-17 | 三星电子株式会社 | For playing the method for virtual musical instrument and for supporting its electronic equipment |
| CN107644631A (en) * | 2017-10-13 | 2018-01-30 | 深圳市明德智慧教育科技有限公司 | Method, system and the virtual reality device of music input based on virtual reality |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020155915A1 (en) * | 2019-01-29 | 2020-08-06 | 北京字节跳动网络技术有限公司 | Method and apparatus for playing back audio |
| CN111862910A (en) * | 2019-04-25 | 2020-10-30 | 逢甲大学 | Virtual electronic musical instrument system and operation method thereof |
| CN110223664A (en) * | 2019-05-22 | 2019-09-10 | 上海墩庐生物医学科技有限公司 | The methods and applications of opera gong and drum are played in a kind of mobile phone and plate mobile terminal |
| CN111103983A (en) * | 2019-12-31 | 2020-05-05 | 维沃移动通信有限公司 | AR musical instrument playing method and electronic equipment |
| CN111679806A (en) * | 2020-06-10 | 2020-09-18 | 浙江商汤科技开发有限公司 | Play control method and device, electronic equipment and storage medium |
| CN112203114A (en) * | 2020-09-07 | 2021-01-08 | 佛山创视嘉科技有限公司 | Collaborative playing method, system, terminal device and storage medium |
| CN112203114B (en) * | 2020-09-07 | 2022-07-12 | 佛山创视嘉科技有限公司 | Collaborative playing method, system, terminal device and storage medium |
| CN112289288A (en) * | 2020-11-03 | 2021-01-29 | 上海音乐学院 | Music creation system based on intelligent mobile terminal |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109166565A (en) | Virtual musical instrument processing method, device, virtual musical instrument equipment and storage medium | |
| US20230044635A1 (en) | Audiovisual source separation and localization using generative adversarial networks | |
| CN104902012B (en) | The method and singing contest system of singing contest are carried out by network | |
| CN105706161B (en) | Automated audio based on pitch distributions is coordinated | |
| CN109271542A (en) | Cover determines method, apparatus, equipment and readable storage medium storing program for executing | |
| CN109215413A (en) | A kind of mold design teaching method, system and mobile terminal based on mobile augmented reality | |
| CN112883804B (en) | Error correction method and device for musical instrument playing hand actions and electronic equipment | |
| CN110189394A (en) | Shape of the mouth as one speaks generation method, device and electronic equipment | |
| CN108377417A (en) | Video reviewing method, device, computer equipment and storage medium | |
| CN116880701B (en) | Multimode interaction method and system based on holographic equipment | |
| CN110310613A (en) | A kind of method and apparatus for generating color encoded music | |
| CN109961800A (en) | A kind of page turning of score processing method and processing device | |
| CN109564756B (en) | Intelligence piano system | |
| CN114758636A (en) | Dance music generation method, device, terminal and readable storage medium | |
| CN106984044A (en) | Method and apparatus for starting default flow | |
| CN112818981B (en) | Musical instrument playing key position prompting method and device, electronic equipment and storage medium | |
| CN111104964B (en) | Method, equipment and computer storage medium for matching music with action | |
| US20230090995A1 (en) | Virtual-musical-instrument-based audio processing method and apparatus, electronic device, computer-readable storage medium, and computer program product | |
| WO2022221716A1 (en) | Multimedia music creation using visual input | |
| CN116115995A (en) | Image rendering processing method and device and electronic equipment | |
| CN103136277A (en) | Multimedia file playing method and electronic device | |
| CN114693848B (en) | Method, device, electronic equipment and medium for generating two-dimensional animation | |
| CN109801613B (en) | Terminal-based cello playing method and device and terminal | |
| Martin | Touchless gestural control of concatenative sound synthesis | |
| JP7059811B2 (en) | Information processing method and information processing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190108 |