CN101504832A - Virtual performance system based on hand motion sensing - Google Patents
Virtual performance system based on hand motion sensing Download PDFInfo
- Publication number
- CN101504832A CN101504832A CNA2009100808544A CN200910080854A CN101504832A CN 101504832 A CN101504832 A CN 101504832A CN A2009100808544 A CNA2009100808544 A CN A2009100808544A CN 200910080854 A CN200910080854 A CN 200910080854A CN 101504832 A CN101504832 A CN 101504832A
- Authority
- CN
- China
- Prior art keywords
- module
- hand
- hand motion
- scene
- system based
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention relates to a set of virtual performance system based on hand motion induction, which comprises a data acquisition glove, a machine vision module, a transmission module, a processing module and a peripheral device, wherein the data acquisition glove induces hand motion; the machine vision module induces hand position information; the transmission module transmits information acquired by the data acquisition glove and the machine vision module to the processing module; the processing module identifies the motion performed by a musical instrument from the received hand information, and generates performance sound and specific scene of the musical instrument; and the peripheral device feeds back the sound and the scene to an operator. The invention provides a virtual musical instrument for carrying out human-computer interaction by using the glove, which has the same or similar play skill as a real musical instrument, so that the operator can naturally and flexibly carry out virtual performance; and simultaneously the performance sound and the specific scene of the musical instrument are fed back to the operator, so that the operator can carry out performance more directly and naturally.
Description
Technical field
The present invention relates to the virtual performance system of a cover based on hand motion sensing.
Background technology
Along with the computer system performance fast lifting, many hardware that use on music making have transformed into software and have realized.The soft source of sound of a large amount of high-qualitys occurs in succession, makes people needn't spend hundreds of thousands even fund up to a million again and removes to buy the first-class grand piano of tone color and just can hear the sound that the top piano in the world sends even play.Meanwhile, the virtual musical instrument that is based upon software view also emerges in succession, and virtual musical instrument softwares such as virtual guitarist, REAL GUITAR have the abundant playing function and the tone color of high-quality.
The virtual musical instrument that utilizes soft source of sound technology is a kind of new trend of musical instrument development.But the development of virtual musical instrument also rests on the software view to a great extent in the market.A large amount of employings still is the MIDI keyboard controller on the hardware view, and its man-machine interaction mode is not still broken away from the form of keyboard, and the gimmick of playing of its input mode and real instrument differs greatly.Therefore, man-machine interaction mode how to improve virtual musical instrument has become the technical matters that those skilled in the art need to be resolved hurrily.
The Air Guitar that Virtual Air Guitar Company Oy produces is the comparatively successful product in present this field.It is by IP Camera identification user's gesture, but this is that the hand that can photograph the user with IP Camera is a prerequisite, and certain limitation is arranged, if the user back to this camera or hand outside the visual range of camera, just system can lose efficacy.It is imported the music intelligent object with gesture information and handles, and control produces sound, but music intelligent object principle is that two hands are considered as two points, and the wired equivalent of two points is in string, and this only is equivalent to play a string sounding, differs greatly with real guitar.It utilizes ground two pedals control beginning and stops simultaneously, makes troubles to the user.
The patent No. is 200480004097.8, and the patent that name is called " virtual reality musical glove system " has proposed a kind of virtual reality musical system.Its gloves assembly is combined with deformable tube and deformable tube is connected on the signal enabling device, discerns hand motion with this, but this only can discern the flexure operation of finger, can only realize impacting the instruments function of class, can not satisfy the demand of instrument playing fully.It switches musical instrument, pattern by the button button, has brought very big inconvenience to the user.Therefore the described virtual reality musical of this patent system can't bring approximate real instrument playing impression and comfortable man-machine interaction to experience to the user.
Summary of the invention
The purpose of this invention is to provide a kind of virtual musical instrument that utilizes gloves to carry out man-machine interaction, it plays gimmick and plays real instrument same or similar, makes the operator carry out virtual performance naturally, neatly.
Another object of the present invention is to realize playing flute, piano, three kinds of musical instruments of guitar with pair of glove.
The present invention can also feed back to the operator with the performance sound and the special effect scene of musical instrument, makes the operator more directly, play more naturally.
In order to achieve the above object, the invention provides the virtual performance system of a cover, comprising based on hand motion sensing:
The data acquisition gloves are responded to hand motion;
Machine vision module is responded to hand position information;
Transport module passes to processing module with the information of data acquisition gloves and machine vision module collection;
Processing module identifies the action of instrument playing from the hand information that receives, and produces the performance sound and the special effect scene of musical instrument;
Peripherals feeds back to the operator with sound and scene.
It plays gimmick and to play real instrument same or similar virtual performance system provided by the invention, the performance sound and the special effect scene of musical instrument can also be fed back to the operator, makes the operator more directly, play more naturally.
Description of drawings
Fig. 1 is the system global structure synoptic diagram.
Fig. 2 realizes synoptic diagram for system hardware.
Fig. 3 is a hand information acquisition circuit process flow diagram.
Fig. 4 is sensor installation site figure.
Fig. 5 is a system software podium level hierarchical structure chart.
Fig. 6 is for playing up realization flow figure.
Fig. 7 is the virtual performance scene.
Fig. 8 is a sound engine processing flow chart.
Wherein, 1-infrared LED, 2-bend sensor, 3-acceleration transducer, 4-baroceptor.
Embodiment
As shown in Figure 1, overall plan of the present invention is to utilize data acquisition gloves and machine vision module to gather the action and the positional information of hand, and is transferred to processing module; Processing module produces the performance sound and the special effect scene of musical instrument, and utilizes peripherals to feed back to the user.
System hardware is realized as shown in Figure 2.Total system comprises data acquisition gloves, USB camera, transport module, processing module, audio amplifier and display.
Described data acquisition gloves comprise bend sensor, acceleration transducer, baroceptor, infrarede emitting diode (LED) and the main control chip that is installed on the gloves, and sensor is settled according to playing needs; Signal of sensor is sampled through signal conditioning circuit and with A/D, passes to main control chip then, and passes to processing module through transport module.
Hand information acquisition circuit flow process on the described data acquisition gloves as shown in Figure 3.Every number of elements has used 5 bend sensors, 1 acceleration transducer, 1 baroceptor according to gathering on the gloves, always has 9 road signals.Because each signal of sensor is simulating signal, therefore need A/D that signal of sensor is sampled.The main control chip that system selects for use is integrated multichannel analog to digital converter (ADC), but if want system that 9 road signals are gathered, will make the signal timesharing enter the ADC port.Native system selects for use eight to select a multidiameter option switch, and 3 tunnel outputs of output of the bend sensor of five fingers and acceleration transducer are inserted selector switch, then selector switch is exported the ADC1 that receives main control chip, and the output of baroceptor is met ADC2.So just can finish 9 tunnel signals sampling, and the information that obtains is passed to processing module by transport module by main control chip is programmed to be implemented in the sampling period.Utilize the existing control of I/O cause for gossip of main control chip simultaneously to infrared LED.
Described data acquisition gloves, sensor should be settled according to playing needs when making.Bend sensor is used for the finger sensing bending, should be positioned over dorsal surfaces of fingers; Acceleration transducer induction hand is swept the string acceleration, is placed on the back of the hand; The baroceptor induction is blown and is moved, and is placed on the thumb; Infrared LED is to be used for the seizure of camera to hand position information, and three infrared LEDs are formed an array and placed the middle finger front end.The installation position of sensor on gloves as shown in Figure 4.
Described transport module both can be selected wired mode for use, also can select wireless mode for use.Described wired mode is to transmit by transmission line; Described wireless transmission method is Bluetooth transmission, Infrared Transmission or ZigBee transmission etc.Native system selects for use the ZigBee wireless transport module to realize data acquisition gloves transmission of Information, selects for use transmission line to realize the machine vision module transmission of Information.
The machine vision module of described system has adopted infrared LED as light source, utilizes infrared fileter to carry out squelch.The data that camera is caught pass to processing module by USB interface.
Described processing module comprises gloves message processing module, machine vision processing module, virtual performance scene module and audio frequency Compositing Engine, finishes the function that hand motion recognition realization, realizing of Robot Vision, figure realization and sound are realized respectively.System has also realized sending various control commands with hand motion on the basis of this framework, plays flute, piano, three kinds of musical instruments of guitar and impromptu pattern, bootmode, three kinds of functional modes of authoring modes with pair of glove.System software podium level aggregated(particle) structure as shown in Figure 5.
Described gloves message processing module obtains hand motion information by transport module, thereby calculates corresponding hand motion.
The machine vision processing module of described system is BMP because the single frames that camera provides is caught storage format, and the system that makes can directly handle image, adopts the RGB color model to come the remarked pixel point in the BMP image.System carries out the coordinate that coordinate computation can obtain photocentre by the conversion and the binary conversion treatment of rgb space to gray space, realizes the seizure to hand position information.
Described virtual performance scene module comprises that concert hall makes up module, the musical instrument model imports module, video camera roam module, performance special efficacy realization module, viewport and panel module and audio visualization window module are set.Scene needing to realize the scene rendering of multiframe.Wherein said concert hall is made of ground, left surface, right flank, positive these four basic quadrilateral surfaces.These four quadrilateral surfaces are carried out pinup picture play up, thereby constituted the concert hall scene.Described musical instrument model imports, and is about to the musical instrument model through processes such as world's conversion, coordinate transforms, is placed on the tram of scene.A video camera is promptly set up in described video camera roaming in scene, change the position coordinates and the pointing direction of video camera, can reach the effect of roaming in scene.Described performance special efficacy comprise lighting effects, note jump out fade out, three kinds of special efficacys of shock wave light beam.For the lighting effects of scene, the illumination system that is based on the summit of use is realized.Jump out for note and to fade out special efficacy, adopt the particIe system technology to realize.For shock wave light beam special efficacy, realize the main summit model of an annulus of structure and the summit model of a cylinder of relying on, the texture mapping of sticking shock wave realizes.Described viewport and panel setting are rectangular target zones of definition, draw scene in the zone.In the design of native system, screen display is divided into two parts, and last 3/5 part of screen is the three-dimensional scene of playing, and 2/5 part is a status panel down.There is content corresponding to show for different performance patterns.Described audio visualization window comes out sound exactly with avatars.System adopts the mode analyzing audio stream of Fourier transform, and the amplitude on each frequency band of sound is come out with 2D graphic presentation.
Every frame scene rendering realization flow figure may further comprise the steps as shown in Figure 6:
(1) the scene viewport is set;
(2) lighting effects are set;
(3) play up concert hall;
(4) play up musical instrument;
(5) video camera framework;
(6) judge whether shoegaze,, then continue following steps if play:
(7) realize shock wave light beam special efficacy;
(8) realize the floating special efficacy of fading out of note;
(9) the panel viewport is set;
(10) rendering state panel.
System virtualization is played scene effect figure as shown in Figure 7.
Described audio frequency Compositing Engine utilizes the sound loading technique to realize.System has adopted the stream buffer technology when sound loads, this method has guaranteed that audio file only dynamically calls in the internal memory from hard disk when needed, thereby has saved memory headroom.Sound loads the problem that also has load format, all uses with a kind of audio format and hardware output when all bufferings also to be to use this form, and audio mixing will need not spent any time on format conversion so, thereby reach optimum effect.The flow process that the sound engine is handled music as shown in Figure 8.
The present invention is by increasing the processing to various hand motions in processing module, thereby realize sending various control commands by hand motion, the switching of selecting, playing an instrument as musical instrument, the selection of playing model selection, menu etc. have broken away from traditional button input mode fully.
Described control command makes up realization mutually by particular sensor, avoiding control action and musical instrument to play under the identical prerequisite of action, makes everything succinctly understandable as far as possible.For example: be that musical instrument is selected scene when just entering performance system, the selection of the three kinds of musical instruments of mobile realization by the Machine Vision Recognition hand, both hands are clenched fist and are confirmed that musical instrument selects then.Switching between three instrument playing scenes then is to realize by playing the similar action of action with each musical instrument.Switch to flute play scene just left hand clench fist and blow to the baroceptor of left hand simultaneously; Switching to the piano performance scene just holds both hands simultaneously and gets rid of downwards; Switch to guitar performance scene and just hold the right hand, left hand is done and is swept the string action.
In processing module, increase processing, can also realize playing flute, piano, three kinds of musical instruments of guitar with pair of glove to hand motion.
Described flute is played, and utilizes baroceptor and bend sensor to discern the performance action of flute, has realized and the identical fingering of true flute.Use six fingers of two hands except that little finger of toe and thumb to play, utilize baroceptor induction air blowing air pressure simultaneously, whether decision operation person is carrying out the flute performance.
Described piano performance utilizes bend sensor and machine vision to discern the performance action of piano.System has adopted three ranges of sound, and the control in the range of sound realizes that by machine vision it can discern the volume coordinate position of gloves, and is mapped to on-screen display (osd) area, and evenly cutting apart of directions X carried out in the visual field that is about to camera.The small character group of giving tacit consent to central C place is the range of sound, center.CDEFGAB sound level is arranged in each range of sound, and these sound levels can come correspondence to play by the different fingers of gloves.
Described guitar is played, and utilizes bend sensor and acceleration transducer to discern the performance action of guitar.System has adopted sample and has controlled implementation one to one in operation control, i.e. the corresponding independent musical sound of each independent finger action.Simultaneity factor also utilizes the string of sweeping of acceleration transducer induction hand to move, and whether decision operation person is carrying out the guitar performance.
Native system is played the mode treatment module by increasing in processing module, realize extemporize, and guiding is played, and three kinds of performance patterns are played in creation.
The pattern that described extemporize is promptly freely played, lower panel shows the current performance operating state of musical instrument in real time in the mode of animation.
Described guiding is played, and promptly the lower panel zone is a guide panel, and a visual guide mode of music score is arranged.System presses corresponding key by music score guidance announcement user in the corresponding time.Thereby the guiding user carries out the performance of corresponding melody.
Described creation is played, and promptly the lower panel zone is the creation panel.In the playing procedure of musical instrument, automatically show the note of being played on the panel, the pitch and the duration of a sound, and displaying contents is saved as music score file.
Described peripherals comprises audio amplifier and display.They feed back to the user with the performance sound and the special effect scene of musical instrument, bring approximate true, comfortable instrument playing impression to the user.
Claims (10)
1, a kind of virtual performance system based on hand motion sensing is characterized in that comprising:
The data acquisition gloves are responded to hand motion;
Machine vision module is responded to hand position information;
Transport module passes to processing module with the information of data acquisition gloves and machine vision module collection;
Processing module identifies the action of instrument playing from the hand information that receives, and produces the performance sound and the special effect scene of musical instrument;
Peripherals feeds back to the operator with sound and scene.
2, the virtual performance system based on hand motion sensing as claimed in claim 1 is characterized in that described data acquisition gloves comprise bend sensor, acceleration transducer, baroceptor, infrarede emitting diode and the main control chip that is installed on the gloves; Signal of sensor is sampled through signal conditioning circuit and with A/D, passes to main control chip then, and passes to processing module through transport module; Sensor is settled according to playing needs; Bend sensor is used for the finger sensing bending, is positioned over dorsal surfaces of fingers; Acceleration transducer induction hand is swept the string acceleration, is placed on the back of the hand; The baroceptor induction is blown and is moved, and is placed on the thumb; Infrarede emitting diode is used for the seizure of camera adversary positional information, and three infrarede emitting diodes are formed an array and placed the middle finger front end.
3, the virtual performance system based on hand motion sensing as claimed in claim 1 is characterized in that described machine vision module, adopts infrared LED as light source, utilizes infrared fileter to carry out squelch.
4, the virtual performance system based on hand motion sensing as claimed in claim 1, it is characterized in that described processing module comprises gloves message processing module, machine vision processing module, virtual performance scene module and audio frequency Compositing Engine, finish the function that hand motion recognition, realizing of Robot Vision, figure realization and sound are realized respectively.
5, the virtual performance system based on hand motion sensing as claimed in claim 4, it is characterized in that described machine vision processing module is by conversion and the binary conversion treatment of rgb space to gray space, carry out coordinate computation and obtain the coordinate of photocentre, realize seizure hand position information.
6, the virtual performance system based on hand motion sensing as claimed in claim 4 is characterized in that described virtual performance scene module comprises that concert hall makes up module, the musical instrument model imports module, video camera roam module, performance special efficacy realization module, viewport and panel module and audio visualization window module are set; Described concert hall is made of ground, left surface, right flank, positive these four basic quadrilateral surfaces, these four quadrilateral surfaces is carried out pinup picture play up, thereby constituted the concert hall scene; Described musical instrument model imports module and the musical instrument model is placed on the tram of scene; Described video camera roam module reaches the effect of roaming by changing the position coordinates and the pointing direction of video camera in scene; Described performance special efficacy realizes that module realizes the special-effect of playing; Described viewport and panel are provided with rectangular target zone of module definition, draw scene in the zone; Described audio visualization window comes out sound with avatars.
7, the virtual performance system based on hand motion sensing as claimed in claim 4 is characterized in that described audio frequency Compositing Engine utilizes the sound loading technique to realize; System has adopted the stream buffer technology when sound loads, and the output of all bufferings and hardware is all used with a kind of audio format.
8, the virtual performance system based on hand motion sensing as claimed in claim 1 is characterized in that increasing the processing to various hand motions in processing module, realize sending various control commands by hand motion.
9, the virtual performance system based on hand motion sensing as claimed in claim 1 is characterized in that increasing the processing to various hand motions in processing module, realize playing flute, piano, three kinds of musical instruments of guitar with pair of glove.
10, the virtual performance system based on hand motion sensing as claimed in claim 1 is characterized in that increasing in processing module and plays the mode treatment module, realizes extemporize, and guiding is played, and three kinds of performance patterns are played in creation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2009100808544A CN101504832A (en) | 2009-03-24 | 2009-03-24 | Virtual performance system based on hand motion sensing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2009100808544A CN101504832A (en) | 2009-03-24 | 2009-03-24 | Virtual performance system based on hand motion sensing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101504832A true CN101504832A (en) | 2009-08-12 |
Family
ID=40977054
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2009100808544A Pending CN101504832A (en) | 2009-03-24 | 2009-03-24 | Virtual performance system based on hand motion sensing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101504832A (en) |
Cited By (35)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102270446A (en) * | 2010-06-01 | 2011-12-07 | 卡西欧计算机株式会社 | Performance apparatus and electronic musical instrument |
| CN102573193A (en) * | 2010-12-24 | 2012-07-11 | 上海广茂达光艺科技股份有限公司 | Interactive type three-dimensional editing device and method of LED (Light-emitting Diode) lamplight scene |
| CN102723019A (en) * | 2012-05-23 | 2012-10-10 | 苏州奇可思信息科技有限公司 | Sign language teaching system |
| CN102907797A (en) * | 2012-10-24 | 2013-02-06 | 天津大学 | Air-blown gloves based on acceleration sensors and control method thereof |
| CN103000171A (en) * | 2011-08-23 | 2013-03-27 | 卡西欧计算机株式会社 | Musical instrument, light-emission controller, and control method of musical instrument |
| CN103207677A (en) * | 2013-04-22 | 2013-07-17 | 北京工业大学 | System and method for realizing virtual-real somatosensory interaction of digital Zenghouyi bells |
| CN103310769A (en) * | 2012-03-14 | 2013-09-18 | 卡西欧计算机株式会社 | Musical performance device,and method for controlling musical performance device |
| CN103310767A (en) * | 2012-03-15 | 2013-09-18 | 卡西欧计算机株式会社 | Musical performance device,and method for controlling musical performance device |
| CN103412649A (en) * | 2013-08-20 | 2013-11-27 | 苏州跨界软件科技有限公司 | Control system based on non-contact type hand motion capture |
| CN104599553A (en) * | 2014-12-29 | 2015-05-06 | 闽南师范大学 | Barcode recognition-based sign language teaching system and method |
| CN104856707A (en) * | 2015-05-14 | 2015-08-26 | 上海大学 | Pressure sensing data glove based on machine vision and gripping process judgment method thereof |
| CN105786162A (en) * | 2014-12-15 | 2016-07-20 | 上海贝尔股份有限公司 | Method and device for virtual performance commanding |
| CN105867656A (en) * | 2016-04-27 | 2016-08-17 | 牛亚锋 | Motion-sensing music wearable device and playing motion recognition method thereof |
| CN105913498A (en) * | 2016-03-28 | 2016-08-31 | 重庆工程学院 | Full-platform butt joint service system for providing technical terminal by use of unreal engine technology |
| CN105929956A (en) * | 2016-04-26 | 2016-09-07 | 苏州冰格智能科技有限公司 | Virtual reality-based input method |
| CN105955464A (en) * | 2016-04-26 | 2016-09-21 | 苏州冰格智能科技有限公司 | Input device based on virtual reality |
| CN104599554B (en) * | 2014-12-29 | 2017-01-25 | 闽南师范大学 | Two-dimensional code recognition-based sign language teaching system and method |
| CN106355974A (en) * | 2016-11-09 | 2017-01-25 | 大连文森特软件科技有限公司 | Violin supplementary learning experience system based on augmented reality (AR) |
| CN106373440A (en) * | 2016-11-09 | 2017-02-01 | 大连文森特软件科技有限公司 | Augmented reality and instant communication-based violin learning experience assisting system |
| CN106373456A (en) * | 2016-11-09 | 2017-02-01 | 大连文森特软件科技有限公司 | Augmented reality and performance evaluation-based violin learning experience assisting system |
| CN106601217A (en) * | 2016-12-06 | 2017-04-26 | 北京邮电大学 | Interactive-type musical instrument performing method and device |
| CN106782458A (en) * | 2016-12-01 | 2017-05-31 | 山东大学 | Play method and device in space based on three-dimensional localization principle |
| CN106952638A (en) * | 2017-04-28 | 2017-07-14 | 安徽克洛斯威智能乐器科技有限公司 | One kind sensing piano key disc type gloves and its implementation |
| CN107373826A (en) * | 2017-08-17 | 2017-11-24 | 国网四川省电力公司技能培训中心 | One kind is used for virtual reality emulation analogue data gloves |
| CN107407961A (en) * | 2015-01-19 | 2017-11-28 | 寇夫音乐有限公司 | The hand-held controller of computer, the control system of computer and computer system |
| CN107578765A (en) * | 2017-08-24 | 2018-01-12 | 得理电子(上海)有限公司 | Applied to musical instrument or the controller of musical instruments |
| CN107993376A (en) * | 2012-12-06 | 2018-05-04 | 陆洪瑞 | A kind of transaction system, code keyboard system and its operating method |
| CN108986777A (en) * | 2018-06-14 | 2018-12-11 | 森兰信息科技（上海）有限公司 | Method, somatosensory device and the musical instrument terminal of music simulation are carried out by body-sensing |
| CN109559593A (en) * | 2018-12-29 | 2019-04-02 | 深圳市掌网科技股份有限公司 | A kind of piano training system and method based on mixed reality |
| CN109584843A (en) * | 2018-12-27 | 2019-04-05 | 章微微 | Height reviews one's lessons by oneself formal kahikatea Chinese zither |
| CN109782918A (en) * | 2019-01-25 | 2019-05-21 | 南阳理工学院 | A kind of computer-assisted instruction device |
| CN109887481A (en) * | 2019-02-01 | 2019-06-14 | 中央民族大学 | Electronic organ performance method and device |
| CN110767031A (en) * | 2019-10-28 | 2020-02-07 | 武强嘉华乐器有限公司 | Music teaching platform of VR technique |
| CN111128098A (en) * | 2019-12-30 | 2020-05-08 | 休止符科技深圳有限公司 | Hand-wearing type intelligent musical instrument and playing method thereof |
| CN111862910A (en) * | 2019-04-25 | 2020-10-30 | 逢甲大学 | Virtual electronic musical instrument system and operation method thereof |
-
2009
- 2009-03-24 CN CNA2009100808544A patent/CN101504832A/en active Pending
Cited By (53)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102270446B (en) * | 2010-06-01 | 2014-08-13 | 卡西欧计算机株式会社 | Performance apparatus and electronic musical instrument |
| CN102270446A (en) * | 2010-06-01 | 2011-12-07 | 卡西欧计算机株式会社 | Performance apparatus and electronic musical instrument |
| CN102573193A (en) * | 2010-12-24 | 2012-07-11 | 上海广茂达光艺科技股份有限公司 | Interactive type three-dimensional editing device and method of LED (Light-emitting Diode) lamplight scene |
| CN102573193B (en) * | 2010-12-24 | 2016-04-27 | 上海广茂达光艺科技股份有限公司 | The interactive three-dimensional editing device of LED lamplight scene and method |
| CN103000171A (en) * | 2011-08-23 | 2013-03-27 | 卡西欧计算机株式会社 | Musical instrument, light-emission controller, and control method of musical instrument |
| CN103310769B (en) * | 2012-03-14 | 2015-12-23 | 卡西欧计算机株式会社 | The control method of music performance apparatus and music performance apparatus |
| CN103310769A (en) * | 2012-03-14 | 2013-09-18 | 卡西欧计算机株式会社 | Musical performance device,and method for controlling musical performance device |
| CN103310767B (en) * | 2012-03-15 | 2015-12-23 | 卡西欧计算机株式会社 | The control method of music performance apparatus and music performance apparatus |
| CN103310767A (en) * | 2012-03-15 | 2013-09-18 | 卡西欧计算机株式会社 | Musical performance device,and method for controlling musical performance device |
| CN102723019A (en) * | 2012-05-23 | 2012-10-10 | 苏州奇可思信息科技有限公司 | Sign language teaching system |
| CN102907797A (en) * | 2012-10-24 | 2013-02-06 | 天津大学 | Air-blown gloves based on acceleration sensors and control method thereof |
| CN107993376A (en) * | 2012-12-06 | 2018-05-04 | 陆洪瑞 | A kind of transaction system, code keyboard system and its operating method |
| CN103207677B (en) * | 2013-04-22 | 2017-05-10 | 北京工业大学 | System and method for realizing virtual-real somatosensory interaction of digital Zenghouyi bells |
| CN103207677A (en) * | 2013-04-22 | 2013-07-17 | 北京工业大学 | System and method for realizing virtual-real somatosensory interaction of digital Zenghouyi bells |
| CN103412649A (en) * | 2013-08-20 | 2013-11-27 | 苏州跨界软件科技有限公司 | Control system based on non-contact type hand motion capture |
| CN105786162A (en) * | 2014-12-15 | 2016-07-20 | 上海贝尔股份有限公司 | Method and device for virtual performance commanding |
| CN104599554B (en) * | 2014-12-29 | 2017-01-25 | 闽南师范大学 | Two-dimensional code recognition-based sign language teaching system and method |
| CN104599553B (en) * | 2014-12-29 | 2017-01-25 | 闽南师范大学 | Barcode recognition-based sign language teaching system and method |
| CN104599553A (en) * | 2014-12-29 | 2015-05-06 | 闽南师范大学 | Barcode recognition-based sign language teaching system and method |
| CN107407961A (en) * | 2015-01-19 | 2017-11-28 | 寇夫音乐有限公司 | The hand-held controller of computer, the control system of computer and computer system |
| CN104856707B (en) * | 2015-05-14 | 2018-10-23 | 上海大学 | Pressure sensing data glove based on machine vision and its grasping process judgment method |
| CN104856707A (en) * | 2015-05-14 | 2015-08-26 | 上海大学 | Pressure sensing data glove based on machine vision and gripping process judgment method thereof |
| CN105913498A (en) * | 2016-03-28 | 2016-08-31 | 重庆工程学院 | Full-platform butt joint service system for providing technical terminal by use of unreal engine technology |
| CN105929956A (en) * | 2016-04-26 | 2016-09-07 | 苏州冰格智能科技有限公司 | Virtual reality-based input method |
| CN105955464A (en) * | 2016-04-26 | 2016-09-21 | 苏州冰格智能科技有限公司 | Input device based on virtual reality |
| CN105867656A (en) * | 2016-04-27 | 2016-08-17 | 牛亚锋 | Motion-sensing music wearable device and playing motion recognition method thereof |
| CN105867656B (en) * | 2016-04-27 | 2019-03-08 | 牛亚锋 | A kind of performance action identification method based on body-sensing music wearable device |
| CN106373440B (en) * | 2016-11-09 | 2019-03-26 | 快创科技（大连）有限公司 | Based on AR augmented reality and the violin assisted learning experiencing system that exchanges immediately |
| CN106373456A (en) * | 2016-11-09 | 2017-02-01 | 大连文森特软件科技有限公司 | Augmented reality and performance evaluation-based violin learning experience assisting system |
| CN106355974B (en) * | 2016-11-09 | 2019-03-12 | 快创科技（大连）有限公司 | Violin assisted learning experiencing system based on AR augmented reality |
| CN106373440A (en) * | 2016-11-09 | 2017-02-01 | 大连文森特软件科技有限公司 | Augmented reality and instant communication-based violin learning experience assisting system |
| CN106373456B (en) * | 2016-11-09 | 2019-03-05 | 快创科技（大连）有限公司 | Based on AR augmented reality and play the violin assisted learning experiencing system evaluated |
| CN106355974A (en) * | 2016-11-09 | 2017-01-25 | 大连文森特软件科技有限公司 | Violin supplementary learning experience system based on augmented reality (AR) |
| CN106782458A (en) * | 2016-12-01 | 2017-05-31 | 山东大学 | Play method and device in space based on three-dimensional localization principle |
| CN106601217B (en) * | 2016-12-06 | 2021-03-02 | 北京邮电大学 | Interactive musical instrument playing method and device |
| CN106601217A (en) * | 2016-12-06 | 2017-04-26 | 北京邮电大学 | Interactive-type musical instrument performing method and device |
| CN106952638A (en) * | 2017-04-28 | 2017-07-14 | 安徽克洛斯威智能乐器科技有限公司 | One kind sensing piano key disc type gloves and its implementation |
| CN107373826A (en) * | 2017-08-17 | 2017-11-24 | 国网四川省电力公司技能培训中心 | One kind is used for virtual reality emulation analogue data gloves |
| CN107578765A (en) * | 2017-08-24 | 2018-01-12 | 得理电子(上海)有限公司 | Applied to musical instrument or the controller of musical instruments |
| CN108986777A (en) * | 2018-06-14 | 2018-12-11 | 森兰信息科技（上海）有限公司 | Method, somatosensory device and the musical instrument terminal of music simulation are carried out by body-sensing |
| US20220366884A1 (en) * | 2018-06-14 | 2022-11-17 | Sunland Information Technology Co., Ltd. | Systems and methods for music simulation via motion sensing |
| US11437006B2 (en) * | 2018-06-14 | 2022-09-06 | Sunland Information Technology Co., Ltd. | Systems and methods for music simulation via motion sensing |
| US11749246B2 (en) * | 2018-06-14 | 2023-09-05 | Sunland Information Technology Co., Ltd. | Systems and methods for music simulation via motion sensing |
| CN109584843A (en) * | 2018-12-27 | 2019-04-05 | 章微微 | Height reviews one's lessons by oneself formal kahikatea Chinese zither |
| CN109584843B (en) * | 2018-12-27 | 2022-02-11 | 余姚市恒正金属制品有限公司 | Highly self-correcting white pine koto |
| CN109559593A (en) * | 2018-12-29 | 2019-04-02 | 深圳市掌网科技股份有限公司 | A kind of piano training system and method based on mixed reality |
| CN109559593B (en) * | 2018-12-29 | 2021-06-04 | 深圳市掌网科技股份有限公司 | Piano training system and method based on mixed reality |
| CN109782918A (en) * | 2019-01-25 | 2019-05-21 | 南阳理工学院 | A kind of computer-assisted instruction device |
| CN109887481B (en) * | 2019-02-01 | 2021-06-01 | 中央民族大学 | Electronic organ playing method and device |
| CN109887481A (en) * | 2019-02-01 | 2019-06-14 | 中央民族大学 | Electronic organ performance method and device |
| CN111862910A (en) * | 2019-04-25 | 2020-10-30 | 逢甲大学 | Virtual electronic musical instrument system and operation method thereof |
| CN110767031A (en) * | 2019-10-28 | 2020-02-07 | 武强嘉华乐器有限公司 | Music teaching platform of VR technique |
| CN111128098A (en) * | 2019-12-30 | 2020-05-08 | 休止符科技深圳有限公司 | Hand-wearing type intelligent musical instrument and playing method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101504832A (en) | Virtual performance system based on hand motion sensing | |
| US6388183B1 (en) | Virtual musical instruments with user selectable and controllable mapping of position input to sound output | |
| US7199301B2 (en) | Freely specifiable real-time control | |
| CN103235641B (en) | Dummy keyboard musical instrument system that a kind of 6 dimension sense organs are mutual and its implementation | |
| US8259072B2 (en) | Input control apparatus and an interactive system using the same | |
| JP4895352B2 (en) | Object selection program, object selection device, object selection system, and object selection method | |
| US7890199B2 (en) | Storage medium storing sound output control program and sound output control apparatus | |
| US20060123982A1 (en) | Wearable sensor matrix system for machine control | |
| US20110119580A1 (en) | Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen | |
| US20060000345A1 (en) | Musical sound production apparatus and musical | |
| US20160063975A1 (en) | Performance method of electronic musical instrument and music | |
| CN201570011U (en) | Terminal control device and terminal | |
| US7884802B2 (en) | Information selecting apparatus and storage medium storing information selecting program | |
| KR20060108766A (en) | Method and apparatus for light input device | |
| US20220137701A1 (en) | Systems and methods for controlling secondary devices using mixed, virtual or augmented reality | |
| GB2496521A (en) | Computerised musical instrument using motion capture and analysis | |
| JP2008217543A (en) | Information selecting device, information selecting program, game device, and game program | |
| JP2012113624A (en) | Information processing program, information processing device, information processing system, and information processing method | |
| US20160307553A1 (en) | Electronic apparatus and control method thereof | |
| CN108200703A (en) | Control method, device and the electronic equipment of a kind of light with audio interaction | |
| CN106713699A (en) | Control system | |
| JPWO2019123888A1 (en) | Information processing system, information processing method, and program | |
| JP4922853B2 (en) | Viewing environment control device, viewing environment control system, and viewing environment control method | |
| KR100742664B1 (en) | Interactive motion-graphics display system and method | |
| JP2022159386A (en) | Entertainment apparatus, light emission controlling apparatus, operation device, light emission controlling method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20090812 |