CN106601218A - Audio and video and piano action synchronization device and method - Google Patents

Audio and video and piano action synchronization device and method Download PDF

Info

Publication number
CN106601218A
CN106601218A CN201611198031.8A CN201611198031A CN106601218A CN 106601218 A CN106601218 A CN 106601218A CN 201611198031 A CN201611198031 A CN 201611198031A CN 106601218 A CN106601218 A CN 106601218A
Authority
CN
China
Prior art keywords
module
piano
audio
video
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201611198031.8A
Other languages
Chinese (zh)
Inventor
郭立
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hunan Joy And Cloud Network Technology Co Ltd
Original Assignee
Hunan Joy And Cloud Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hunan Joy And Cloud Network Technology Co Ltd filed Critical Hunan Joy And Cloud Network Technology Co Ltd
Priority to CN201611198031.8A priority Critical patent/CN106601218A/en
Publication of CN106601218A publication Critical patent/CN106601218A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00Details of electrophonic musical instruments
    • G10H1/0033Recording/reproducing or transmission of music for electrophonic musical instruments
    • G10H1/0041Recording/reproducing or transmission of music for electrophonic musical instruments in coded form
    • G10H1/0058Transmission between separate instruments or between individual components of a musical system
    • G10H1/0066Transmission between separate instruments or between individual components of a musical system using a MIDI interface
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10FAUTOMATIC MUSICAL INSTRUMENTS
    • G10F1/00Automatic musical instruments
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10FAUTOMATIC MUSICAL INSTRUMENTS
    • G10F1/00Automatic musical instruments
    • G10F1/02Pianofortes with keyboard
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10FAUTOMATIC MUSICAL INSTRUMENTS
    • G10F3/00Independent players for keyboard instruments

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Electrophonic Musical Instruments (AREA)

Abstract

The invention discloses an audio and video and piano action synchronization device and method. The device comprises an audio and video acquisition module arranged at a mobile terminal, a data processing module, a data storage module and a mobile terminal data transceiver module and also comprises a piano action collection module and a piano data transceiver module arranged on a piano. The piano action collection module collects piano action and corresponding time stamps and sends the piano action and corresponding time stamps to the piano data transceiver module in the form of a midi message. The piano data transceiver module and the mobile terminal data transceiver module carry out data interaction. The mobile terminal data transceiver module sends the received midi message to the data processing module. The audio and video acquisition module collects audio and video information and sends the information to the data processing module. The data processing module synchronizes the midi message and the audio and video information to generate a music file and sends the music file to the data storage module. The data storage module stores the music file. When the audio and video information and the piano information are collected to generate an automatic playing file, the synchronization of audio and video and the piano action is ensured.

Description

A kind of audio frequency and video and piano action sychronisation and method
Technical field
The present invention relates to automatic Playing musical instrument field, more particularly to a kind of audio frequency and video and piano action sychronisation and side Method.
Background technology
Being provided with the piano of automatic playing system can realize, according to instruction automatic playing piano, playing without very important person, It is to drive piano key to carry out automatic Playing by automatic playing system.Automatic playing system stores the number prerecorded Digital data, these data be for controlling piano key and pedal bullet under and reset motion, including the speed under bullet and other Some information.It is more rich and varied in order to allow this system to play, the good audio frequency and video recording of meeting played pre-recorded, and Audio frequency and video need and piano is synchronously played.In order to ensure can to realize that audio frequency and video are same with piano action in piano automatic Playing Step, it is necessary to ensure that audio frequency and video are same with piano action when audio frequency and video are gathered and piano information generates automatic Playing file Step.
Therefore how could be when audio frequency and video be gathered and piano information generates automatic Playing file, it is ensured that audio frequency and video and piano The synchronization of action, improving the sight of automatic playing system becomes the problem of those skilled in the art's urgent need to resolve.
The content of the invention
In view of this, it is an object of the invention to provide a kind of audio frequency and video and piano action sychronisation, regard in collection sound When frequency and piano information generate automatic Playing file, it is ensured that audio frequency and video are synchronous with piano action, automatic playing system is improved Sight.
The present invention provides a kind of audio frequency and video and piano action sychronisation, adopts including the audio frequency and video being arranged on mobile terminal Collection module, data processing module, data memory module and mobile terminal data transceiver module, and the piano being arranged on piano Action acquisition module and piano data transmit-receive module, wherein:
The piano action acquisition module, for gathering piano action and the corresponding timestamp of piano action, and will collect Information piano data transmit-receive module is sent in the form of midi message;
The piano data transmit-receive module, for carrying out data interaction with mobile terminal data transceiver module;
The mobile terminal data transceiver module, for receiving midi message data processing module is sent to;
The audio-video collection module, for gathering audio/video information during piano performance, and is sent to data processing module;
The data processing module, for being generated as music text after synchronizing the midi for receiving message and audio/video information Part, is sent to data memory module;
The data memory module, for storing music file.
Preferably, it is described that data processing module is included into synchronizing information module and file generating module, wherein:
Described information synchronization module is relative for the timestamp by timestamp in midi message and audio/video information per frame data Should;
The file generating module, for the midi information after correspondence and audio/video information to be generated into audio/video information in front midi The posterior music file of message.
Preferably, described information synchronization module includes coarse adjustment module, for determining the time for receiving first midi message The overall offset amount of the timestamp of the initial time of stamp and audio/video information.
Preferably, described information synchronization module also includes fine setting module, for determine to follow-up certain midi information when Between stamp and its corresponding audio/video information timestamp fine setting side-play amount.
Preferably, the data processing module also includes document analysis module, for accessing data memory module, by music File is decomposed into audio/video information and midi information.
Preferably, the sychronisation also includes the audio and video playing module being arranged on mobile terminal and is arranged at piano On piano control instruction performing module, wherein:
The audio and video playing module is connected with data processing module, for receiving the audio/video information of document analysis module and broadcasting Put;
The piano control instruction performing module is connected with piano data transmit-receive module, and document analysis module leads to midi message Cross mobile terminal data transceiver module and be sent to piano data transmit-receive module, the piano control instruction performing module receives piano Data transmit-receive module receives midi message, and is controlled playing music to piano.
Preferably, the coarse adjustment module, is additionally operable to determine the broadcasting initial time of midi message earlier than audio/video information Play initial time.
Preferably, the fine setting module, is additionally operable to the performance of the piano when piano control instruction performing module control Action and audio/video information not to it is corresponding when determine that audio/video information is played and suspend the stand-by period.
Preferably, the audio and video playing module with 24 frames/point speed playing audio-video information.
An additional aspect of the present invention, also provides a kind of audio frequency and video and piano action synchronous method, comprises the following steps:
Step 1:Collection piano action and the corresponding timestamp of piano action, and by the information for collecting with midi message Form is sent to mobile terminal;
Step 2:Audio/video information during mobile terminal collection piano performance;
Step 3:Mobile terminal is generated as music file and goes forward side by side after synchronizing the midi message for receiving and audio/video information Row storage.
The audio frequency and video that the present invention is provided and piano action sychronisation and method, generate in collection audio frequency and video and piano information During automatic Playing file, it is ensured that audio frequency and video are synchronous with piano action, so as to improve the sight of automatic playing system.
Description of the drawings
A kind of audio frequency and video and the structural representation of piano action sychronisation that Fig. 1 is provided for the present invention;
A kind of structural representation of data processing module that Fig. 2 is provided for the present invention;
A kind of audio frequency and video and the flow chart of piano action synchronous method that Fig. 3 is provided for the present invention.
Specific embodiment
In order that those skilled in the art more fully understand technical scheme, below in conjunction with the accompanying drawings to the present invention It is described in further detail.
The structural representation of a kind of audio frequency and video and piano action sychronisation that provide for the present invention referring to Fig. 1 and Fig. 2, Fig. 1 Figure, a kind of structural representation of data processing module that Fig. 2 is provided for the present invention.
The present invention provides a kind of audio frequency and video and piano action sychronisation, regards including the sound being arranged in mobile terminal 1 00 Frequency acquisition module 130, data processing module 120, data memory module 140 and mobile terminal data transceiver module 110, Yi Jishe The piano action acquisition module 230 being placed on piano 200 and piano data transmit-receive module 210, wherein:
The piano action acquisition module 230, for gathering piano action and the corresponding timestamp of piano action, and will collection To information piano data transmit-receive module 210 is sent in the form of midi message;Piano data transmit-receive module 210, for Mobile terminal data transceiver module 110 carries out data interaction;Mobile terminal data transceiver module 110, for midi will to be received Message is sent to data processing module 120;Audio-video collection module 130, for gathering audio/video information during piano performance, and It is sent to data processing module 120;Data processing module 120, the midi message and audio/video information for receiving is carried out Music file is generated as after synchronization, data memory module 140 is sent to;Data memory module 140, for storing music file.
When audio frequency and video are gathered and piano information generates automatic Playing file, player's playing piano, piano action collection Module 230, by collection piano action and the corresponding timestamp of piano action, generates the midi message with timestamp and is sent to Piano data transmit-receive module 210;Piano data transmit-receive module 210 carries out data interaction with mobile terminal data transceiver module 110, Midi message with timestamp is sent to into data processing module 120;At the same time, the audio-video collection mould on mobile terminal Block 130, gathers the sound and picture when player's playing piano is played, and generates audio/video information, and is sent to data processing mould Block 120;Data processing module 120, it is raw after the midi message with timestamp for receiving and audio/video information are synchronized Become music file, be sent to data memory module 140;Data memory module 140, for storing music file.
When audio frequency and video are gathered and piano information generates automatic Playing file, ensure that audio frequency and video are same with piano action Step, so as to provide synchronism for the performance of follow-up automatic playing system file is preferably played, and improves the sight of automatic playing system Reward property.
Referring to Fig. 2, below will be to further being told about in data processing module.
Data processing module 120 includes synchronizing information module 121 and file generating module 122, wherein:Described information synchronization Module 121, it is corresponding for the timestamp by timestamp in midi message with audio/video information per frame data;The file generated Module 122, for the midi information after correspondence and audio/video information to be generated into audio/video information in the posterior sound of front midi message Music file.The music file includes audio-video document and midi files, and audio/video information in front midi message rear.
In further scheme, described information synchronization module 121 includes coarse adjustment module 1211, and coarse adjustment module 1211 is first Receive audio/video information and determine that the initial time of audio/video information is 0 point of file, then coarse adjustment module 1211 receives first The individual midi message with timestamp, may thereby determine that the initial time timestamp of midi message and the starting of audio/video information The overall offset amount of moment timestamp.After determining midi message and audio-visual message initial time, follow-up audio frequency and video disappear The timestamp of breath correspondence corresponding with work by the timestamp of midi message.
In further scheme, described information synchronization module 121 also includes fine setting module 1212, finely tunes module 1212 The waveform of midi message and audio/video information can be launched, for the timestamp and its corresponding audio frequency and video of certain midi information The setting side-play amount for shifting of information time stamp is finely adjusted.
Data processing module 120 also includes document analysis module 123, for accessing data memory module 140, by music text Part is decomposed into audio/video information and midi information.The sychronisation also includes that the audio frequency and video being arranged in mobile terminal 1 00 are broadcast Amplification module 150 and the piano control instruction performing module 220 being arranged on piano 200, wherein:Audio and video playing module 150 with Data processing module 120 connects, and the audio/video information for receiving data processing module 120 is simultaneously played;Piano control instruction is held Row module 220 is connected with piano data transmit-receive module 210, and midi message is passed through mobile terminal number by data processing module 120 Piano data transmit-receive module 210 is sent to according to transceiver module 110, the piano control instruction performing module 220 receives piano number Midi message is received according to transceiver module 210, and playing music is controlled to piano.
When automatic Playing file and playing music is played, document analysis module 123 accesses data memory module 140, music file is decomposed into into audio/video information and midi information.The audio and video playing module 150 of mobile terminal 1 00 receives text The audio/video information of part parsing module 123 is simultaneously played;Piano control instruction performing module 220 and piano data transmit-receive module 210 It is connected, midi message is sent to piano data transmit-receive by document analysis module 123 by mobile terminal data transceiver module 110 Module 210, piano control instruction performing module 220 receives piano data transmit-receive module 210 and receives midi message, and to piano It is controlled playing music.
Because just to audio/video information and midi information having done synchronization when music file is recorded, hereby it is ensured that music text Synchronization of the part in automatic Playing.
Consider that piano data transmit-receive module 210 receives midi message and has the regular hour poor, add piano control instruction Performing module 220 has buffer time receiving midi message, so that determining midi message by coarse adjustment module 1211 Play time difference of the initial time earlier than the broadcasting initial time of audio/video information.
The performance action in the piano when piano control instruction performing module control and sound during automatic Playing Video information does not suspend the stand-by period to determining that audio/video information is played by fine setting module 1212 during correspondence, fixes tentatively the stand-by period Generally 80ms.
Preferably, the audio and video playing module 150 with 24 frames/point speed playing audio-video information.
Referring to Fig. 3, a kind of audio frequency and video and the flow chart of piano action synchronous method that Fig. 3 is provided for the present invention.
An additional aspect of the present invention, also provides a kind of audio frequency and video and piano action synchronous method, comprises the following steps:
Step 1:Collection piano action and the corresponding timestamp of piano action, and by the information for collecting with midi message Form is sent to mobile terminal;
Step 2:Audio/video information during mobile terminal collection piano performance;
Step 3:Mobile terminal is generated as music file and goes forward side by side after synchronizing the midi message for receiving and audio/video information Row storage.
When audio frequency and video are gathered and piano information generates automatic Playing file, player's playing piano gathers piano action And the corresponding timestamp of piano action, generate the midi message with timestamp and be sent to mobile terminal;At the same time, it is mobile Sound and picture when terminal collection player playing piano is played, generates audio/video information.Mobile terminal is by the band for receiving The midi message and audio/video information for having timestamp is generated as music file and is stored after synchronizing.
When audio frequency and video are gathered and piano information generates automatic Playing file, ensure that audio frequency and video are same with piano action Step, so as to provide synchronism for the performance of follow-up automatic playing system file is preferably played, and improves the sight of automatic playing system Reward property.
A kind of audio frequency and video provided by the present invention and piano action sychronisation and method are described in detail above. Specific case used herein is set forth to the principle and embodiment of the present invention, and the explanation of above example is simply used Understand the core concept of the present invention in help.It should be pointed out that for those skilled in the art, without departing from On the premise of the principle of the invention, some improvement and modification can also be carried out to the present invention, these are improved and modification also falls into this In bright scope of the claims.

Claims (10)

1. a kind of audio frequency and video and piano action sychronisation, it is characterised in that include:The audio frequency and video being arranged on mobile terminal are adopted Collection module, data processing module, data memory module and mobile terminal data transceiver module, and the piano being arranged on piano Action acquisition module and piano data transmit-receive module, wherein:
The piano action acquisition module, for gathering piano action and the corresponding timestamp of piano action, and will collect Information piano data transmit-receive module is sent in the form of midi message;
The piano data transmit-receive module, for carrying out data interaction with mobile terminal data transceiver module;
The mobile terminal data transceiver module, for receiving midi message data processing module is sent to;
The audio-video collection module, for gathering audio/video information during piano performance, and is sent to data processing module;
The data processing module, for being generated as music text after synchronizing the midi for receiving message and audio/video information Part, is sent to data memory module;
The data memory module, for storing music file.
2. audio frequency and video according to claim 1 and piano action sychronisation, it is characterised in that described by data processing mould Block includes synchronizing information module and file generating module, wherein:
Described information synchronization module is relative for the timestamp by timestamp in midi message and audio/video information per frame data Should;
The file generating module, for the midi information after correspondence and audio/video information to be generated into audio/video information in front midi The posterior music file of message.
3. audio frequency and video according to claim 2 and piano action sychronisation, it is characterised in that described information synchronization module Including coarse adjustment module, for determining the timestamp for receiving the timestamp of first midi message and the initial time of audio/video information Overall offset amount.
4. audio frequency and video according to claim 3 and piano action sychronisation, it is characterised in that described information synchronization module Also include fine setting module, for determining the timestamp to follow-up certain midi information and its corresponding audio/video information timestamp Fine setting side-play amount.
5. the audio frequency and video according to claim 2 to 4 and piano action sychronisation, it is characterised in that the data processing Module also includes document analysis module, for accessing data memory module, music file is decomposed into into audio/video information and midi Information.
6. audio frequency and video according to claim 5 and piano action sychronisation, it is characterised in that the sychronisation is also wrapped The audio and video playing module being arranged on mobile terminal and the piano control instruction performing module being arranged on piano are included, wherein:
The audio and video playing module is connected with data processing module, for receiving the audio/video information of document analysis module and broadcasting Put;
The piano control instruction performing module is connected with piano data transmit-receive module, and document analysis module leads to midi message Cross mobile terminal data transceiver module and be sent to piano data transmit-receive module, the piano control instruction performing module receives piano Data transmit-receive module receives midi message, and is controlled playing music to piano.
7. audio frequency and video according to claim 6 and piano action sychronisation, it is characterised in that the coarse adjustment module, also For determining the broadcasting initial time of the broadcasting initial time earlier than audio/video information of midi message.
8. audio frequency and video according to claim 7 and piano action sychronisation, it is characterised in that the fine setting module, also For when the performance action of the piano that the piano control instruction performing module is controlled and audio/video information not to it is corresponding when determine sound Video information is played and suspends the stand-by period.
9. audio frequency and video according to claim 8 and piano action sychronisation, it is characterised in that the audio and video playing mould Block with 24 frames/point speed playing audio-video information.
10. a kind of audio frequency and video and piano action synchronous method, it is characterised in that the method comprising the steps of:
Step 1:Collection piano action and the corresponding timestamp of piano action, and by the information for collecting with midi message Form is sent to mobile terminal;
Step 2:Audio/video information during mobile terminal collection piano performance;
Step 3:Mobile terminal is generated as music file and goes forward side by side after synchronizing the midi message for receiving and audio/video information Row storage.
CN201611198031.8A 2016-12-22 2016-12-22 Audio and video and piano action synchronization device and method Pending CN106601218A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611198031.8A CN106601218A (en) 2016-12-22 2016-12-22 Audio and video and piano action synchronization device and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611198031.8A CN106601218A (en) 2016-12-22 2016-12-22 Audio and video and piano action synchronization device and method

Publications (1)

Publication Number Publication Date
CN106601218A true CN106601218A (en) 2017-04-26

Family

ID=58602593

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611198031.8A Pending CN106601218A (en) 2016-12-22 2016-12-22 Audio and video and piano action synchronization device and method

Country Status (1)

Country Link
CN (1) CN106601218A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109413476A (en) * 2018-10-17 2019-03-01 湖南乐和云服网络科技有限公司 A kind of audio-video and piano action live broadcasting method and system
CN110570832A (en) * 2019-09-29 2019-12-13 湖北美和易思教育科技有限公司 internet-of-things air electronic organ

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101004865A (en) * 2006-01-17 2007-07-25 雅马哈株式会社 Music performance system,music stations synchronized with one another and computer program used therein
CN101145288A (en) * 2007-08-03 2008-03-19 上海欧亚钢琴乐器有限公司 Piano remote-distance teaching system based on broad band internet
CN103403794A (en) * 2011-01-07 2013-11-20 雅马哈株式会社 Automatic musical performance device
CN104796471A (en) * 2015-04-20 2015-07-22 宜昌金宝乐器制造有限公司 Music radio station for automatic playing piano
CN105895066A (en) * 2015-08-10 2016-08-24 杨飞 Digitalized piano capable of network demonstration piano performing and real time video teaching

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101004865A (en) * 2006-01-17 2007-07-25 雅马哈株式会社 Music performance system,music stations synchronized with one another and computer program used therein
CN101145288A (en) * 2007-08-03 2008-03-19 上海欧亚钢琴乐器有限公司 Piano remote-distance teaching system based on broad band internet
CN103403794A (en) * 2011-01-07 2013-11-20 雅马哈株式会社 Automatic musical performance device
CN104796471A (en) * 2015-04-20 2015-07-22 宜昌金宝乐器制造有限公司 Music radio station for automatic playing piano
CN105895066A (en) * 2015-08-10 2016-08-24 杨飞 Digitalized piano capable of network demonstration piano performing and real time video teaching

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109413476A (en) * 2018-10-17 2019-03-01 湖南乐和云服网络科技有限公司 A kind of audio-video and piano action live broadcasting method and system
CN110570832A (en) * 2019-09-29 2019-12-13 湖北美和易思教育科技有限公司 internet-of-things air electronic organ

Similar Documents

Publication Publication Date Title
CN101257582B (en) Media generation system
CN103403794B (en) Automatic musical performance device
JP5729393B2 (en) Performance system
CN101303880A (en) Method and apparatus for recording and playing audio-video document
US7622664B2 (en) Performance control system, performance control apparatus, performance control method, program for implementing the method, and storage medium storing the program
US20130219444A1 (en) Receiving apparatus and subtitle processing method
CN1960485B (en) Method for playing back video and audio synchronistically in mobile media broadcast
CN106993239A (en) Method for information display during live
CN101119461B (en) System and method for maintaining video frame and audio frame synchronous broadcasting
EP2328140B1 (en) Storage system and storage device of music files
CN1972407A (en) A video and audio synchronization playing method for mobile multimedia broadcasting
CN112437336A (en) Audio and video playing method and device, electronic equipment and storage medium
CN106601218A (en) Audio and video and piano action synchronization device and method
CN106782459B (en) Piano automatic playing control system and method based on mobile terminal application program
CN109949792B (en) Multi-audio synthesis method and device
CN101916577B (en) The method and device that a kind of audio and video playing synchronizes
CN107135407B (en) Synchronous method and system in a kind of piano video teaching
JP2006020102A (en) Broadcast recording/reproducing device and broadcast recording/reproducing processing program
CN107911740A (en) A kind of method and device of the sound collecting based on video playing
CN111540377B (en) System for intelligent fragmentation of broadcast program
JP2008219608A (en) Method and apparatus for displaying information with content, program therefor and recording medium
KR102524870B1 (en) Device for serving dance image based on midi sound and method for the same, and recording medium readable by computing device
JP7051923B2 (en) Video generator, video generation method, video generator, playback device, video distribution device, and video system
JP5687961B2 (en) Synchronous playback apparatus and synchronous playback method
JP2011205439A (en) Reproducing apparatus and method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20170426