CN106162293A - A kind of video sound and the method and device of image synchronization - Google Patents
A kind of video sound and the method and device of image synchronization Download PDFInfo
- Publication number
- CN106162293A CN106162293A CN201510195494.8A CN201510195494A CN106162293A CN 106162293 A CN106162293 A CN 106162293A CN 201510195494 A CN201510195494 A CN 201510195494A CN 106162293 A CN106162293 A CN 106162293A
- Authority
- CN
- China
- Prior art keywords
- sound
- video
- image
- audio frequency
- section
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/4424—Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The invention discloses the method and device of a kind of video sound and image synchronization, in order to reduce sound and the workload in image synchronization operating process, save the working time, improve operating efficiency.Method includes: the reproduction time according to currently playing sound and the sound time section dividing in advance, sound time section belonging to reproduction time for the detection；Transfer the key frame of video associating in advance with sound time section, and in the sound playing process of the sound time section of audio frequency, show the image of key frame of video.The program makes automatically to show corresponding image while sound is play, when there is sound and image not corresponding situation in order, can directly be adjusted on the track of audio frequency, go to play the corresponding audio file of clicked on image without clicking on image one by one, simplify operating process, and the synchro edit process for sound and image saved the time, improve editorial efficiency.
Description
Technical field
The present invention relates to Internet technical field, particularly to a kind of video sound and image synchronization method and
Device.
Background technology
With scientific and technical development, the information data of the multimedia forms such as video is more and more abundanter, and thereafter
The workload of platform data edition is also increasing, especially in the industries such as time-bounded very strong news, for
The accuracy of video data editing and editorial efficiency require very high.During video data editing, video
The synchronization of sound and image is necessary problem to be processed, and in prior art solutions, due to editor system
Audio file in system be typically once load after begin to play, and in video extract image when
Between and the time not automatic aligning of audio frequency, therefore when automatically playing, the sound of broadcasting and display
Image is frequently not and belongs to section at the same time；And when being synchronized editor, only when editorial staff clicks
Image just can trigger the audio frequency in the corresponding period, so needs the editorial staff audio frequency and video are divided into one section one section
After checking one by one, just can find nonsynchronous position and edlin is entered to it, operate sufficiently complex and
Inefficiency, operating time are long.
Content of the invention
The present invention provides the method and device of a kind of video sound and image synchronization, in order to reduce sound and image
Workload during simultaneously operating, saves the working time, improves operating efficiency.
First aspect according to embodiments of the present invention, provides a kind of method of video sound and image synchronization, bag
Include:
After receiving synchro edit instruction, when detecting broadcasting in the audio frequency play for the currently playing sound
Between；
According to the sound time section dividing in advance in described reproduction time and described audio frequency, detect described broadcasting
Time sound time section affiliated in described audio frequency；
Transfer the key frame of video associating in advance with the described sound time section in described audio frequency, and described
The sound playing process of the described sound time section of audio frequency shows the image of described key frame of video.
In one embodiment, described according to the sound dividing in advance in described reproduction time and described audio frequency when
Between section, before detecting described reproduction time sound time section affiliated in described audio frequency, comprising:
Key frame of video, and each described Video Key of corresponding storage is extracted in video corresponding with described audio frequency
Frame image temporal section affiliated when playing in described video；
According to each described key frame of video image temporal section affiliated in described video, by described audio frequency
Reproduction time is divided into and multiple sound time sections so that multiple described sound time sections are described with multiple
Image temporal section one_to_one corresponding；
Would correspond to the sound time section of each described image temporal section and corresponding to each described image temporal
Storage after the key frame of video association of section.
In one embodiment, described transferring associates in advance with the described sound time section in described audio frequency
Key frame of video, and regard described in display in the sound playing process of the described sound time section of described audio frequency
Frequently the image of key frame, comprising:
When the sound time section affiliated in described audio frequency of described reproduction time being detected, transfer with described
The key frame of video that described sound time section in audio frequency associates in advance；
The image of described key frame of video is shown in the display interface that described audio frequency is play.
In one embodiment, described transferring associates in advance with the described sound time section in described audio frequency
Key frame of video, and regard described in display in the sound playing process of the described sound time section of described audio frequency
Frequently, after the image of key frame, also include:
In real time the currently playing sound time section belonging to sound of the described audio frequency of detection whether with currently displaying
Image temporal section coupling belonging to image；
At the sound time section belonging to currently playing sound and the image temporal belonging to currently displaying image
When section does not mates, transfer and close with the video that the sound time section belonging to currently playing sound associates in advance
Key frame, and in the display interface that described audio frequency is play, show the image of described key frame of video；
At the sound time section belonging to currently playing sound and the image temporal belonging to currently displaying image
During section coupling, continue to show currently displaying image in the display interface that described audio frequency is play.
In one embodiment, described transferring associates in advance with the described sound time section in described audio frequency
Key frame of video, and simultaneous display institute in the sound playing process of the described sound time section of described audio frequency
After stating the image of key frame of video, also include:
When currently playing sound occurs asynchronous with currently displaying image, belonging to currently playing sound
The initial time of sound time section or/and intermission node is adjusted.
Second aspect according to embodiments of the present invention, also provides the device of a kind of video sound and image synchronization,
Including:
First detection module, after being used for receiving synchro edit instruction, detects currently playing sound and broadcasts
Reproduction time in the audio frequency put；
Second detection module, for according to the sound time dividing in advance in described reproduction time and described audio frequency
Section, detects described reproduction time sound time section affiliated in described audio frequency；
Simultaneous display module, for transferring and regarding that the described sound time section in described audio frequency associates in advance
Frequently key frame, and show described video in the sound playing process of the described sound time section of described audio frequency
The image of key frame.
In one embodiment, described device also includes:
Key-frame extraction module, is used for certainly extracting in corresponding video key frame of video with described audio frequency, and right
Image temporal section affiliated when each described key frame of video is play in described video should be stored；
Sound time section partition module, for according to each described key frame of video in described video affiliated
The reproduction time of described audio frequency is divided into and multiple sound time sections so that multiple by image temporal section
Described sound time section and multiple described image temporal section one_to_one corresponding；
Association memory module, for would correspond to the sound time section of each described image temporal section with corresponding
Storage after the key frame of video of each described image temporal section associates.
In one embodiment, described simultaneous display module includes:
Key frame transfers submodule, for the sound affiliated in described audio frequency of described reproduction time being detected
During time section, transfer the key frame of video associating in advance with the described sound time section in described audio frequency；
Simultaneous display submodule, for showing described key frame of video in the display interface that described audio frequency is play
Image.
In one embodiment, described device also includes:
Real-time detection module, for the currently playing sound time district belonging to sound of the described audio frequency of real-time detection
Whether section mates with the image temporal section belonging to currently displaying image；
First display module, at the sound time section belonging to currently playing sound and currently displaying figure
When not mating as affiliated image temporal section, transfer and the sound time section belonging to currently playing sound
The key frame of video associating in advance, and show described key frame of video in the display interface that described audio frequency is play
Image；
Second display module, at the sound time section belonging to currently playing sound and currently displaying figure
When mating as affiliated image temporal section, continue display in the display interface that described audio frequency is play current aobvious
Diagram picture.
In one embodiment, described device also includes:
Adjusting module, for when currently playing sound occurs asynchronous with currently displaying image, to currently
The initial time of the sound time section belonging to sound play is or/and intermission node is adjusted.
The technical scheme that the embodiment of the present invention provides can produce following beneficial effect: receives synchro edit instruction
After, detect reproduction time in the audio frequency play for the currently playing sound；According to described reproduction time and
The sound time section dividing in advance in described audio frequency, detects described reproduction time affiliated in described audio frequency
Sound time section；Transfer the Video Key associating in advance with the described sound time section in described audio frequency
Frame, and show described key frame of video in the sound playing process of the described sound time section of described audio frequency
Image.The sound time section of the image of the key frame of video of extraction and the audio frequency of broadcasting is carried out by the program
Association so that automatically show corresponding image while sound is play, order occurs sound with image not
During corresponding situation, directly can be adjusted on the track of audio frequency, go without clicking on image one by one
Play the corresponding audio file of clicked on image, simplify operating process, and the synchronization for sound and image is compiled
The process of collecting has saved the time, improves editorial efficiency.
Other features and advantages of the present invention will illustrate in the following description, and, partly from explanation
Book becomes apparent, or is understood by implementing the present invention.The purpose of the present invention and other advantages can
Realized by structure specifically noted in the specification write, claims and accompanying drawing and obtain
?.
Below by drawings and Examples, technical scheme is described in further detail.
Brief description
Accompanying drawing is used for providing a further understanding of the present invention, and constitutes a part for specification, with this
Bright embodiment is used for explaining the present invention together, is not intended that limitation of the present invention.
In the accompanying drawings:
Fig. 1 is the method for a kind of video sound according to an exemplary embodiment for the present invention and image synchronization
Flow chart；
Fig. 2 is the side of another kind of video sound according to an exemplary embodiment for the present invention and image synchronization
The flow chart of method；
Fig. 3 is video sound according to an exemplary embodiment for the present invention and step in the method for image synchronization
The embodiment flow chart of rapid S30；
Fig. 4 is the side of the present invention another video sound according to an exemplary embodiment and image synchronization
The flow chart of method；
Fig. 5 is the side of the present invention another video sound according to an exemplary embodiment and image synchronization
The flow chart of method；
Fig. 6 is the device of a kind of video sound according to an exemplary embodiment for the present invention and image synchronization
Block diagram；
Fig. 7 is the dress of another kind of video sound according to an exemplary embodiment for the present invention and image synchronization
The block diagram put；
Fig. 8 is that video sound according to an exemplary embodiment for the present invention is same with the device of image synchronization
The block diagram of step display module 63；
Fig. 9 is the dress of the present invention another video sound according to an exemplary embodiment and image synchronization
The block diagram put；
Figure 10 is the present invention another video sound according to an exemplary embodiment and image synchronization
The block diagram of device.
Detailed description of the invention
Below in conjunction with accompanying drawing, the preferred embodiments of the present invention are illustrated, it will be appreciated that described herein
Preferred embodiment is merely to illustrate and explains the present invention, is not intended to limit the present invention.
Disclosure embodiment provides a kind of method of video sound and image synchronization, is used for reducing sound and figure
As the workload during simultaneously operating, save the working time, improve operating efficiency.As it is shown in figure 1, should
Method includes step S10-S30:
In step slo, after receiving synchro edit instruction, detect currently playing sound and playing
Reproduction time in audio frequency.That is, when audio frequency starts loaded and commences play out, then initially receive
Synchro edit instructs, and now detects concrete broadcasting in present video for the currently playing sound of described audio frequency
Time.
In step S20, according to the sound time section dividing in advance in described reproduction time and described audio frequency,
Detect described reproduction time sound time section affiliated in described audio frequency.Institute detected in step slo
After stating concrete reproduction time in present video for the currently playing sound of audio frequency, it is judged that during described broadcasting
Between sound time section affiliated in described audio frequency, that is, which in described audio frequency this reproduction time belong to
After one sound time section is determined, it is possible to use this sound time section is by currently playing sound and figure
As simultaneous display after mating.Wherein said sound time section divides in described audio frequency in advance,
And in advance divide after sound time section be with video in extract key frame of video belonging to image when
Between section one to one.
In step s 30, transfer and the video pass that the described sound time section in described audio frequency associates in advance
Key frame, and show described Video Key in the sound playing process of the described sound time section of described audio frequency
The image of frame.Wherein, the key frame of video extracting in the described sound time section in described audio frequency and video
Associate in advance, when therefore the sound at the described sound time section of described audio frequency is play, can set
Automatically the image of display key frame of video associated with it.
In one embodiment, as it is shown on figure 3, described step S30 mays include:
Step S301, when the sound time section affiliated in described audio frequency of described reproduction time being detected,
Transfer the key frame of video associating in advance with the described sound time section in described audio frequency；That is, described sound
Described sound time section in Pin associates in advance with the key frame of video extracting in video, therefore in institute
When the sound of the described sound time section stating audio frequency is play, first find the described Video Key associating in advance
Frame, and it is taken out from database, it is shown in afterwards on the interface that described audio frequency is play, in order to
Editorial staff checks sound described in comparison and image whether simultaneously match.
Step S302, the image showing described key frame of video in the display interface that described audio frequency is play.Institute
The track stating image and described sound shows in same display interface, and both be preferably formed as one mutual right
The display relation answered, such as, when being shown in sound currently playing in the track of described audio frequency by described image
Between the either above or below of section, form a corresponding relation, in order to editorial staff checks sound described in comparison
Sound and image whether simultaneously match.
In one embodiment, as in figure 2 it is shown, also include step S40-S60 before step S20 of the method:
In step s 40, corresponding video extracts key frame of video certainly with described audio frequency, and corresponding storage
Each described key frame of video image temporal section affiliated when playing in described video.That is, enter at needs
The video of row audiovideo synchronization extracts key frame of video, and each described key frame of video is corresponding described respectively
Described time zone segment information and described key frame of video are stored by one of video image temporal section in the lump
In database, in order to transfer when needed.
In step s 50, according to the image temporal section that each described key frame of video is affiliated in described video,
The reproduction time of described audio frequency is divided into and multiple sound time sections so that multiple described sound time districts
Section and multiple described image temporal section one_to_one corresponding.That is, at extraction key frame of video and by itself and each institute
After stating the corresponding storage of image temporal section affiliated when key frame of video is play in described video, now will
Also divide out one by one in each image temporal section corresponding sound time section, during by multiple described sound
Between after section and multiple described image temporal section one_to_one corresponding, as long as playing the sound of audio frequency, will be by
Belonging to this sound, the image of the key frame of video of sound time section takes out and shows.
In step S60, would correspond to the sound time section of each described image temporal section and corresponding to respectively
Storage after the key frame of video association of described image temporal section.By described time zone segment information and described video
The described image temporal section correspondence of key frame is stored in database, in order to automatically adjust when sound is play
Take and show its corresponding image.
In one embodiment, as shown in Figure 4, step S70-S90 is also included after step S30 of the method:
In step S70, detecting the currently playing sound time section belonging to sound of described audio frequency in real time is
Image temporal section belonging to no and currently displaying image mates；That is, during audio frequency is play, when
After the sound of front sound time section finishes, due to currently displaying be a upper sound time section
The image of the key frame of video of association, it is therefore desirable to currently displaying image is replaced by currently playing sound in real time
The corresponding image of sound time section of sound, it is therefore desirable to carry out in real time detecting the currently playing sound of described audio frequency
Whether the sound time section belonging to sound mates with the image temporal section belonging to currently displaying image, in order to
Replace currently displaying image in real time.
In step S80, in the sound time section belonging to currently playing sound and currently displaying image institute
When the image temporal section belonging to does not mates, transfer with the sound time section belonging to currently playing sound in advance
The key frame of video of association, and in the display interface that described audio frequency is play, show the figure of described key frame of video
Picture；In the sound time section belonging to currently playing sound and the image temporal district belonging to currently displaying image
When section is not mated, need to be replaced by currently displaying image in real time the sound time section of currently playing sound
Corresponding image, accordingly, it would be desirable to cancel the display of present image, and by during with the sound of currently playing sound
Between the corresponding image of section take out and show in real time.
In step S90, in the sound time section belonging to currently playing sound and currently displaying image institute
During the image temporal section coupling belonging to, continue to show currently displaying figure in the display interface that described audio frequency is play
Picture.And at the sound time section belonging to currently playing sound and the image temporal belonging to currently displaying image
During section coupling, illustrate that the sound of current sound time section is play and not yet finish, proceed to play and aobvious
Show present image.
In one embodiment, as it is shown in figure 5, also include step S100 after step S30 of the method:
In the step s 100, when currently playing sound and currently displaying image occur asynchronous, to working as
The initial time of the sound time section belonging to the sound of front broadcasting is or/and intermission node is adjusted.
That is, when editorial staff finds that currently playing sound occurs asynchronous with currently displaying image, than
If, currently playing sound is not the speech of the speech personage of display in image, or its speech not with
The Hp-synchronization of speech personage, during at this time, it may be necessary to adjust the current sound time section of described audio frequency initial
Between or/and intermission node, namely on currently displaying track, above-mentioned timing node is adjusted, adjusts
Whole rear sound and image synchronization coupling are play, and owing to only timing node being adjusted, sound and image
Between corresponding relation do not change, therefore when the described sound of the audio frequency play after modification time node
Between section when, or can the image of the corresponding key frame of video of simultaneous display, also not interfere with right simultaneously
Follow-up synchronized relation is adjusted.
The said method that the embodiment of the present invention provides, after receiving synchro edit instruction, detects currently playing sound
Reproduction time in the audio frequency play for the sound；Divide in advance according in described reproduction time and described audio frequency
Sound time section, detect described reproduction time sound time section affiliated in described audio frequency；Transfer
The key frame of video associating in advance with the described sound time section in described audio frequency, and in the institute of described audio frequency
The sound playing process stating sound time section shows the image of described key frame of video.The program will be extracted
The image of key frame of video be associated with the sound time section of the audio frequency of broadcasting so that sound play
While automatically show corresponding image, when there is sound and image not corresponding situation in order, Ke Yizhi
It is connected on and is adjusted on the track of audio frequency, go the clicked on image of broadcasting corresponding without clicking on image one by one
Audio file, simplifies operating process, and the synchro edit process for sound and image has saved the time, carries
High editorial efficiency.
The video sound of corresponding embodiment of the present invention offer and the method for image synchronization, the present invention also provides video
Sound and the device of image synchronization, as shown in Figure 6, this device mays include:
First detection module 61, after being used for receiving synchro edit instruction, detects currently playing sound
Reproduction time in the audio frequency play；
Second detection module 62, for during according to the sound dividing in advance in described reproduction time and described audio frequency
Between section, detect described reproduction time sound time section affiliated in described audio frequency；
Simultaneous display module 63, associates with the described sound time section in described audio frequency in advance for transferring
Key frame of video, and regard described in display in the sound playing process of the described sound time section of described audio frequency
Frequently the image of key frame.
In one embodiment, as it is shown in fig. 7, described device also includes:
Key-frame extraction module 64, is used for certainly extracting in corresponding video key frame of video with described audio frequency, and
Corresponding storage each described key frame of video image temporal section affiliated when playing in described video；
Sound time section partition module 65, for according to each described key frame of video in described video affiliated
Image temporal section, the reproduction time of described audio frequency is divided into and multiple sound time sections so that many
Individual described sound time section and multiple described image temporal section one_to_one corresponding；
Association memory module 66, for would correspond to the sound time section of each described image temporal section with right
Should storage after the key frame of video of each described image temporal section associates.
In one embodiment, as shown in Figure 8, described simultaneous display module 63 includes:
Key frame transfers submodule 631, for detect described reproduction time in described audio frequency affiliated
During sound time section, transfer the Video Key associating in advance with the described sound time section in described audio frequency
Frame；
Simultaneous display submodule 632, for showing that described video closes in the display interface that described audio frequency is play
The image of key frame.
In one embodiment, as it is shown in figure 9, described device also includes:
Real-time detection module 67, for the currently playing sound time belonging to sound of the described audio frequency of real-time detection
Whether section mates with the image temporal section belonging to currently displaying image；
First display module 68, for the sound time section belonging to currently playing sound with currently displaying
When image temporal section belonging to image does not mates, transfer and the sound time district belonging to currently playing sound
The key frame of video that section associates in advance, and show described Video Key in the display interface that described audio frequency is play
The image of frame；
Second display module 69, for the sound time section belonging to currently playing sound with currently displaying
When image temporal section belonging to image mates, continue display in the display interface that described audio frequency is play current
Display image.
In one embodiment, as shown in Figure 10, described device also includes:
Adjusting module 70, for when currently playing sound and currently displaying image occur asynchronous, to working as
The initial time of the sound time section belonging to the sound of front broadcasting is or/and intermission node is adjusted.
Those skilled in the art are it should be appreciated that embodiments of the invention can be provided as method, system or meter
Calculation machine program product.Therefore, the present invention can use complete hardware embodiment, complete software implementation or knot
The form of the embodiment in terms of closing software and hardware.And, the present invention can use and wherein wrap one or more
Computer-usable storage medium containing computer usable program code (including but not limited to magnetic disc store and
Optical memory etc.) form of the upper computer program implemented.
The present invention is with reference to method according to embodiments of the present invention, equipment (system) and computer program product
The flow chart of product and/or block diagram describe.It should be understood that can by computer program instructions flowchart and
/ or block diagram in each flow process and/or the flow process in square frame and flow chart and/or block diagram and/
Or the combination of square frame.These computer program instructions can be provided to all-purpose computer, special-purpose computer, embed
The processor of formula processor or other programmable data processing device is to produce a machine so that by calculating
The instruction that the processor of machine or other programmable data processing device performs produces for realizing at flow chart one
The device of the function specified in individual flow process or multiple flow process and/or one square frame of block diagram or multiple square frame.
These computer program instructions may be alternatively stored in and computer or the process of other programmable datas can be guided to set
In the standby computer-readable memory working in a specific way so that be stored in this computer-readable memory
Instruction produce and include the manufacture of command device, this command device realizes in one flow process or multiple of flow chart
The function specified in flow process and/or one square frame of block diagram or multiple square frame.
These computer program instructions also can be loaded in computer or other programmable data processing device, makes
Sequence of operations step must be performed to produce computer implemented place on computer or other programmable devices
Reason, thus the instruction performing on computer or other programmable devices provides for realizing flow chart one
The step of the function specified in flow process or multiple flow process and/or one square frame of block diagram or multiple square frame.
Obviously, those skilled in the art can carry out various change and modification without deviating from this to the present invention
The spirit and scope of invention.So, if these modifications of the present invention and modification belong to the claims in the present invention
And within the scope of equivalent technologies, then the present invention is also intended to comprise these changes and modification.
Claims (10)
1. the method for a video sound and image synchronization, it is characterised in that include:
After receiving synchro edit instruction, when detecting broadcasting in the audio frequency play for the currently playing sound
Between；
According to the sound time section dividing in advance in described reproduction time and described audio frequency, detect described broadcasting
Time sound time section affiliated in described audio frequency；
Transfer the key frame of video associating in advance with the described sound time section in described audio frequency, and described
The sound playing process of the described sound time section of audio frequency shows the image of described key frame of video.
2. the method for claim 1, it is characterised in that described according to described reproduction time and institute
State the sound time section dividing in advance in audio frequency, detect described reproduction time sound affiliated in described audio frequency
Before sound time section, comprising:
Key frame of video, and each described Video Key of corresponding storage is extracted in video corresponding with described audio frequency
Frame image temporal section affiliated when playing in described video；
According to each described key frame of video image temporal section affiliated in described video, by described audio frequency
Reproduction time is divided into and multiple sound time sections so that multiple described sound time sections are described with multiple
Image temporal section one_to_one corresponding；
Would correspond to the sound time section of each described image temporal section and corresponding to each described image temporal
Storage after the key frame of video association of section.
3. method as claimed in claim 2, it is characterised in that described transfer and the institute in described audio frequency
State the key frame of video that sound time section associates in advance, and at the described sound time section of described audio frequency
Sound playing process shows the image of described key frame of video, comprising:
When the sound time section affiliated in described audio frequency of described reproduction time being detected, transfer with described
The key frame of video that described sound time section in audio frequency associates in advance；
The image of described key frame of video is shown in the display interface that described audio frequency is play.
4. method as claimed in claim 2, it is characterised in that described transfer and the institute in described audio frequency
State the key frame of video that sound time section associates in advance, and at the described sound time section of described audio frequency
After sound playing process shows the image of described key frame of video, also include:
In real time the currently playing sound time section belonging to sound of the described audio frequency of detection whether with currently displaying
Image temporal section coupling belonging to image；
At the sound time section belonging to currently playing sound and the image temporal belonging to currently displaying image
When section does not mates, transfer and close with the video that the sound time section belonging to currently playing sound associates in advance
Key frame, and in the display interface that described audio frequency is play, show the image of described key frame of video；
At the sound time section belonging to currently playing sound and the image temporal belonging to currently displaying image
During section coupling, continue to show currently displaying image in the display interface that described audio frequency is play.
5. the method for claim 1, it is characterised in that described transfer and the institute in described audio frequency
State the key frame of video that sound time section associates in advance, and at the described sound time section of described audio frequency
In sound playing process key frame of video described in simultaneous display image after, also include:
When currently playing sound occurs asynchronous with currently displaying image, belonging to currently playing sound
The initial time of sound time section or/and intermission node is adjusted.
6. the device of a video sound and image synchronization, it is characterised in that include:
First detection module, after being used for receiving synchro edit instruction, detects currently playing sound and broadcasts
Reproduction time in the audio frequency put；
Second detection module, for according to the sound time dividing in advance in described reproduction time and described audio frequency
Section, detects described reproduction time sound time section affiliated in described audio frequency；
Simultaneous display module, for transferring and regarding that the described sound time section in described audio frequency associates in advance
Frequently key frame, and show described video in the sound playing process of the described sound time section of described audio frequency
The image of key frame.
7. device as claimed in claim 6, it is characterised in that described device also includes:
Key-frame extraction module, is used for certainly extracting in corresponding video key frame of video with described audio frequency, and right
Image temporal section affiliated when each described key frame of video is play in described video should be stored；
Sound time section partition module, for according to each described key frame of video in described video affiliated
The reproduction time of described audio frequency is divided into and multiple sound time sections so that multiple by image temporal section
Described sound time section and multiple described image temporal section one_to_one corresponding；
Association memory module, for would correspond to the sound time section of each described image temporal section with corresponding
Storage after the key frame of video of each described image temporal section associates.
8. device as claimed in claim 7, it is characterised in that described simultaneous display module includes:
Key frame transfers submodule, for the sound affiliated in described audio frequency of described reproduction time being detected
During time section, transfer the key frame of video associating in advance with the described sound time section in described audio frequency；
Simultaneous display submodule, for showing described key frame of video in the display interface that described audio frequency is play
Image.
9. device as claimed in claim 7, it is characterised in that described device also includes:
Real-time detection module, for the currently playing sound time district belonging to sound of the described audio frequency of real-time detection
Whether section mates with the image temporal section belonging to currently displaying image；
First display module, at the sound time section belonging to currently playing sound and currently displaying figure
When not mating as affiliated image temporal section, transfer and the sound time section belonging to currently playing sound
The key frame of video associating in advance, and show described key frame of video in the display interface that described audio frequency is play
Image；
Second display module, at the sound time section belonging to currently playing sound and currently displaying figure
When mating as affiliated image temporal section, continue display in the display interface that described audio frequency is play current aobvious
Diagram picture.
10. device as claimed in claim 6, it is characterised in that described device also includes:
Adjusting module, for when currently playing sound occurs asynchronous with currently displaying image, to currently
The initial time of the sound time section belonging to sound play is or/and intermission node is adjusted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510195494.8A CN106162293B (en) | 2015-04-22 | 2015-04-22 | A kind of method and device of video sound and image synchronization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510195494.8A CN106162293B (en) | 2015-04-22 | 2015-04-22 | A kind of method and device of video sound and image synchronization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106162293A true CN106162293A (en) | 2016-11-23 |
| CN106162293B CN106162293B (en) | 2019-11-08 |
Family
ID=57346946
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510195494.8A Expired - Fee Related CN106162293B (en) | 2015-04-22 | 2015-04-22 | A kind of method and device of video sound and image synchronization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106162293B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107895581A (en) * | 2017-11-16 | 2018-04-10 | 福建弧聚网络科技有限公司 | The method and device that a kind of image is combined with audio |
| CN108616768A (en) * | 2018-05-02 | 2018-10-02 | 腾讯科技（上海）有限公司 | Synchronous broadcast method, device, storage location and the electronic device of multimedia resource |
| CN109862384A (en) * | 2019-03-13 | 2019-06-07 | 北京河马能量体育科技有限公司 | A kind of audio-video automatic synchronous method and synchronization system |
| CN110035326A (en) * | 2019-04-04 | 2019-07-19 | 北京字节跳动网络技术有限公司 | Subtitle generation, the video retrieval method based on subtitle, device and electronic equipment |
| CN111953921A (en) * | 2020-08-14 | 2020-11-17 | 杭州视洞科技有限公司 | Display and interaction scheme of round-corner swimming lane |
| CN112653916A (en) * | 2019-10-10 | 2021-04-13 | 腾讯科技（深圳）有限公司 | Method and device for audio and video synchronization optimization |
| CN113225596A (en) * | 2021-04-28 | 2021-08-06 | 百度在线网络技术（北京）有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN113435294A (en) * | 2021-06-23 | 2021-09-24 | 中国矿业大学(北京) | Coal-rock interface positioning and identifying method based on image and sound fusion |
| WO2022166573A1 (en) * | 2021-02-04 | 2022-08-11 | 上海哔哩哔哩科技有限公司 | Reading processing method, apparatus and system |
| WO2022247014A1 (en) * | 2021-05-26 | 2022-12-01 | 深圳市沃特沃德信息有限公司 | Audio and video frame synchronization method and apparatus based on ink screen device, and computer device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080313683A1 (en) * | 2007-06-13 | 2008-12-18 | Minako Shimizu | Moving image communication device, moving image communication system and semiconductor integrated circuit used for communication of moving image |
| US20120081567A1 (en) * | 2010-09-30 | 2012-04-05 | Apple Inc. | Techniques for synchronizing audio and video data in an image signal processing system |
| CN103546662A (en) * | 2013-09-23 | 2014-01-29 | 浙江工业大学 | Audio and video synchronizing method in network monitoring system |
| CN103686337A (en) * | 2013-12-30 | 2014-03-26 | 深圳Tcl新技术有限公司 | Transmitting method and system of video image data |
| CN104104987A (en) * | 2014-07-29 | 2014-10-15 | 小米科技有限责任公司 | Method and device for synchronizing picture and sound in video playing |
-
2015
- 2015-04-22 CN CN201510195494.8A patent/CN106162293B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080313683A1 (en) * | 2007-06-13 | 2008-12-18 | Minako Shimizu | Moving image communication device, moving image communication system and semiconductor integrated circuit used for communication of moving image |
| US20120081567A1 (en) * | 2010-09-30 | 2012-04-05 | Apple Inc. | Techniques for synchronizing audio and video data in an image signal processing system |
| CN103546662A (en) * | 2013-09-23 | 2014-01-29 | 浙江工业大学 | Audio and video synchronizing method in network monitoring system |
| CN103686337A (en) * | 2013-12-30 | 2014-03-26 | 深圳Tcl新技术有限公司 | Transmitting method and system of video image data |
| CN104104987A (en) * | 2014-07-29 | 2014-10-15 | 小米科技有限责任公司 | Method and device for synchronizing picture and sound in video playing |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107895581A (en) * | 2017-11-16 | 2018-04-10 | 福建弧聚网络科技有限公司 | The method and device that a kind of image is combined with audio |
| CN113992638A (en) * | 2018-05-02 | 2022-01-28 | 腾讯科技（上海）有限公司 | Synchronous playing method and device of multimedia resources, storage position and electronic device |
| CN108616768A (en) * | 2018-05-02 | 2018-10-02 | 腾讯科技（上海）有限公司 | Synchronous broadcast method, device, storage location and the electronic device of multimedia resource |
| CN113992638B (en) * | 2018-05-02 | 2023-07-14 | 腾讯科技（上海）有限公司 | Synchronous playing method and device for multimedia resources, storage position and electronic device |
| CN108616768B (en) * | 2018-05-02 | 2021-10-15 | 腾讯科技（上海）有限公司 | Synchronous playing method and device of multimedia resources, storage position and electronic device |
| CN109862384A (en) * | 2019-03-13 | 2019-06-07 | 北京河马能量体育科技有限公司 | A kind of audio-video automatic synchronous method and synchronization system |
| CN110035326A (en) * | 2019-04-04 | 2019-07-19 | 北京字节跳动网络技术有限公司 | Subtitle generation, the video retrieval method based on subtitle, device and electronic equipment |
| CN112653916A (en) * | 2019-10-10 | 2021-04-13 | 腾讯科技（深圳）有限公司 | Method and device for audio and video synchronization optimization |
| CN112653916B (en) * | 2019-10-10 | 2023-08-29 | 腾讯科技（深圳）有限公司 | Method and equipment for synchronously optimizing audio and video |
| CN111953921A (en) * | 2020-08-14 | 2020-11-17 | 杭州视洞科技有限公司 | Display and interaction scheme of round-corner swimming lane |
| WO2022166573A1 (en) * | 2021-02-04 | 2022-08-11 | 上海哔哩哔哩科技有限公司 | Reading processing method, apparatus and system |
| CN113225596B (en) * | 2021-04-28 | 2022-11-01 | 百度在线网络技术（北京）有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN113225596A (en) * | 2021-04-28 | 2021-08-06 | 百度在线网络技术（北京）有限公司 | Video processing method and device, electronic equipment and storage medium |
| WO2022247014A1 (en) * | 2021-05-26 | 2022-12-01 | 深圳市沃特沃德信息有限公司 | Audio and video frame synchronization method and apparatus based on ink screen device, and computer device |
| CN113435294B (en) * | 2021-06-23 | 2023-05-23 | 中国矿业大学(北京) | Coal-rock interface positioning and identifying method based on image and sound fusion |
| CN113435294A (en) * | 2021-06-23 | 2021-09-24 | 中国矿业大学(北京) | Coal-rock interface positioning and identifying method based on image and sound fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106162293B (en) | 2019-11-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106162293A (en) | A kind of video sound and the method and device of image synchronization | |
| US8307287B2 (en) | Heads-up-display for use in a media manipulation operation | |
| US8810728B2 (en) | Method and apparatus for synchronizing audio and video streams | |
| CN105070304B (en) | Realize method and device, the electronic equipment of multi-object audio recording | |
| US9854294B2 (en) | Localized audio source extraction from video recordings | |
| EP2899718A1 (en) | Device and method for changing shape of lips on basis of automatic word translation | |
| JP2008148121A (en) | Motion picture summary automatic generation apparatus and method, and computer program | |
| JP2022095945A (en) | Ancillary data synchronization for content including audio | |
| US11164604B2 (en) | Video editing method and apparatus, computer device and readable storage medium | |
| CN103546698B (en) | A kind of mobile terminal recorded video store method and device | |
| CN104284057A (en) | Video processing method and device | |
| CN106162323A (en) | A kind of video data handling procedure and device | |
| CA3139473A1 (en) | System and method of synchronizing video and audio clips with audio data | |
| CN111771385B (en) | Coordinates as assistance data | |
| CN106162222A (en) | A kind of method and device of video lens cutting | |
| CN111970560A (en) | Video acquisition method and device, electronic equipment and storage medium | |
| CN104240697A (en) | Audio data feature extraction method and device | |
| WO2014049989A1 (en) | Image output device | |
| CA2972051C (en) | Use of program-schedule text and closed-captioning text to facilitate selection of a portion of a media-program recording | |
| US8145031B2 (en) | Recording apparatus | |
| WO2024131555A1 (en) | Video music matching method and device, storage medium, and program product | |
| CN105573844B (en) | A kind of control method and electronic equipment | |
| JP3668092B2 (en) | Image processing apparatus and target detection tracking apparatus using the same | |
| CN104469546A (en) | Video clip processing method and device | |
| CN110267080A (en) | Data processing method and device for robot casting content synchronization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A method and device for video sound and image synchronization Effective date of registration: 20210104 Granted publication date: 20191108 Pledgee: Inner Mongolia Huipu Energy Co.,Ltd. Pledgor: WUXI TVMINING MEDIA SCIENCE & TECHNOLOGY Co.,Ltd. Registration number: Y2020990001517 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20191108 Termination date: 20210422 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |