CN109194899A - A kind of method and terminal of audio-visual synchronization - Google Patents
A kind of method and terminal of audio-visual synchronization Download PDFInfo
- Publication number
- CN109194899A CN109194899A CN201811401913.9A CN201811401913A CN109194899A CN 109194899 A CN109194899 A CN 109194899A CN 201811401913 A CN201811401913 A CN 201811401913A CN 109194899 A CN109194899 A CN 109194899A
- Authority
- CN
- China
- Prior art keywords
- audio
- video
- terminal
- user
- visual synchronization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4398—Processing of audio elementary streams involving reformatting operations of audio signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44213—Monitoring of end-user related data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44213—Monitoring of end-user related data
- H04N21/44218—Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program
Abstract
The present invention provides a kind of method of audio-visual synchronization and terminals.The described method includes: acquiring the video image of user in video call process；When detecting the terminal plays music, the second audio in the first audio and environment of the terminal plays is obtained；According to first audio, second audio and the video image, the target audio-video of audio-visual synchronization is generated；Send the target audio-video to the target object of video calling.Through the embodiment of the present invention, terminal can be in video call process, the music of broadcasting, user speech, video image are generated to the target audio-video of audio-visual synchronization, and send target audio-video to the target object of video calling, make the target object of video calling that can hear the music of terminal plays while watching the video, the effect that music and audio video synchronization are realized using a terminal, improves the usage experience of user.
Description
Technical field
The present invention relates to technical field of mobile terminals more particularly to the methods and terminal of a kind of audio-visual synchronization.
Background technique
With the rapid development of mobile terminal and network technology, mobile terminal is in order to adapt to a variety of demands of user, exploitation
More and more functions out.In daily life and work, user usually uses mobile terminal to carry out video calling.But
In video call process, if necessary to play music, it just will appear the problem of other side can't hear the music of user's broadcasting.For example,
User will demonstrate dancing for other side when video calling, other equipment can only be used to play accompaniment music, otherwise other side can only see
The dancing of user's demonstration, and can't hear the accompaniment music of user's broadcasting, user experience is very poor.
Summary of the invention
The embodiment of the present invention provides the method and terminal of a kind of audio-visual synchronization, to solve in the prior art in video calling
In the process, if necessary to play music, it just will appear the problem of other side can't hear the music of user's broadcasting.
In order to solve the above-mentioned technical problem, the embodiment of the invention provides a kind of methods of audio-visual synchronization, are applied to eventually
End, which comprises
The video image of user is acquired in video call process；
When detecting the terminal plays music, the second sound in the first audio and environment of the terminal plays is obtained
Frequently；
According to first audio, second audio and the video image, the target sound view of audio-visual synchronization is generated
Frequently；
Send the target audio-video to the target object of video calling
The embodiment of the invention also provides a kind of terminal of audio-visual synchronization, the terminal includes:
Video image acquisition module, for acquiring the video image of user in video call process；
Audio obtains module, for when detecting the terminal plays music, obtaining the first sound of the terminal plays
The second audio in frequency and environment；
Target audio-video generation module, for according to first audio, second audio and the video image, life
At the target audio-video of audio-visual synchronization；
Target audio-video sending module, for sending the target audio-video to the target object of video calling.
The embodiment of the invention also provides a kind of terminal, including processor, memory and it is stored on the memory simultaneously
The computer program that can be run on the processor is realized when the computer program is executed by the processor as above-mentioned
The step of method of audio-visual synchronization.
The embodiment of the invention also provides a kind of computer readable storage medium, deposited on the computer readable storage medium
The step of storing up computer program, the method such as above-mentioned audio-visual synchronization realized when the computer program is executed by processor.
In the embodiment of the present invention, terminal acquires the video image of user in video call process；When detecting that terminal broadcasts
When putting the music on, the second audio in the first audio and environment of terminal plays is obtained；According to the first audio, the second audio and video
Image generates the target audio-video of audio-visual synchronization, and sends target audio-video to the target object of video calling.Pass through this
The music of broadcasting, user speech, video image can be generated audio-video in video call process by inventive embodiments, terminal
Synchronous target audio-video, and send target audio-video to the target object of video calling, make the target object of video calling
The music of terminal plays can be heard while watching the video, i.e., the effect of music and audio video synchronization is realized using a terminal
Fruit improves the usage experience of user.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below by institute in the description to the embodiment of the present invention
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the invention
Example, for those of ordinary skill in the art, without any creative labor, can also be according to these attached drawings
Obtain other attached drawings.
Fig. 1 is a kind of step flow chart of the method for audio-visual synchronization of the embodiment of the present invention one；
Fig. 2 is a kind of step flow chart of the method for audio-visual synchronization of the embodiment of the present invention two；
Fig. 3 is a kind of one of the structural block diagram of terminal of audio-visual synchronization of the embodiment of the present invention three；
Fig. 4 is the two of the structural block diagram of the terminal of a kind of audio-visual synchronization of the embodiment of the present invention three；
Fig. 5 is a kind of hardware structural diagram of mobile terminal of the embodiment of the present invention four.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
Embodiment one
Referring to Fig.1, a kind of step flow chart of the method for audio-visual synchronization provided in an embodiment of the present invention is shown.Using
In terminal, which comprises
Step 101, the video image of user is acquired in video call process.
In the present embodiment, in video call process, terminal can acquire the video image of user by camera.Example
Such as, in video call process, user is that the target object of video calling demonstrates dancing, the video image that acquisition user dances.
Step 102, it when detecting the terminal plays music, obtains in the first audio and environment of the terminal plays
The second audio.
In the present embodiment, terminal can play music according to user instructions.For example, music application program is opened,
Music is chosen in music application program and is played out.When terminal plays music, in order to make the target pair of video calling
As that can also hear music, realizes audio-visual synchronization, obtain the second audio in the first audio and environment of terminal plays.
For example, obtaining the first audio of music from music application program, being obtained by microphone includes user speech, ambient sound
The second audio.
Step 103, according to first audio, second audio and the video image, audio-visual synchronization is generated
Target audio-video.
In the present embodiment, after getting the first audio, the second audio, the first audio, the second audio and video image are closed
At the target audio-video of audio-visual synchronization.For example, music A, user to be explained to the voice B and user's demonstration dance of dance movement
The target audio-video for the video image synthesis audio-visual synchronization stepped.The embodiment of the present invention does not make the generating mode of target audio-video
It limits, can be configured according to the actual situation in detail.
Step 104, the target audio-video is sent to the target object of video calling.
In the present embodiment, it sends target audio-video to the target object of video calling, makes the target object of video calling
While watching dancing, the music of terminal plays can be heard, the user without terminal plays accompaniment tone using other equipment
It is happy, improve the usage experience of user.
In conclusion in embodiments of the present invention, terminal acquires the video image of user in video call process；Work as inspection
When measuring terminal plays music, the second audio in the first audio and environment of terminal plays is obtained；According to the first audio, second
Audio and video image generates the target audio-video of audio-visual synchronization, and sends target audio-video to the target of video calling
Object.Through the embodiment of the present invention, terminal can be in video call process, by the music of broadcasting, user speech, video image
The target audio-video of audio-visual synchronization is generated, and sends target audio-video to the target object of video calling, makes video calling
Target object can hear the music of terminal plays while watching the video, i.e., music and video are realized using a terminal
Synchronous effect improves the usage experience of user.
Embodiment two
Referring to Fig. 2, the step flow chart for the method that a kind of audio-visual synchronization provided in an embodiment of the present invention is sent is shown.
Applied to terminal, which comprises
Step 201, the video image of user is acquired in video call process.
Step 202, when detecting the terminal plays music, audio-visual synchronization instruction is received.
In the present embodiment, in terminal plays music, whether it can also choose according to the actual situation by music and video
Image synchronization.For example, carrying out the processing of audio-visual synchronization if receiving audio-visual synchronization instruction.If not receiving sound
Audio video synchronization instruction, then without the processing of audio-visual synchronization.
Receiving audio-visual synchronization instruction may include various ways:
Mode one judges in the video image with the presence or absence of user images；When there are the user images, detection is used
The position of family hand；The user gesture is determined according to the change in location of user's hand；When the user gesture with it is described
In terminal when the matching of preset synchronization gesture, determines and receive the audio-visual synchronization instruction.
Specifically, judge not detect user then if there is no user images with the presence or absence of user images in video image
The position of hand, the processing without audio-visual synchronization；If there is user images, then the position of user's hand is detected.It is optional
Ground detects the position of user's hand by the ultrasonic unit that terminal has.For example, the receiver sending using terminal the first surpasses
Acoustic signals receive the second ultrasonic signal returned using microphone, are believed according to the first ultrasonic signal and the second ultrasonic wave
Number detection user's hand position.
User gesture is determined after detecting the position of user's hand, for example, determining that user gesture is user's hand and terminal
The distance between shorten, the distance between user's hand and terminal become far；It can also determine that user gesture is that user's hand is opposite
Terminal increases, and user's hand relative termination reduces；It can also determine that user gesture is that user's hand relative termination moves to the left,
User's hand relative termination is mobile etc. to the right.
From terminal search with the matched synchronous gesture of user gesture, if found and the matched synchronous hand of user gesture
Gesture, it is determined that receive audio-visual synchronization instruction, carry out the step of obtaining the first audio, the second audio；If do not find with
The matched synchronous gesture of user gesture, it is determined that audio-visual synchronization instruction is not received, without obtaining the first audio, the second sound
The step of frequency.Optionally, the synchronous gesture includes that the distance between user's hand and the terminal reduce, the user
The relatively described terminal of hand increases, the relatively described terminal of user's hand at least one on the move to the left.The present invention is real
It applies example not limit synchronous gesture in detail, can be configured according to the actual situation.
It is just determined there are user images in terminal plays music, video image and when user gesture is matched with synchronous gesture
Audio-visual synchronization instruction being received, sound view can be carried out to avoid misjudging other instructions received for audio-visual synchronization instruction
The problem of frequency synchronization process.
Mode two shows synchronous switch in video calling interface；Receive the touch command for opening the synchronous switch.
Specifically, it can also be and show synchronous switch in video calling interface.Synchronous switch is opened if received
Touch command then carries out the processing of audio-visual synchronization；If receiving the touch command for closing synchronous switch, sound is no longer carried out
The processing of audio video synchronization.User can open or close synchronous switch at any time according to demand, user-friendly, improve user
Usage experience.
Step 203, the second audio in the first audio and environment of the terminal plays is obtained.
Step 204, according to first audio, second audio and the video image, audio-visual synchronization is generated
Target audio-video.
In the present embodiment, the target audio-video for generating audio-visual synchronization can specifically include following steps:
Sub-step one carries out noise reduction process to second audio according to first audio.
It specifically,, can other than it can get user speech when obtaining the second audio in environment using microphone
It can also get the music of terminal plays.For target audio-video, the music of the terminal plays got from microphone
Belong to noise, therefore noise reduction process can be carried out to the second audio according to the first audio, i.e., it will be from microphone from the second audio
The music of the terminal plays got removes.
The second audio after noise reduction process is synthesized third audio with first audio by sub-step two.
For example, time tag is arranged when obtaining the first audio, the second audio, user speech will be retained after noise reduction process
Second audio synthesizes third audio according to time tag with the music of the first audio.Wherein, third audio is also provided with time mark
Label.
Sub-step three, the third audio is corresponding according to the time with the video image, generate the target audio-video.
For example, third audio and video image is respectively provided with time tag, according to time tag by third audio and video figure
As corresponding to, the target audio-video of audio-visual synchronization is generated.It can be nonsynchronous with video image to avoid music according to time correspondence
Problem can also make music and audio video synchronization using other modes, and the embodiment of the present invention does not limit this in detail, can basis
Actual conditions are configured.
Step 205, the target audio-video is sent to the target object of video calling.
In conclusion in embodiments of the present invention, terminal acquires the video image of user in video call process；Work as inspection
When measuring terminal plays music, the second audio in the first audio and environment of terminal plays is obtained；According to the first audio, second
Audio and video image generates the target audio-video of audio-visual synchronization, and sends target audio-video to the target of video calling
Object.Through the embodiment of the present invention, terminal can be in video call process, by the music of broadcasting, user speech, video image
The target audio-video of audio-visual synchronization is generated, and sends target audio-video to the target object of video calling, makes video calling
Target object can hear the music of terminal plays while watching the video, i.e., music and video are realized using a terminal
Synchronous effect improves the usage experience of user.
Embodiment three
Referring to Fig. 3, a kind of structural block diagram of the terminal of audio-visual synchronization provided in an embodiment of the present invention is shown.The end
End includes video image acquisition module 301, audio acquisition module 302, target audio-video generation module 303, target audio-video hair
Send module 304:
Video image acquisition module 301, for acquiring the video image of user in video call process；
Audio obtains module 302, for obtaining the first of the terminal plays when detecting the terminal plays music
The second audio in audio and environment；
Target audio-video generation module 303, for according to first audio, second audio and the video figure
Picture generates the target audio-video of audio-visual synchronization；
Target audio-video sending module 304, for sending the target audio-video to the target object of video calling.
On the basis of Fig. 3, optionally, before the audio obtains module 302, the terminal further includes synchronic command
Receiving module 305, is shown in Fig. 4:
Synchronic command receiving module 305, for receiving audio-visual synchronization instruction.
On the basis of fig. 4, optionally, the audio-visual synchronization command reception module 305 includes:
Judging submodule, for judging in the video image with the presence or absence of user images；
Detection sub-module, for detecting the position of user's hand when there are the user images；
User gesture determines submodule, for determining the user gesture according to the change in location of user's hand；
First command reception submodule, for being matched when the user gesture with synchronous gesture preset in the terminal
When, it determines and receives the audio-visual synchronization instruction.
On the basis of fig. 4, optionally, the audio-visual synchronization command reception module 305 includes:
Display sub-module, for showing synchronous switch in video calling interface；
Second command reception submodule, for receiving the touch command for opening the synchronous switch.
On the basis of Fig. 3, optionally, the target audio-video generation module 303 includes:
Noise reduction process submodule, for carrying out noise reduction process to second audio according to first audio；
Audio generates submodule, for the second audio after noise reduction process to be synthesized third audio with first audio；
Target audio-video generates submodule, raw for the third audio is corresponding according to the time with the video image
At the target audio-video of the audio-visual synchronization.
The terminal of audio-visual synchronization provided in an embodiment of the present invention can be realized to be realized in the embodiment of the method for Fig. 1 and Fig. 2
Each process, to avoid repeating, which is not described herein again.Through the embodiment of the present invention, terminal can in video call process,
The music of broadcasting, user speech, video image are generated into the target audio-video of audio-visual synchronization, and send target audio-video to
The target object of video calling makes the target object of video calling that can hear the sound of terminal plays while watching the video
It is happy, i.e., the effect of music and audio video synchronization is realized using a terminal, improves the usage experience of user.
Example IV
A kind of hardware structural diagram of Fig. 5 mobile terminal of each embodiment to realize the present invention.
The mobile terminal 400 includes but is not limited to: radio frequency unit 401, network module 402, audio output unit 403, defeated
Enter unit 404, sensor 405, display unit 406, user input unit 407, interface unit 408, memory 409, processor
The components such as 410 and power supply 411.It will be understood by those skilled in the art that mobile terminal structure shown in Fig. 5 is not constituted
Restriction to mobile terminal, mobile terminal may include than illustrating more or fewer components, perhaps combine certain components or
Different component layouts.In embodiments of the present invention, mobile terminal include but is not limited to mobile phone, tablet computer, laptop,
Palm PC, car-mounted terminal, wearable device and pedometer etc..
Wherein, input unit 404, for acquiring the video image of user in video call process.
Processor 410, for when detecting the terminal plays music, obtain the terminal plays the first audio and
The second audio in environment；According to first audio, second audio and the video image, audio-visual synchronization is generated
Target audio-video；Send the target audio-video to the target object of video calling.
Through the embodiment of the present invention, terminal can be in video call process, by the music of broadcasting, user speech, video
Image generates the target audio-video of audio-visual synchronization, and sends target audio-video to the target object of video calling, makes video
The target object of call can hear the music of terminal plays while watching the video, i.e., using a terminal realize music and
The effect of audio video synchronization improves the usage experience of user.
It should be understood that the embodiment of the present invention in, radio frequency unit 401 can be used for receiving and sending messages or communication process in, signal
Send and receive, specifically, by from base station downlink data receive after, to processor 410 handle；In addition, by uplink
Data are sent to base station.In general, radio frequency unit 401 includes but is not limited to antenna, at least one amplifier, transceiver, coupling
Device, low-noise amplifier, duplexer etc..In addition, radio frequency unit 401 can also by wireless communication system and network and other set
Standby communication.
Mobile terminal provides wireless broadband internet by network module 402 for user and accesses, and such as user is helped to receive
It sends e-mails, browse webpage and access streaming video etc..
Audio output unit 403 can be received by radio frequency unit 401 or network module 402 or in memory 409
The audio data of storage is converted into audio signal and exports to be sound.Moreover, audio output unit 403 can also be provided and be moved
The relevant audio output of specific function that dynamic terminal 400 executes is (for example, call signal receives sound, message sink sound etc.
Deng).Audio output unit 403 includes loudspeaker, buzzer and receiver etc..
Input unit 404 is for receiving audio or video signal.Input unit 404 may include graphics processor
(Graphics Processing Unit, GPU) 4041 and microphone 4042, graphics processor 4041 is in video acquisition mode
Or the image data of the static images or video obtained in image capture mode by image capture apparatus (such as camera) carries out
Reason.Treated, and picture frame may be displayed on display unit 406.Through graphics processor 4041, treated that picture frame can be deposited
Storage is sent in memory 409 (or other storage mediums) or via radio frequency unit 401 or network module 402.Mike
Wind 4042 can receive sound, and can be audio data by such acoustic processing.Treated audio data can be
The format output that mobile communication base station can be sent to via radio frequency unit 401 is converted in the case where telephone calling model.
Mobile terminal 400 further includes at least one sensor 405, such as optical sensor, motion sensor and other biographies
Sensor.Specifically, optical sensor includes ambient light sensor and proximity sensor, wherein ambient light sensor can be according to environment
The light and shade of light adjusts the brightness of display panel 4061, and proximity sensor can close when mobile terminal 400 is moved in one's ear
Display panel 4061 and/or backlight.As a kind of motion sensor, accelerometer sensor can detect in all directions (general
For three axis) size of acceleration, it can detect that size and the direction of gravity when static, can be used to identify mobile terminal posture (ratio
Such as horizontal/vertical screen switching, dependent game, magnetometer pose calibrating), Vibration identification correlation function (such as pedometer, tap)；It passes
Sensor 405 can also include fingerprint sensor, pressure sensor, iris sensor, molecule sensor, gyroscope, barometer, wet
Meter, thermometer, infrared sensor etc. are spent, details are not described herein.
Display unit 406 is for showing information input by user or being supplied to the information of user.Display unit 406 can wrap
Display panel 4061 is included, liquid crystal display (Liquid Crystal Display, LCD), Organic Light Emitting Diode can be used
Forms such as (Organic Light-Emitting Diode, OLED) configure display panel 4061.
User input unit 407 can be used for receiving the number or character information of input, and generate the use with mobile terminal
Family setting and the related key signals input of function control.Specifically, user input unit 407 include touch panel 4071 and
Other input equipments 4072.Touch panel 4071, also referred to as touch screen collect the touch operation of user on it or nearby
(for example user uses any suitable objects or attachment such as finger, stylus on touch panel 4071 or in touch panel 4071
Neighbouring operation).Touch panel 4071 may include both touch detecting apparatus and touch controller.Wherein, touch detection
Device detects the touch orientation of user, and detects touch operation bring signal, transmits a signal to touch controller；Touch control
Device processed receives touch information from touch detecting apparatus, and is converted into contact coordinate, then gives processor 410, receiving area
It manages the order that device 410 is sent and is executed.Furthermore, it is possible to more using resistance-type, condenser type, infrared ray and surface acoustic wave etc.
Seed type realizes touch panel 4071.In addition to touch panel 4071, user input unit 407 can also include other input equipments
4072.Specifically, other input equipments 4072 can include but is not limited to physical keyboard, function key (such as volume control button,
Switch key etc.), trace ball, mouse, operating stick, details are not described herein.
Further, touch panel 4071 can be covered on display panel 4061, when touch panel 4071 is detected at it
On or near touch operation after, send processor 410 to determine the type of touch event, be followed by subsequent processing device 410 according to touching
The type for touching event provides corresponding visual output on display panel 4061.Although in Fig. 5, touch panel 4071 and display
Panel 4061 is the function that outputs and inputs of realizing mobile terminal as two independent components, but in some embodiments
In, can be integrated by touch panel 4071 and display panel 4061 and realize the function that outputs and inputs of mobile terminal, it is specific this
Place is without limitation.
Interface unit 408 is the interface that external device (ED) is connect with mobile terminal 400.For example, external device (ED) may include having
Line or wireless head-band earphone port, external power supply (or battery charger) port, wired or wireless data port, storage card end
Mouth, port, the port audio input/output (I/O), video i/o port, earphone end for connecting the device with identification module
Mouthful etc..Interface unit 408 can be used for receiving the input (for example, data information, electric power etc.) from external device (ED) and
By one or more elements that the input received is transferred in mobile terminal 400 or can be used in 400 He of mobile terminal
Data are transmitted between external device (ED).
Memory 409 can be used for storing software program and various data.Memory 409 can mainly include storing program area
The storage data area and, wherein storing program area can (such as the sound of application program needed for storage program area, at least one function
Sound playing function, image player function etc.) etc.；Storage data area can store according to mobile phone use created data (such as
Audio data, phone directory etc.) etc..In addition, memory 409 may include high-speed random access memory, it can also include non-easy
The property lost memory, a for example, at least disk memory, flush memory device or other volatile solid-state parts.
Processor 410 is the control centre of mobile terminal, utilizes each of various interfaces and the entire mobile terminal of connection
A part by running or execute the software program and/or module that are stored in memory 409, and calls and is stored in storage
Data in device 409 execute the various functions and processing data of mobile terminal, to carry out integral monitoring to mobile terminal.Place
Managing device 410 may include one or more processing units；Preferably, processor 410 can integrate application processor and modulatedemodulate is mediated
Manage device, wherein the main processing operation system of application processor, user interface and application program etc., modem processor is main
Processing wireless communication.It is understood that above-mentioned modem processor can not also be integrated into processor 410.
Mobile terminal 400 can also include the power supply 411 (such as battery) powered to all parts, it is preferred that power supply 411
Can be logically contiguous by power-supply management system and processor 410, to realize management charging by power-supply management system, put
The functions such as electricity and power managed.
In addition, mobile terminal 400 includes some unshowned functional modules, details are not described herein.
Preferably, the embodiment of the present invention also provides a kind of terminal, including processor 410, and memory 409 is stored in storage
It is real when which is executed by processor 410 on device 409 and the computer program that can be run on the processor 410
Each process of the embodiment of the method for existing above-mentioned audio-visual synchronization, and identical technical effect can be reached, to avoid repeating, here
It repeats no more.
The embodiment of the present invention also provides a kind of computer readable storage medium, and meter is stored on computer readable storage medium
Calculation machine program, the computer program realize each process of the embodiment of the method for above-mentioned audio-visual synchronization when being executed by processor,
And identical technical effect can be reached, to avoid repeating, which is not described herein again.Wherein, the computer readable storage medium,
Such as read-only memory (Read-Only Memory, abbreviation ROM), random access memory (Random Access Memory, letter
Claim RAM), magnetic or disk etc..
It should be noted that, in this document, the terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, method, article or the device that include a series of elements not only include those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or device institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including being somebody's turn to do
There is also other identical elements in the process, method of element, article or device.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art
The part contributed out can be embodied in the form of software products, which is stored in a storage medium
In (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that a terminal (can be mobile phone, computer, service
Device, air conditioner or network equipment etc.) execute method described in each embodiment of the present invention.
The embodiment of the present invention is described with above attached drawing, but the invention is not limited to above-mentioned specific
Embodiment, the above mentioned embodiment is only schematical, rather than restrictive, those skilled in the art
Under the inspiration of the present invention, without breaking away from the scope protected by the purposes and claims of the present invention, it can also make very much
Form, all of these belong to the protection of the present invention.
Claims (12)
1. a kind of method of audio-visual synchronization, which is characterized in that be applied to terminal, which comprises
The video image of user is acquired in video call process；
When detecting the terminal plays music, the second audio in the first audio and environment of the terminal plays is obtained；
According to first audio, second audio and the video image, the target audio-video of audio-visual synchronization is generated；
Send the target audio-video to the target object of video calling.
2. the method according to claim 1, wherein in first audio for obtaining the terminal plays and
Before the second audio in environment, the method also includes:
Receive audio-visual synchronization instruction.
3. according to the method described in claim 2, it is characterized in that, the reception audio-visual synchronization instructs, comprising:
Judge in the video image with the presence or absence of user images；
When there are the user images, the position of user's hand is detected；
The user gesture is determined according to the change in location of user's hand；
When the user gesture is matched with synchronous gesture preset in the terminal, determine that receiving the audio-visual synchronization refers to
It enables.
4. according to the method described in claim 2, it is characterized in that, the reception audio-visual synchronization instructs, comprising:
Synchronous switch is shown in video calling interface；
Receive the touch command for opening the synchronous switch.
5. the method according to claim 1, wherein it is described according to first audio, second audio and
The video image generates the target audio-video of audio-visual synchronization, comprising:
Noise reduction process is carried out to second audio according to first audio；
The second audio after noise reduction process is synthesized into third audio with first audio；
The third audio is corresponding according to the time with the video image, generate the target audio-video of the audio-visual synchronization.
6. a kind of terminal of audio-visual synchronization, which is characterized in that the terminal includes:
Video image acquisition module, for acquiring the video image of user in video call process；
Audio obtain module, for when detecting the terminal plays music, obtain the terminal plays the first audio and
The second audio in environment；
Target audio-video generation module, for generating sound according to first audio, second audio and the video image
The target audio-video of audio video synchronization；
Target audio-video sending module, for sending the target audio-video to the target object of video calling.
7. terminal according to claim 6, which is characterized in that before the audio obtains module, the terminal is also wrapped
It includes:
Synchronic command receiving module, for receiving audio-visual synchronization instruction.
8. terminal according to claim 7, which is characterized in that the audio-visual synchronization command reception module includes:
Judging submodule, for judging in the video image with the presence or absence of user images；
Detection sub-module, for detecting the position of user's hand when there are the user images；
User gesture determines submodule, for determining the user gesture according to the change in location of user's hand；
First command reception submodule, for when the user gesture is matched with preset synchronous gesture in the terminal, really
Surely the audio-visual synchronization instruction is received.
9. terminal according to claim 7, which is characterized in that the audio-visual synchronization command reception module includes:
Display sub-module, for showing synchronous switch in video calling interface；
Second command reception submodule, for receiving the touch command for opening the synchronous switch.
10. terminal according to claim 6, which is characterized in that the target audio-video generation module includes:
Noise reduction process submodule, for carrying out noise reduction process to second audio according to first audio；
Audio generates submodule, for the second audio after noise reduction process to be synthesized third audio with first audio；
Target audio-video generates submodule, for the third audio is corresponding according to the time with the video image, generation institute
State target audio-video.
11. a kind of terminal, which is characterized in that including processor, memory and be stored on the memory and can be at the place
The computer program run on reason device, is realized as described in claim 1-5 when the computer program is executed by the processor
Audio-visual synchronization method the step of.
12. a kind of computer readable storage medium, which is characterized in that store computer journey on the computer readable storage medium
The step of sequence, the computer program realizes the method for audio-visual synchronization as claimed in claims 1-5 when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811401913.9A CN109194899A (en) | 2018-11-22 | 2018-11-22 | A kind of method and terminal of audio-visual synchronization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811401913.9A CN109194899A (en) | 2018-11-22 | 2018-11-22 | A kind of method and terminal of audio-visual synchronization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109194899A true CN109194899A (en) | 2019-01-11 |
Family
ID=64938178
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811401913.9A Pending CN109194899A (en) | 2018-11-22 | 2018-11-22 | A kind of method and terminal of audio-visual synchronization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109194899A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109842795A (en) * | 2019-02-28 | 2019-06-04 | 苏州科达科技股份有限公司 | Audio-visual synchronization performance test methods, device, electronic equipment, storage medium |
| CN111128104A (en) * | 2019-12-26 | 2020-05-08 | 北京塞宾科技有限公司 | Wireless karaoke method, audio device and intelligent terminal |
| WO2021019342A1 (en) * | 2019-07-30 | 2021-02-04 | International Business Machines Corporation | Synchronized sound generation from videos |
| CN113595869A (en) * | 2021-06-28 | 2021-11-02 | 青岛海尔科技有限公司 | Voice playing method and device, storage medium and electronic device |
| CN113676567A (en) * | 2021-07-14 | 2021-11-19 | 维沃移动通信有限公司 | Electronic device, conversation method and readable storage medium |
| WO2022111599A1 (en) * | 2020-11-30 | 2022-06-02 | 百果园技术(新加坡)有限公司 | Call interaction method and apparatus, and device and storage medium |
| CN115209175A (en) * | 2022-07-18 | 2022-10-18 | 忆月启函(盐城)科技有限公司 | Voice transmission method and system |
| JP7475423B2 (en) | 2019-07-30 | 2024-04-26 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Synchronized speech generation from video |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102426480A (en) * | 2011-11-03 | 2012-04-25 | 康佳集团股份有限公司 | Man-machine interactive system and real-time gesture tracking processing method for same |
| CN106302087A (en) * | 2015-05-19 | 2017-01-04 | 深圳市腾讯计算机系统有限公司 | Instant communication method, Apparatus and system |
| CN107027050A (en) * | 2017-04-13 | 2017-08-08 | 广州华多网络科技有限公司 | Auxiliary live audio/video processing method and device |
| US20170310926A1 (en) * | 2016-04-20 | 2017-10-26 | Disney Enterprises, Inc. | System and method for providing co-delivery of content |
-
2018
- 2018-11-22 CN CN201811401913.9A patent/CN109194899A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102426480A (en) * | 2011-11-03 | 2012-04-25 | 康佳集团股份有限公司 | Man-machine interactive system and real-time gesture tracking processing method for same |
| CN106302087A (en) * | 2015-05-19 | 2017-01-04 | 深圳市腾讯计算机系统有限公司 | Instant communication method, Apparatus and system |
| US20170310926A1 (en) * | 2016-04-20 | 2017-10-26 | Disney Enterprises, Inc. | System and method for providing co-delivery of content |
| CN107027050A (en) * | 2017-04-13 | 2017-08-08 | 广州华多网络科技有限公司 | Auxiliary live audio/video processing method and device |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109842795A (en) * | 2019-02-28 | 2019-06-04 | 苏州科达科技股份有限公司 | Audio-visual synchronization performance test methods, device, electronic equipment, storage medium |
| GB2600600B (en) * | 2019-07-30 | 2022-10-26 | Ibm | Synchronized sound generation from videos |
| WO2021019342A1 (en) * | 2019-07-30 | 2021-02-04 | International Business Machines Corporation | Synchronized sound generation from videos |
| US11276419B2 (en) | 2019-07-30 | 2022-03-15 | International Business Machines Corporation | Synchronized sound generation from videos |
| GB2600600A (en) * | 2019-07-30 | 2022-05-04 | Ibm | Synchronized sound generation from videos |
| JP7475423B2 (en) | 2019-07-30 | 2024-04-26 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Synchronized speech generation from video |
| CN111128104A (en) * | 2019-12-26 | 2020-05-08 | 北京塞宾科技有限公司 | Wireless karaoke method, audio device and intelligent terminal |
| WO2022111599A1 (en) * | 2020-11-30 | 2022-06-02 | 百果园技术(新加坡)有限公司 | Call interaction method and apparatus, and device and storage medium |
| CN113595869A (en) * | 2021-06-28 | 2021-11-02 | 青岛海尔科技有限公司 | Voice playing method and device, storage medium and electronic device |
| CN113595869B (en) * | 2021-06-28 | 2023-10-24 | 青岛海尔科技有限公司 | Voice playing method and device, storage medium and electronic device |
| CN113676567A (en) * | 2021-07-14 | 2021-11-19 | 维沃移动通信有限公司 | Electronic device, conversation method and readable storage medium |
| CN113676567B (en) * | 2021-07-14 | 2024-01-30 | 维沃移动通信有限公司 | Electronic device, communication method, and readable storage medium |
| CN115209175A (en) * | 2022-07-18 | 2022-10-18 | 忆月启函(盐城)科技有限公司 | Voice transmission method and system |
| CN115209175B (en) * | 2022-07-18 | 2023-10-24 | 深圳蓝色鲨鱼科技有限公司 | Voice transmission method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107734378B (en) | A kind of audio and video synchronization method, device and mobile terminal | |
| CN109194899A (en) | A kind of method and terminal of audio-visual synchronization | |
| CN108762640A (en) | A kind of display methods and terminal of barrage information | |
| CN109343759A (en) | A kind of control method and terminal of the display of breath screen | |
| CN107911445A (en) | A kind of information push method, mobile terminal and storage medium | |
| CN107908705A (en) | A kind of information-pushing method, information push-delivery apparatus and mobile terminal | |
| CN108551534A (en) | The method and device of multiple terminals voice communication | |
| CN110012143A (en) | A kind of receiver control method and terminal | |
| CN109144703A (en) | A kind of processing method and its terminal device of multitask | |
| CN108196815A (en) | A kind of adjusting method and mobile terminal of sound of conversing | |
| CN108898555A (en) | A kind of image processing method and terminal device | |
| CN109658886A (en) | A kind of control method and terminal of display screen | |
| CN110198428A (en) | A kind of multimedia file producting method and first terminal | |
| CN110018805A (en) | A kind of display control method and mobile terminal | |
| CN109618218A (en) | A kind of method for processing video frequency and mobile terminal | |
| CN109361797A (en) | A kind of vocal technique and mobile terminal | |
| CN109192153A (en) | A kind of terminal and terminal control method | |
| CN109144393A (en) | A kind of image display method and mobile terminal | |
| CN110460717A (en) | Terminal control method and mobile terminal | |
| CN109348035A (en) | A kind of recognition methods of telephone number and terminal device | |
| CN109672845A (en) | A kind of method, apparatus and mobile terminal of video calling | |
| CN109814773A (en) | A kind of flexible screen control method and display component | |
| CN109443261A (en) | The acquisition methods and mobile terminal of Folding screen mobile terminal folding angles | |
| CN108319440A (en) | Audio-frequency inputting method and mobile terminal | |
| CN108551562A (en) | A kind of method and mobile terminal of video communication |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190111 |