CN115720278A - Synchronous processing method of sound and picture and related device - Google Patents
Synchronous processing method of sound and picture and related device Download PDFInfo
- Publication number
- CN115720278A CN115720278A CN202211368580.0A CN202211368580A CN115720278A CN 115720278 A CN115720278 A CN 115720278A CN 202211368580 A CN202211368580 A CN 202211368580A CN 115720278 A CN115720278 A CN 115720278A
- Authority
- CN
- China
- Prior art keywords
- picture
- intelligent audio
- video
- sound
- instruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 86
- 238000003672 processing method Methods 0.000 title claims abstract description 12
- 239000000872 buffer Substances 0.000 claims abstract description 89
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000012545 processing Methods 0.000 claims description 89
- 230000003111 delayed effect Effects 0.000 claims description 14
- 230000003213 activating effect Effects 0.000 claims description 5
- 238000004891 communication Methods 0.000 description 10
- 230000001960 triggered effect Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000003139 buffering effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 238000009877 rendering Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application is suitable for the technical field of intelligent audio-visual equipment, provides a sound and picture synchronous processing method and a related device applied to the intelligent audio-visual equipment, and aims to solve the problem that the intelligent audio-visual equipment is not synchronous when pictures and sounds are finally output in the prior art. The method of the present application mainly comprises: when a picture delay instruction is received, increasing the caching quantity of picture frames of the pictures played by the intelligent audio-video equipment in a cache according to the picture delay instruction; and when a picture advance instruction is received, the buffer memory is used for reducing the buffer memory quantity of the picture frames of the playing pictures of the intelligent audio-visual equipment according to the picture advance instruction.
Description
Technical Field
The present application belongs to the technical field of intelligent audio-video equipment, and in particular, to a method and a related device for synchronously processing sound and pictures applied to the intelligent audio-video equipment.
Background
The media resources played by the smart television comprise sound and picture resources which are on the same track in a time sequence, and in order to output the sound and the pictures in the media resources, the smart television needs to acquire the media resources first, then the media resources are subjected to related protocol analysis, audio and video decoding, synchronous rendering and other work, and finally the pictures are output through a screen of the smart television and the sound is output through a loudspeaker or an external sound box of the smart television. In the process, from the beginning of audio and video decoding, the video and the audio are separated into different paths to be processed, each link after the video and the audio are separated and processed may cause a certain time difference between the video and the audio, and finally, when the picture is output through a screen of the smart television and the sound is output through a loudspeaker or an external sound of the smart television, a user may feel that the picture and the sound are not synchronous, so that the watching experience is influenced.
In the prior art, no solution for realizing synchronization of sound and pictures is found in the smart television end.
Disclosure of Invention
The application aims to provide a sound and picture synchronous processing method and a related device, and aims to solve the problem that in the prior art, intelligent audio and video equipment is not synchronous when pictures and sounds are finally output.
In a first aspect, a method for synchronously processing a picture and a sound is provided, which is applied to an intelligent audio-visual device, and includes:
when a picture delay instruction is received, increasing the caching quantity of picture frames of the pictures played by the intelligent audio-video equipment in a cache according to the picture delay instruction;
and when a picture advance instruction is received, the buffer memory is used for reducing the buffer memory quantity of the picture frames of the playing pictures of the intelligent audio-visual equipment according to the picture advance instruction.
Optionally, the method further includes:
receiving the synchronous processing instruction, wherein the synchronous processing instruction is used for starting and adjusting the output coordination of the picture and the sound on the intelligent audio-video equipment;
and activating the buffer in a locking state to adjust the buffer amount of the frame of the playing picture of the intelligent audio-visual equipment according to the synchronous processing instruction.
Optionally, the method further includes:
receiving a synchronous processing completion instruction;
and locking the caching number of the frame of the picture, which is cached by the current cache in the playing picture of the intelligent audio-visual equipment, according to the synchronous processing completion instruction.
Optionally, after the buffer in the locked state is activated according to the synchronous processing instruction to adjust the buffer number of the frame of the played frame of the intelligent audio/video device, the method further includes:
receiving the current sound emitted by the intelligent audio-video equipment when the video is played through a microphone;
judging whether the current sound is aligned with a standard sound corresponding to a picture when the intelligent audio-video equipment plays the video in time;
when the current sound is delayed from the standard sound corresponding to the picture when the intelligent audio-visual equipment plays the video, generating a picture delay instruction;
when the current sound is aligned with the standard sound corresponding to the picture when the intelligent audio-video equipment plays the video in time, the synchronous processing completion instruction is generated;
and when the current sound is ahead of the standard sound corresponding to the picture when the intelligent audio-visual equipment plays the video, generating the picture ahead instruction.
Optionally, the receiving, by the microphone, the current sound emitted when the intelligent audio-visual device plays the video includes:
receiving the current sound emitted by a loudspeaker of the intelligent audio-video equipment when the intelligent audio-video equipment plays a video through a microphone of the intelligent audio-video equipment; or the like, or, alternatively,
and receiving the current sound emitted by the wirelessly connected loudspeaker box when the intelligent audio-video equipment plays the video through the microphone of the intelligent audio-video equipment.
Optionally, the intelligent audio/video device is one of an intelligent television, an intelligent projector, an intelligent mobile phone, an intelligent tablet, a notebook computer and a desktop computer.
Optionally, the receiving the synchronous processing instruction includes:
the synchronous processing instruction is generated by triggering a first physical key of the intelligent audio-visual equipment; or the like, or a combination thereof,
the synchronous processing instruction is generated by triggering a first virtual key of the intelligent audio-video equipment;
the receiving a synchronous processing completion instruction includes:
generating the synchronous processing completion instruction by triggering a second physical key of the intelligent audio-visual equipment; or the like, or, alternatively,
and generating the synchronous processing completion instruction by triggering a second virtual key of the intelligent audio-visual equipment.
In a second aspect, the present application provides a synchronous processing system for pictures and sound, which is applied to an intelligent audio-visual device, and includes:
the increasing unit is used for increasing the caching quantity of the frame of the picture played by the intelligent audio-video equipment in a cache according to the picture delay instruction when the picture delay instruction is received;
and the reduction unit is used for reducing the caching quantity of the frame of the playing picture of the intelligent audio-visual equipment in the cache according to the picture advance instruction when the picture advance instruction is received.
Optionally, the system further includes:
the receiving unit is used for receiving the synchronous processing instruction, and the synchronous processing instruction is used for starting and adjusting the output coordination of the picture and the sound on the intelligent audio-visual equipment;
and the activation unit is used for activating the buffer in the locking state to adjust the buffer amount of the frame frames of the played frame of the intelligent audio-visual equipment according to the synchronous processing instruction.
Optionally, the system further includes:
the receiving unit is also used for receiving a synchronous processing completion instruction;
and the locking unit is used for locking the caching number of the frame of the picture which is cached in the current cache by the intelligent audio-visual equipment according to the synchronous processing completion instruction.
Optionally, the system further includes:
the receiving unit is also used for receiving the current sound emitted by the intelligent audio-video equipment when the video is played through a microphone;
the judging unit is used for judging whether the current sound is aligned with the standard sound corresponding to the picture when the intelligent audio-video equipment plays the video in time;
the generating unit is used for generating the picture delay instruction when the current sound is delayed from the standard sound corresponding to the picture when the intelligent audio-video equipment plays the video;
the generating unit is further used for generating the synchronous processing completion instruction when the current sound is aligned with a standard sound corresponding to a picture when the intelligent audio-video equipment plays the video in time;
the generating unit is further configured to generate the picture advance instruction when the current sound is earlier than the standard sound corresponding to the picture when the intelligent audio/video device plays the video.
Optionally, when the receiving unit receives the current sound emitted by the intelligent audio-video device when playing a video through a microphone, the receiving unit is specifically configured to:
receiving the current sound emitted by a loudspeaker of the intelligent audio-video equipment when the intelligent audio-video equipment plays a video through a microphone of the intelligent audio-video equipment; or the like, or, alternatively,
and receiving the current sound emitted by the wirelessly connected loudspeaker box when the intelligent audio-visual equipment plays the video through the microphone of the intelligent audio-visual equipment.
Optionally, the intelligent audio/video device is one of an intelligent television, an intelligent projector, an intelligent mobile phone, an intelligent tablet, a notebook computer and a desktop computer.
Optionally, when receiving the synchronous processing instruction, the receiving unit is specifically configured to:
the synchronous processing instruction is generated by triggering a first physical key of the intelligent audio-video equipment; or the like, or, alternatively,
the synchronous processing instruction is generated by triggering a first virtual key of the intelligent audio-video equipment;
when the receiving unit receives the synchronization processing completion instruction, the receiving unit is specifically configured to:
generating the synchronous processing completion instruction by triggering a second physical key of the intelligent audio-video equipment; or the like, or, alternatively,
and generating the synchronous processing completion instruction by triggering a second virtual key of the intelligent audio-video equipment.
In a third aspect, the present application provides a computer device comprising:
the system comprises a processor, a memory, a bus, an input/output interface and a network interface;
the processor is connected with the memory, the input and output interface and the network interface through the bus;
the memory stores a program;
the processor, when executing the program stored in the memory, implements the method for processing the screen and the sound synchronously as in any one of the first aspect.
In a fourth aspect, the present application provides a computer storage medium having instructions stored therein, which when executed on a computer, cause the computer to perform the method for synchronous processing of a picture and sound according to any one of the preceding first aspects.
In a fifth aspect, the present application provides a computer program product, which when executed on a computer, causes the computer to execute the method for processing the picture and sound synchronously according to any one of the first aspect.
According to the technical scheme, the embodiment of the application has the following advantages:
the synchronous processing method of the picture and the sound is applied to the intelligent audio-visual equipment, the intelligent audio-visual equipment is provided with a buffer for buffering picture frames of a playing picture in a certain amount, when a picture delay instruction is received, the buffer amount of the buffered picture frames of the playing picture of the intelligent audio-visual equipment is increased in the buffer according to the picture delay instruction, and the delayed adjustment of the playing picture of the intelligent audio-visual equipment is realized on the premise that the playing sound is not changed; when receiving the picture advance instruction, the buffer memory is used for reducing the buffer memory number of the buffer memory picture frame of the playing picture of the intelligent audio-visual equipment according to the picture advance instruction, and the reduction delay adjustment of the playing picture of the intelligent audio-visual equipment is realized on the premise that the playing sound is not changed; because the sound output of the intelligent audio-visual equipment is delayed relative to the picture output when the media resource content is played in the prior art, the implementation can realize the control of different delay time lengths of the picture by controlling the caching quantity of the picture frames of the played picture in the cache, and the synchronization of the picture and the sound output by the intelligent audio-visual equipment is realized by adapting the sound delayed than the played picture in the intelligent audio-visual equipment by controlling the different delay time lengths of the played picture.
Drawings
FIG. 1 is a flowchart illustrating an embodiment of a method for synchronously processing pictures and sounds according to the present application;
FIG. 2 is a flowchart illustrating a method for synchronously processing video and audio according to another embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating an embodiment of a system for synchronous processing of pictures and sounds according to the present application;
FIG. 4 is a schematic diagram of an embodiment of a system for synchronous video and audio processing according to the present application;
FIG. 5 is a schematic structural diagram of an embodiment of a computer apparatus of the present application;
FIG. 6 is a schematic structural connection diagram of an embodiment of an intelligent audio-visual device and a sound box according to the present application;
fig. 7 is a schematic diagram illustrating a data flow of an embodiment of an intelligent audio/video device playing a media resource in the prior art.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It should be noted that the intelligent video and audio device in this embodiment is a device for playing media resources, and the media resources in this embodiment are electronic resources having sounds and pictures on the same track in time, such as videos of dramas and movies. The intelligent audio and video equipment capable of playing the media resources can be one of an intelligent television, an intelligent projector, an intelligent mobile phone, an intelligent tablet, a notebook computer, a desktop computer and the like.
Referring to fig. 7, taking an intelligent audio-video device as an example of an intelligent television, when the intelligent television plays the above media resource, in order to realize respective output of sound and pictures in the media resource, the intelligent television needs to first obtain the media resource 800, and then obtain the media resource 800 after performing operations such as relevant protocol parsing, audio-video decoding, rendering, and the like, respectively: a picture frame sequence set 810 based on the time sequence with the precedence order, and a sound frame sequence combination 820 based on the time sequence with the precedence order; finally, the picture frames 811 in the picture frame sequence set 810 are displayed through the screen of the smart television according to the time sequence, and the sound frames 821 in the sound frame sequence combination 820 are played through the loudspeaker of the smart television according to the time sequence; after the smart television is paired and connected with a sound box with the same bluetooth function through the bluetooth function, the sound box can output the sound frame 821, specifically, the smart television converts the sound frame sequence, combined with the sound frame 821 to be played, into the sound frame 821 ″ that can be transmitted through the bluetooth and transmits the sound frame 821 ″ to the sound box through the bluetooth, after receiving the sound frame 821 ″ by the sound box, the sound box performs corresponding processing to obtain the sound frame 821 to be played, and then the sound box plays the sound frame 821. In the process, from the beginning of audio and video decoding, the video picture and the video sound are processed in different paths, each link after the video picture and the video sound are processed separately may cause a certain time difference between the video picture and the video sound, and finally, when the video picture is output through a screen of the smart television and the video sound is output through a loudspeaker or an external sound box of the smart television, a user may feel that the picture and the sound are not synchronous, so that the watching experience is influenced. Especially when the smart television plays through the external audio amplifier that has bluetooth function of bluetooth wireless connection, the smart television need compress the sound signal and send for the audio amplifier through the bluetooth after handling, the audio amplifier is decompressed the sound of compression again and is handled the back broadcast, compare and carry out the sound through smart television self loudspeaker and play and still need consume more time, may let the user feel the smart television broadcast picture more obviously faster than the sound that hears from the audio amplifier, influence the user and watch experience.
The problem that in the prior art, when the intelligent audio-visual equipment finally outputs pictures and sounds, the pictures and the sounds are not synchronous is solved, and particularly when the intelligent audio-visual equipment plays the sounds of media resources through an external sound box, the sounds played by the sound box and the pictures displayed by the intelligent audio-visual equipment are not synchronous in time sequence. Referring to fig. 6, the intelligent audio-visual device of the embodiment adds a buffer to the conventional intelligent audio-visual device to solve the above technical problem. The method for realizing the delay of the final output picture of the intelligent audio-video equipment can be completed by adopting a buffer to realize frame buffer (FrameBuffer) of the picture frame. The frame sequence set 810 comprises a frame-by-frame static picture (picture frame 811), and each frame needs to be buffered before being displayed. The display module displays the pictures in the buffer one by one, so that the dynamic pictures can be seen. When the user wants the display module to realize the picture delay, the effect of adjusting the display picture delay can be achieved by adjusting the buffer quantity of the picture frames in the buffer. Specifically, the intelligent audio-visual device 600 mainly includes: the device includes a processor 601, a speaker 602, a buffer 603, a display module 604, a communication module 605, a button module 606, and a power module (not shown). The processor 601 is mainly used for performing operations such as relevant protocol analysis, audio/video decoding, rendering and the like on the media resource 800 to obtain a picture frame sequence set 810 based on the sequence in the time sequence and a sound frame sequence combination 820 based on the sequence in the time sequence; the processor 601 then sends the picture frames 811 in the picture frame sequence set 810 to the buffer 603 according to the time sequence order; when the communication module 605 determines that there is no external speaker, the processor 601 further combines the sound frame sequence with the sound frame 821 in 820 according to the time sequence order and transmits the sound frame sequence to the loudspeaker 602; and when the communication module 605 determines that the external speaker 700 is available, the processor 601 further combines the sound frame sequence with the sound frame 821 in the sound frame sequence 820 according to the chronological order and transmits the combined sound frame sequence to the external speaker 700 through the communication module 605. The speaker 602 is mainly used for playing the sound frame 821 transmitted by the processor 601 to realize sound output. The buffer 603 is used for buffering a plurality of frames according to the control logic adjusted by the processor 601 and transmitting the frames to the display module 604 in a first-in first-out sequence. The display module 604 is mainly used for displaying the frame transmitted by the buffer 603. The communication module 605 is mainly used for determining whether to establish a connection with an external speaker, and if the connection (for example, bluetooth connection) with the speaker 700 is successfully established, the communication module feeds back a determination that the external speaker 700 is available to the processor 601, so that the processor 601 abandons the speaker carried by the processor to play the sound frame 821, and preferentially selects the external speaker 700 to play the sound frame 821; if no connection (e.g., bluetooth connection) is established with the speaker 700, the speaker 700 that is not externally connected is fed back to the processor 601, so that the processor 601 plays the sound frame 821 using the speaker carried by itself. The key module 606 is mainly used for receiving an operation instruction of the user for the audio/video enabled device 600, and transmitting the relevant operation instruction to the processor 601 for execution. The power module is mainly used for providing proper working voltage for the processor 601, the speaker 602, the buffer 603, the display module 604, the communication module 605, the key module 606, and the like.
Research shows that the problem that the intelligent audio-visual equipment is asynchronous when finally outputting pictures and sounds, particularly when the intelligent audio-visual equipment plays the sounds of media resources through an external sound box, the sounds played by the sound box and the display pictures of the intelligent audio-visual equipment are asynchronous in time sequence, and the main performance is as follows: the picture played by the intelligent audio-video equipment is faster than the sound. In the face of the technical problem that the picture played by the intelligent audio-video equipment is faster than the sound, there are two general solutions: the first is that the picture playing delay needs to be adjusted on the premise of keeping the continuous playing of the sound unchanged according to the time sequence order, so that the user feels that the picture and the sound are output synchronously; the second is to adjust the sound playing ahead of time on the premise of keeping the continuous playing of the picture unchanged according to the time sequence order, so that the user can feel that the picture and the sound are output synchronously. However, the effect of the second scheme in reality is relatively limited, and especially when the intelligent audio-video device plays the sound of the media resource through the external sound box, the time consumed for sending the sound to the external sound box by the intelligent audio-video device to play is long, and the time for adjusting the sound to play ahead is not long enough, which is usually not enough to make up for the asynchronous time difference between the picture and the sound. The present embodiment mainly adopts the above-described first scheme.
Therefore, based on the above understanding of the intelligent audio-visual device 600 of the present application, please refer to fig. 1, a process for solving the above technical problem is shown in the following embodiment process, and an embodiment of the method for synchronously processing the picture and the sound applied to the intelligent audio-visual device of the present application includes:
101. when receiving the picture delay instruction, increasing the buffer quantity of the picture frames of the picture played by the intelligent audio-video equipment in the buffer according to the picture delay instruction.
When a user watches the video played by the intelligent audio-video equipment and feels that the picture is faster than the sound, the step can receive a picture delay instruction triggered by the user through the key module 606, and then the buffer memory quantity of picture frames of the played picture of the video of the intelligent audio-video equipment is increased in the buffer memory 603 according to the picture delay instruction, so that the picture delay adjustment of the intelligent audio-video equipment is realized on the premise that the played sound is not changed. For example, each time the user triggers a frame delay command (the key module 606 receives the frame delay command issued by the user from the physical key), this step stores multiple X frames of frame frames into the buffer 603, where X is a positive integer greater than or equal to 1. In general, the playing frame of the intelligent audio-visual device is 60Hz, that is, 60 frames per second, the buffer 603 stores 1 frame of the frame, which can bring 1/60 second frame delay, and 6 frame buffers can bring 1/10 second (100 ms) delay. The sound box externally connected in the bluetooth mode usually delays by 150ms for calculation, and the synchronization of the output picture of the intelligent audio-visual equipment and the output sound of the sound box can be realized by storing about 9 picture frames in the buffer 603. 102. When the picture advance instruction is received, the buffer memory is used for reducing the buffer memory quantity of the picture frames of the playing picture of the intelligent video equipment according to the picture advance instruction.
When the user stores too many frames into the buffer 603 in the operation of step 101, and may find that the frame is slower than the sound when watching the video played by the intelligent audio/video device, this step may receive a frame advance command triggered by the user through the key module 606, and then the buffer 603 is used to reduce the buffer number of frames of the played frame of the intelligent audio/video device according to the frame advance command, so as to implement the relative advance adjustment of the played frame of the intelligent audio/video device on the premise of not changing the played sound. For example, each time the user triggers a picture advance command (the picture advance command issued by the user from the physical key is received through the key module 606), the step releases multiple X frames of picture frames from the buffer 603, where X is a positive integer greater than or equal to 1, until no buffered picture frame in the buffer 603 reaches the limit of advance picture adjustment. In general, the playing frame of the smart av device is 60Hz, that is, 60 frames per second, 1 frame released by the buffer 603 can bring 1/60 second frame advance, and 6 frame buffers can bring 1/10 second (100 ms) advance.
It can be known that, in the prior art, usually, the sound output of the intelligent audio-visual device is delayed relative to the picture output when playing the media resource content, the implementation can realize the control of different delay durations of the picture by controlling the buffer amount of the picture frame of the playing picture in the buffer, and realize the synchronization of the picture and the sound output by the intelligent audio-visual device by controlling the different delay durations of the playing picture to adapt to the sound delayed from the playing picture in the intelligent audio-visual device.
Referring to fig. 2, another embodiment of the present application, which is applied to a method for synchronously processing a picture and a sound of an intelligent audio-visual device, includes:
201. and receiving a synchronous processing instruction, wherein the synchronous processing instruction is used for starting the output coordination of the adjusting picture and the sound on the intelligent audio-video equipment.
It can be understood that, since the synchronous calibration of the picture and the sound of the intelligent audio-visual device is a matter which has a certain technical requirement and can directly affect the user experience, for safety, the step needs to explicitly receive the synchronous processing instruction sent by the user to the intelligent audio-visual device, and then the related program is activated, and the synchronous processing instruction of the step refers to the output cooperation of the intelligent audio-visual device for starting and adjusting the picture and the sound, so that the picture and the sound can be perceived to be synchronous at the user level. For example, the process of receiving the synchronization processing instruction triggered by the user in this step may be: the synchronous processing instruction is generated by triggering a first physical key of the intelligent audio-video equipment; or, a first virtual key (a virtual key on a touch screen) of the intelligent audio-video equipment is triggered to generate a synchronous processing instruction. The form of receiving the synchronous processing instruction is not limited herein.
202. And activating the buffer in the locking state to adjust the buffer amount of the frame of the playing picture of the intelligent audio-visual equipment according to the synchronous processing instruction.
For safety, in this step, when it is definitely received that the user sends a synchronization processing instruction to the intelligent audio/video device in step 201, the buffer 603 in the locked state is activated according to the synchronization processing instruction to adjust the buffer number of the frame of the playing picture of the intelligent audio/video device, so as to allow the subsequent step to increase or decrease the buffer number of the frame stored in the buffer 603.
203. And receiving the current sound emitted by the intelligent audio-video equipment when the video is played through the microphone.
In some embodiments, an automatic picture and sound synchronization function of the intelligent audio-visual device may be further implemented, and when the intelligent audio-visual device has a microphone recording function, after the buffer in the locked state is activated in step 202 to adjust the buffer amount of the picture frames of the playing picture of the intelligent audio-visual device, the current sound (emitted by a speaker of the intelligent audio-visual device or an enclosure in wireless connection) emitted by the intelligent audio-visual device when the video is played can be received by the microphone in this step.
204. Judging whether the current sound is aligned with the standard sound corresponding to the picture when the intelligent audio-visual equipment plays the video in time, if the current sound is delayed from the standard sound corresponding to the picture when the intelligent audio-visual equipment plays the video, executing step 205; if the current sound is earlier than the standard sound corresponding to the picture when the intelligent audio-visual device plays the video, step 207 is executed; if the current sound is aligned with the standard sound corresponding to the picture when the video is played by the intelligent audio/video device, step 209 is executed.
It should be noted that, when the intelligent audio-video device performs audio-video decoding on the media resource 800, it can know the corresponding relationship between each frame of picture and the corresponding standard sound in the time sequence, then after the current sound (the speaker of the intelligent audio-video device plays or the external sound box plays) emitted when the intelligent audio-video device plays the video is known in step 203, this step can judge whether the current sound is aligned with the standard sound corresponding to the picture when the intelligent audio-video device plays the video in time, if the current sound is delayed from the standard sound corresponding to the picture when the intelligent audio-video device plays the video, the time difference between the current sound and the standard sound corresponding to the picture can be calculated, and further calculate how many frames of picture frames are needed by the time difference to store in the buffer; if the current sound is earlier than the standard sound corresponding to the picture when the intelligent audio-video equipment plays the video, the time difference between the current sound and the standard sound corresponding to the picture can be calculated, and further the number of picture frames which need to be released from the buffer by the time difference is calculated; if the current sound is aligned with the standard sound corresponding to the picture when the intelligent audio-video equipment plays the video in time, the picture and the current sound are proved to be synchronous when the intelligent audio-video equipment plays the video.
205. And generating a picture delay instruction.
When it is determined in step 204 that the current sound is later than the standard sound corresponding to the picture when the intelligent audio/video device plays the video, the step actively generates or calls a preset picture delay instruction, wherein the picture delay instruction is used for increasing the caching number of the picture frames in the buffer.
206. And increasing the caching quantity of the frame of the picture played by the intelligent audio-video equipment in the cache according to the picture delay instruction.
In this step, according to the picture delay instruction in step 205, the calculated number of picture frames that need to be stored in the buffer is usually carried together with the picture delay instruction, and then the corresponding number of buffer storage numbers for the picture frames of the playing picture of the intelligent audio-visual device is added in the buffer in this step, and step 204 is continuously triggered and executed.
207. A picture advance command is generated.
After it is determined in step 204 that the current sound is earlier than the standard sound corresponding to the picture when the intelligent audio-visual device plays the video, a preset picture advance command is actively generated or invoked in this step, and the picture advance command is used for reducing the number of the buffered picture frames from the buffer.
208. And the buffer memory is used for reducing the buffer memory quantity of the frame of the playing picture of the intelligent video and audio equipment according to the picture advance instruction.
In this step, according to the picture advance command in step 207, the calculated number of buffered picture frames that need to be reduced from the buffer is carried along with the picture advance command, and then the corresponding number of buffered picture frames for the playing picture of the intelligent audio-visual device is reduced in the buffer in this step, and step 204 is triggered and executed continuously.
209. And generating a synchronous processing completion instruction.
After it is determined in step 204 that the current sound is temporally aligned with the standard sound corresponding to the picture of the smart audio-visual device playing the video, it is verified that the picture of the smart audio-visual device playing the video is synchronized with the current sound.
It should be noted that, in some embodiments, the synchronization processing completion instruction of this step may also be: triggering and generating a second physical key of the intelligent audio-video equipment; or, a virtual key on a second virtual key touch screen of the intelligent audio-video equipment) is triggered and generated.
210. And locking the caching number of the frame of the playing picture of the intelligent audio-visual equipment cached by the current cache according to the synchronous processing completion instruction.
In this step, the number of frames in the buffer for buffering the playing frames of the intelligent audio/video device in the buffer is locked according to the synchronization processing completion instruction in step 209, so as to complete the automatic synchronization of the frames and sound of the intelligent audio/video device.
The foregoing embodiment describes the method for synchronously processing the picture and the sound applied to the intelligent audio-visual device, and the following describes the system for synchronously processing the picture and the sound applied to the intelligent audio-visual device, please refer to fig. 3, and an embodiment of the system for synchronously processing the picture and the sound applied to the intelligent audio-visual device includes:
the adding unit 301 is configured to, when a picture delay instruction is received, add, in a buffer, the number of buffered picture frames of a picture played by the intelligent audio-video device according to the picture delay instruction;
the reducing unit 302 is configured to reduce, in a buffer according to a picture advance instruction when the picture advance instruction is received, the number of buffered picture frames of a playing picture of the intelligent audio/video device.
The operation performed by the synchronous processing system for pictures and sound applied to the intelligent audio-visual device in this embodiment is similar to the operation performed in the embodiment of fig. 1, and is not repeated herein.
The synchronous processing system of the picture and the sound is applied to the intelligent audio-visual equipment, the intelligent audio-visual equipment is provided with a buffer for buffering picture frames of a playing picture in a certain amount, when a picture delay instruction is received, the buffer amount of the buffered picture frames of the playing picture of the intelligent audio-visual equipment is increased in the buffer according to the picture delay instruction, and the delayed adjustment of the playing picture of the intelligent audio-visual equipment is realized on the premise that the playing sound is not changed; when a picture advance instruction is received, the buffer memory is used for reducing the buffer memory number of the buffer memory picture frame of the playing picture of the intelligent audio-video equipment according to the picture advance instruction, and the reduction delay adjustment of the playing picture of the intelligent audio-video equipment is realized on the premise that the playing sound is not changed; because the sound output of the intelligent audio-visual equipment is delayed relative to the picture output when the media resource content is played in the prior art, the implementation can realize the control of different delay time lengths of the picture by controlling the caching quantity of the picture frames of the played picture in the cache, and the synchronization of the picture and the sound output by the intelligent audio-visual equipment is realized by adapting the sound delayed than the played picture in the intelligent audio-visual equipment by controlling the different delay time lengths of the played picture.
Referring to fig. 4, another embodiment of the present application, which is applied to a system for synchronously processing a picture and a sound of an intelligent audio-visual device, includes:
the adding unit 401 is configured to, when a picture delay instruction is received, add, in a buffer, the number of buffered picture frames of a picture played by the intelligent audio-visual device according to the picture delay instruction;
a reducing unit 402, configured to, when receiving a picture advance instruction, reduce, in a buffer, the number of buffered picture frames of a playing picture of the intelligent audio/video device according to the picture advance instruction.
Optionally, the system further includes:
a receiving unit 403, configured to receive the synchronous processing instruction, where the synchronous processing instruction is used to start output coordination between the picture and the sound in the intelligent audio/video device;
and an activating unit 404, configured to activate, according to the synchronization processing instruction, a function of adjusting, by the buffer in a locked state, the buffer number of the frame frames of the playing frame of the intelligent audio/video device.
Optionally, the system further includes:
the receiving unit 403 is further configured to receive a synchronization processing completion instruction;
and a locking unit 405, configured to lock, according to the synchronization processing completion instruction, the buffer amount of the frame frames currently buffering the played frame of the intelligent audio/video device by the buffer.
Optionally, the system further includes:
the receiving unit 403 is further configured to receive, through a microphone, current sound emitted by the intelligent audio/video device when playing a video;
a determining unit 406, configured to determine whether the current sound is time-aligned with a standard sound corresponding to a picture when the intelligent audio/video device plays a video;
the generating unit 407 is configured to generate the picture delay instruction when the current sound is delayed from a standard sound corresponding to a picture when the intelligent audio/video device plays a video;
the generating unit 407 is further configured to generate the synchronous processing completion instruction when the current sound is aligned with a standard sound corresponding to a picture when the intelligent audio/video device plays a video in time;
the generating unit 407 is further configured to generate the picture advance instruction when the current sound is earlier than a standard sound corresponding to a picture when the intelligent audio/video device plays a video.
Optionally, when the receiving unit 403 receives the current sound emitted by the intelligent audio/video device when playing a video through a microphone, the receiving unit is specifically configured to:
receiving the current sound emitted by a loudspeaker of the intelligent audio-video equipment when the intelligent audio-video equipment plays a video through a microphone of the intelligent audio-video equipment; or the like, or a combination thereof,
and receiving the current sound emitted by the wirelessly connected loudspeaker box when the intelligent audio-visual equipment plays the video through the microphone of the intelligent audio-visual equipment.
Optionally, the intelligent audio/video device is one of an intelligent television, an intelligent projector, an intelligent mobile phone, an intelligent tablet, a notebook computer and a desktop computer.
Optionally, when the receiving unit 403 receives the synchronization processing instruction, the receiving unit is specifically configured to:
the synchronous processing instruction is generated by triggering a first physical key of the intelligent audio-visual equipment; or the like, or a combination thereof,
the synchronous processing instruction is generated by triggering a first virtual key of the intelligent audio-video equipment;
when the receiving unit 403 receives the synchronization processing completion instruction, it is specifically configured to:
generating the synchronous processing completion instruction by triggering a second physical key of the intelligent audio-video equipment; or the like, or, alternatively,
and generating the synchronous processing completion instruction by triggering a second virtual key of the intelligent audio-visual equipment.
The operation performed by the synchronous processing system for frames and sounds applied to the intelligent audio-visual device in this embodiment is similar to the operation performed in the embodiment of fig. 1 or fig. 2, and is not repeated herein.
Referring to fig. 5, a computer device in an embodiment of the present application is described below, where an embodiment of the computer device in the embodiment of the present application includes:
the computer device 500 may include one or more processors (CPUs) 501 and a memory 502, where the memory 502 stores one or more applications or data. Wherein memory 502 is volatile storage or persistent storage. The program stored in memory 502 may include one or more modules, each of which may include a sequence of instructions operating on a computer device. Still further, the processor 501 may be arranged in communication with the memory 502 to execute a series of instruction operations in the memory 502 on the computer device 500. The computer device 500 may also include one or more network interfaces 503, one or more input-output interfaces 504, and/or one or more operating systems, such as Windows Server, mac OS, unix, linux, freeBSD, etc. The processor 501 may perform the operations performed in the embodiments shown in fig. 1 or fig. 2, which are not described herein again.
In the several embodiments provided in the embodiments of the present application, it should be understood by those skilled in the art that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the unit is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. The integrated unit, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, which are essential or part of the technical solutions contributing to the prior art, or all or part of the technical solutions, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the present application, and any modifications, equivalents, improvements, etc. made within the spirit and scope of the present application should be included in the present application.
Claims (10)
1. The synchronous processing method of the picture and the sound is characterized in that the method is applied to intelligent audio-visual equipment and comprises the following steps:
when a picture delay instruction is received, increasing the caching quantity of picture frames of pictures played by the intelligent audio-video equipment in a cache according to the picture delay instruction;
and when a picture advance instruction is received, the buffer memory is used for reducing the buffer memory quantity of the picture frames of the playing pictures of the intelligent audio-visual equipment according to the picture advance instruction.
2. The synchronous processing method according to claim 1, characterized in that the method further comprises:
receiving the synchronous processing instruction, wherein the synchronous processing instruction is used for starting and adjusting the output coordination of the picture and the sound on the intelligent audio-video equipment;
and activating the buffer in a locking state to adjust the buffer amount of the frame of the playing picture of the intelligent audio-visual equipment according to the synchronous processing instruction.
3. The synchronous processing method according to claim 2, characterized in that the method further comprises:
receiving a synchronous processing completion instruction;
and locking the caching number of the frame of the picture, which is cached by the current cache in the playing picture of the intelligent audio-visual equipment, according to the synchronous processing completion instruction.
4. The synchronous processing method according to claim 3, wherein after the buffer in the locked state is activated according to the synchronous processing instruction to adjust the buffer amount of the frame played by the intelligent video/audio device, the method further comprises:
receiving the current sound emitted by the intelligent audio-video equipment when the video is played through a microphone;
judging whether the current sound is aligned with a standard sound corresponding to a picture when the intelligent audio-video equipment plays the video in time;
when the current sound is delayed from the standard sound corresponding to the picture when the intelligent audio-visual equipment plays the video, generating a picture delay instruction;
when the current sound is aligned with the standard sound corresponding to the picture when the intelligent audio-video equipment plays the video in time, the synchronous processing completion instruction is generated;
and when the current sound is ahead of the standard sound corresponding to the picture when the intelligent audio-visual equipment plays the video, generating the picture ahead instruction.
5. The synchronous processing method according to claim 4, wherein the receiving, by a microphone, the current sound emitted by the smart audio/video device while playing the video comprises:
receiving the current sound emitted by a loudspeaker of the intelligent audio-video equipment when the intelligent audio-video equipment plays a video through a microphone of the intelligent audio-video equipment; or the like, or a combination thereof,
and receiving the current sound emitted by the wirelessly connected loudspeaker box when the intelligent audio-video equipment plays the video through the microphone of the intelligent audio-video equipment.
6. The synchronous processing method according to claim 4, wherein the smart video device is one of a smart television, a smart projector, a smart phone, a smart tablet, a notebook computer, and a desktop computer.
7. The synchronous processing method according to claim 3, wherein the receiving the synchronous processing instruction includes:
the synchronous processing instruction is generated by triggering a first physical key of the intelligent audio-video equipment; or the like, or, alternatively,
the synchronous processing instruction is generated by triggering a first virtual key of the intelligent audio-visual equipment;
the receiving of the synchronous processing completion instruction comprises:
generating the synchronous processing completion instruction by triggering a second physical key of the intelligent audio-video equipment; or the like, or a combination thereof,
and generating the synchronous processing completion instruction by triggering a second virtual key of the intelligent audio-video equipment.
8. Synchronous processing system of picture and sound, its characterized in that, be applied to intelligent audio-visual equipment, include:
the increasing unit is used for increasing the caching quantity of the frame of the picture played by the intelligent audio-visual equipment in a cache according to the picture delay instruction when the picture delay instruction is received;
and the reducing unit is used for reducing the buffer amount of the frame of the playing picture of the intelligent audio-visual equipment in a buffer according to the picture advance instruction when the picture advance instruction is received.
9. A computer device, comprising:
the system comprises a processor, a memory, a bus, an input/output interface and a network interface;
the processor is connected with the memory, the input and output interface and the network interface through the bus;
the memory stores a program;
the processor, when executing the program stored in the memory, implements the method for synchronous processing of the picture and sound according to any one of claims 1 to 7.
10. A computer storage medium having stored therein instructions that, when executed on a computer, cause the computer to execute the method of processing the picture and sound in synchronization as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211368580.0A CN115720278A (en) | 2022-11-03 | 2022-11-03 | Synchronous processing method of sound and picture and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211368580.0A CN115720278A (en) | 2022-11-03 | 2022-11-03 | Synchronous processing method of sound and picture and related device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115720278A true CN115720278A (en) | 2023-02-28 |
Family
ID=85254667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211368580.0A Pending CN115720278A (en) | 2022-11-03 | 2022-11-03 | Synchronous processing method of sound and picture and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115720278A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004170444A (en) * | 2002-11-15 | 2004-06-17 | Casio Comput Co Ltd | Electronic equipment, instruction code generating device and electronic equipment control program, instruction code generation processing program |
| CN103167342A (en) * | 2013-03-29 | 2013-06-19 | 天脉聚源(北京)传媒科技有限公司 | Audio and video synchronous processing device and method |
| CN204305260U (en) * | 2014-12-28 | 2015-04-29 | 冠捷显示科技（厦门）有限公司 | The television set of a kind of video and wireless sound box Audio Matching |
| CN105376628A (en) * | 2014-08-27 | 2016-03-02 | 深圳Tcl新技术有限公司 | Method and device for synchronizing audio-video signals |
| CN107801080A (en) * | 2017-11-10 | 2018-03-13 | 普联技术有限公司 | A kind of audio and video synchronization method, device and equipment |
| CN108449617A (en) * | 2018-02-11 | 2018-08-24 | 浙江大华技术股份有限公司 | A kind of method and device of control audio-visual synchronization |
| CN109167890A (en) * | 2018-08-22 | 2019-01-08 | 青岛海信电器股份有限公司 | A kind of sound draws synchronous method and device and display equipment |
| CN111432258A (en) * | 2019-04-29 | 2020-07-17 | 杭州海康威视数字技术股份有限公司 | Audio and video signal synchronization method and device |
| CN113301410A (en) * | 2021-04-30 | 2021-08-24 | 西安诺瓦星云科技股份有限公司 | Audio picture synchronization method, device and system and transmission controller |
| CN114302021A (en) * | 2021-04-30 | 2022-04-08 | 海信视像科技股份有限公司 | Display device and sound picture synchronization method |
-
2022
- 2022-11-03 CN CN202211368580.0A patent/CN115720278A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004170444A (en) * | 2002-11-15 | 2004-06-17 | Casio Comput Co Ltd | Electronic equipment, instruction code generating device and electronic equipment control program, instruction code generation processing program |
| CN103167342A (en) * | 2013-03-29 | 2013-06-19 | 天脉聚源(北京)传媒科技有限公司 | Audio and video synchronous processing device and method |
| CN105376628A (en) * | 2014-08-27 | 2016-03-02 | 深圳Tcl新技术有限公司 | Method and device for synchronizing audio-video signals |
| CN204305260U (en) * | 2014-12-28 | 2015-04-29 | 冠捷显示科技（厦门）有限公司 | The television set of a kind of video and wireless sound box Audio Matching |
| CN107801080A (en) * | 2017-11-10 | 2018-03-13 | 普联技术有限公司 | A kind of audio and video synchronization method, device and equipment |
| CN108449617A (en) * | 2018-02-11 | 2018-08-24 | 浙江大华技术股份有限公司 | A kind of method and device of control audio-visual synchronization |
| CN109167890A (en) * | 2018-08-22 | 2019-01-08 | 青岛海信电器股份有限公司 | A kind of sound draws synchronous method and device and display equipment |
| CN111432258A (en) * | 2019-04-29 | 2020-07-17 | 杭州海康威视数字技术股份有限公司 | Audio and video signal synchronization method and device |
| CN113301410A (en) * | 2021-04-30 | 2021-08-24 | 西安诺瓦星云科技股份有限公司 | Audio picture synchronization method, device and system and transmission controller |
| CN114302021A (en) * | 2021-04-30 | 2022-04-08 | 海信视像科技股份有限公司 | Display device and sound picture synchronization method |
Non-Patent Citations (1)
| Title |
|---|
| 王国晖;朱振华;张科;魏铮;王贞松;: "一种新的高清数字电影播放系统的高速缓存和音视频同步设计", 高技术通讯, no. 08, 15 August 2008 (2008-08-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2892239A1 (en) | Living room computer with small form-factor pluggable port | |
| US10992451B2 (en) | Audio and video playback system and method for playing audio data applied thereto | |
| CN111541913B (en) | Video playing method and device of spliced screen, computer equipment and medium | |
| JP2019135828A (en) | Method for transmitting video frames from video stream to display and corresponding device | |
| WO2020038344A1 (en) | Method and device for audio and video synchronization, and display apparatus | |
| US20130166769A1 (en) | Receiving device, screen frame transmission system and method | |
| TWI663875B (en) | Video processing method and device thereof | |
| CN114554277B (en) | Multimedia processing method, device, server and computer readable storage medium | |
| JP2017147594A (en) | Audio apparatus | |
| CN112423074B (en) | Audio and video synchronization processing method and device, electronic equipment and storage medium | |
| EP4202912A1 (en) | Image processing method and apparatus for virtual reality device, and virtual reality device | |
| US10506138B2 (en) | Driving displays at cinematic frame rates | |
| CN115720278A (en) | Synchronous processing method of sound and picture and related device | |
| US20230016118A1 (en) | System and method to synchronize rendering of multi-channel audio to video presentation | |
| CN114095771B (en) | Audio and video synchronization method, storage medium and electronic equipment | |
| CN111416994B (en) | Method and device for synchronously presenting video stream and tracking information and electronic equipment | |
| WO2024114171A1 (en) | Method and apparatus for stabilizing image frames | |
| CN111586152A (en) | Bluetooth transmission data playing method and device and Bluetooth transmission data sending method and device | |
| CN115811628A (en) | Synchronous processing method of sound and picture and related device | |
| US11689690B2 (en) | Method and device for audio and video synchronization | |
| US11546675B2 (en) | Methods, systems, and media for streaming video content using adaptive buffers | |
| WO2023160100A1 (en) | Display device, external device, and audio playing and sound effect processing method | |
| WO2024093570A1 (en) | Screen projection method and apparatus | |
| US20240048727A1 (en) | Method and system of low latency video coding with interactive applications | |
| CN116017012A (en) | Multi-screen synchronization method, device, display equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |