CN115065852B - Sound and picture synchronization method and device, electronic equipment and readable storage medium - Google Patents

Sound and picture synchronization method and device, electronic equipment and readable storage medium Download PDF

Info

Publication number
CN115065852B
CN115065852B CN202211002992.2A CN202211002992A CN115065852B CN 115065852 B CN115065852 B CN 115065852B CN 202211002992 A CN202211002992 A CN 202211002992A CN 115065852 B CN115065852 B CN 115065852B
Authority
CN
China
Prior art keywords
data
white noise
audio
time point
sound
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211002992.2A
Other languages
Chinese (zh)
Other versions
CN115065852A (en
Inventor
丁龙河
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Skyworth RGB Electronics Co Ltd
Original Assignee
Shenzhen Skyworth RGB Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Skyworth RGB Electronics Co Ltd filed Critical Shenzhen Skyworth RGB Electronics Co Ltd
Priority to CN202211002992.2A priority Critical patent/CN115065852B/en
Publication of CN115065852A publication Critical patent/CN115065852A/en
Application granted granted Critical
Publication of CN115065852B publication Critical patent/CN115065852B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43072Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439Processing of audio elementary streams
    • H04N21/4394Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The application discloses a sound and picture synchronization method, a device, electronic equipment and a readable storage medium, which are applied to the technical field of screen projection, wherein the sound and picture synchronization method comprises the following steps: acquiring video data through network transmission, and acquiring audio data through Bluetooth transmission, wherein the audio data comprises white noise data and screen projection sound data; extracting a video time stamp from the video data, analyzing the white noise data into white noise audio content, and extracting an audio time stamp according to the white noise audio content; and carrying out synchronous processing on the time point of the video data, the time point of the white noise data and the time point of the screen projection sound data according to the video time stamp and the audio time stamp, and playing the video data, the white noise data and the screen projection sound data after the video data, the white noise data and the screen projection sound data are synchronized. The application solves the technical problem that the sound and the picture are not synchronous when the screen is projected based on the private protocol in the prior art.

Description

Sound and picture synchronization method and device, electronic equipment and readable storage medium
Technical Field
The present application relates to the field of screen projection technologies, and in particular, to a method and an apparatus for synchronizing audio and video, an electronic device, and a readable storage medium.
Background
With the rapid development of screen projection technology, the requirements for screen projection quality are gradually increased. When each manufacturer projects screens based on the private screen projecting protocol realized by the manufacturer, the application program of the corresponding manufacturer needs to be installed on the mobile phone. However, for an android mobile phone, the security protocol of the android bottom layer does not allow the current private protocol to process a sound source, and a picture and a sound cannot be synchronously projected to a television terminal, so that the condition that the sound picture is asynchronous is easily caused during screen projection and playing.
Disclosure of Invention
The application mainly aims to provide a sound and picture synchronization method, a sound and picture synchronization device, an electronic device and a readable storage medium, and aims to solve the technical problem that sound and pictures are not synchronized when a screen is projected based on a private protocol in the prior art.
In order to achieve the above object, the present application provides a sound and picture synchronization method, applied to a sound and picture synchronization device, the sound and picture synchronization method including:
acquiring video data through network transmission, and acquiring audio data through Bluetooth transmission, wherein the audio data comprises white noise data and screen projection sound data;
extracting a video time stamp from the video data, analyzing the white noise data into white noise audio content, and extracting an audio time stamp according to the white noise audio content;
and carrying out synchronous processing on the time point of the video data, the time point of the white noise data and the time point of the screen projection sound data according to the video time stamp and the audio time stamp, and playing the video data, the white noise data and the screen projection sound data after the video data, the white noise data and the screen projection sound data are synchronized.
Optionally, the step of extracting an audio time stamp from the white noise audio content includes:
determining the audio characteristic change information of the white noise audio content, and recoding the white noise audio content into characteristic coding data according to the audio characteristic change information;
and extracting the audio time stamp in the characteristic coding data.
Optionally, the step of re-encoding the white noise audio content into feature encoded data according to the audio feature variation information includes:
dividing the white noise audio content into a plurality of sections of audio contents according to the audio characteristic change information;
determining audio characteristic change information of each section of audio content, and coding each section of audio content into a corresponding characteristic code according to a preset white noise coding rule and the audio characteristic change information;
each bit of feature code is combined into feature encoded data.
Optionally, the step of extracting the audio time stamp in the feature encoding data includes:
reading a piece of time point data from the feature coded data according to preset digit at intervals;
and converting the read time point data into an audio time stamp.
Optionally, the step of reading a piece of time point data from the feature encoded data every preset number of bits includes:
taking the feature coded data of each preset digit as a group of time point coded data;
converting the time point coded data into natural number coded data conforming to a preset format;
the data of each natural number is used as time point data.
Optionally, the step of performing three-way synchronization processing on the time point of the video data, the time point of the white noise data, and the time point of the screen projecting sound data according to the video timestamp and the audio timestamp, and playing the video data, the white noise data, and the screen projecting sound data after synchronization includes:
determining a first time point corresponding to the video data according to the video timestamp;
determining a second time point corresponding to the white noise data and a third time point corresponding to the screen projection sound data according to the audio time stamp;
playing the video data and continuously monitoring a first time point of the video data;
determining a first matching time point matched with the first time point in the second time point, and playing white noise data corresponding to the first matching time point;
and determining a second matching time point matched with the first time point in the third time point, and playing the screen projection sound data corresponding to the second matching time point.
Optionally, after the step of playing the screen projection sound data corresponding to the second matching time point, the method further includes:
detecting whether the video data, the white noise data and the screen projection sound data which are being played are synchronously played at each interval preset detection time point according to the video time stamp and the audio time stamp;
if the video data, the white noise data and the screen projection sound data are detected not to be synchronously played, determining whether the white noise data and the screen projection sound data are played faster than the video data;
if the white noise data is determined to be faster than the video data, pausing the playing of the white noise data and the screen projection sound data until the playing of the video data, the playing of the white noise data and the playing of the screen projection sound data are synchronous;
and if the white noise data is determined to be slower than the video data, accelerating the playing of the white noise data and the screen projection sound data until the playing of the video data, the playing of the white noise data and the playing of the screen projection sound data are synchronous.
In order to achieve the above object, the present application further provides a sound and picture synchronization device, where the sound and picture synchronization device is applied to a sound and picture synchronization apparatus, the sound and picture synchronization device includes:
the data acquisition module is used for acquiring video data through network transmission and acquiring audio data through Bluetooth transmission, wherein the audio data comprises white noise data and screen projection sound data;
the time stamp module is used for extracting a video time stamp from the video data, analyzing the white noise data into white noise audio content, and extracting an audio time stamp according to the white noise audio content;
and the synchronization module is used for carrying out synchronous processing on the time point of the video data, the time point of the white noise data and the time point of the screen projection sound data according to the video time stamp and the audio time stamp, and playing the video data, the white noise data and the screen projection sound data after synchronization.
Optionally, the timestamp module is further configured to:
determining audio characteristic change information of the white noise audio content, and recoding the white noise audio content into characteristic coded data according to the audio characteristic change information;
and extracting the audio time stamp in the characteristic coding data.
Optionally, the timestamp module is further configured to:
dividing the white noise audio content into a plurality of sections of audio contents according to the audio characteristic change information;
determining audio characteristic change information of each section of audio content, and coding each section of audio content into a corresponding characteristic code according to a preset white noise coding rule and the audio characteristic change information;
each bit of feature code is combined into feature encoded data.
Optionally, the timestamp module is further configured to:
reading a piece of time point data from the feature coded data according to preset digit at intervals;
and converting the read time point data into an audio time stamp.
Optionally, the timestamp module is further configured to:
taking the feature coded data of each preset digit as a group of time point coded data;
converting the time point coded data into natural number coded data conforming to a preset format;
the data of each natural number is used as time point data.
Optionally, the synchronization module is further configured to:
determining a first time point corresponding to the video data according to the video timestamp;
determining a second time point corresponding to the white noise data and a third time point corresponding to the screen projection sound data according to the audio time stamp;
playing the video data and continuously monitoring a first time point of the video data;
determining a first matching time point matched with the first time point in the second time point, and playing white noise data corresponding to the first matching time point;
and determining a second matching time point matched with the first time point in the third time point, and playing the screen projection voice data corresponding to the second matching time point.
Optionally, the synchronization module is further configured to:
detecting whether the video data, the white noise data and the screen projection sound data which are being played are synchronously played at each interval preset detection time point according to the video time stamp and the audio time stamp;
if the video data, the white noise data and the screen projection sound data are detected not to be synchronously played, determining whether the white noise data and the screen projection sound data are played faster than the video data;
if the white noise data is determined to be faster than the video data, pausing the playing of the white noise data and the screen projection sound data until the playing of the video data, the playing of the white noise data and the playing of the screen projection sound data are synchronous;
and if the white noise data is determined to be slower than the video data, accelerating the playing of the white noise data and the screen projection sound data until the playing of the video data, the playing of the white noise data and the playing of the screen projection sound data are synchronous.
The present application further provides an electronic device, the electronic device including: a memory, a processor and a program of the sound-picture synchronization method stored on the memory and executable on the processor, which program, when executed by the processor, may implement the steps of the sound-picture synchronization method as described above.
The present application further provides a readable storage medium, which is a computer readable storage medium, on which a program for implementing the sound-picture synchronization method is stored, and when the program of the sound-picture synchronization method is executed by a processor, the steps of the sound-picture synchronization method are implemented.
The present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the sound-picture synchronization method as described above.
The application provides a sound and picture synchronization method, a device, electronic equipment and a readable storage medium, namely, video data are acquired through network transmission, and audio data are acquired through Bluetooth transmission, wherein the audio data comprise white noise data and screen projection sound data; extracting a video time stamp from the video data, analyzing the white noise data into white noise audio content, and extracting an audio time stamp according to the white noise audio content; the time point of the video data, the time point of the white noise data and the time point of the screen projecting sound data are synchronously processed according to the video time stamp and the audio time stamp, the video data, the white noise data and the screen projecting sound data are synchronously played, and the audio time stamp is converted into the white noise data in advance, so the white noise data is converted into the audio time stamp again according to the obtained white noise data, sound and picture synchronization can be carried out according to the audio time stamp and the video time stamp, the problem that sound and picture are not synchronous due to the fact that the sound and picture are in different transmission modes in the playing process is solved, the sound and the picture are synchronously played in the playing process, and the quality of screen projecting playing is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic flow chart of a first embodiment of a sound-picture synchronization method according to the present application;
FIG. 2 is a schematic diagram of interaction at two ends of the sound-picture synchronization method of the present application;
FIG. 3 is a flowchart illustrating a second embodiment of a method for synchronizing audio and video signals according to the present application;
FIG. 4 is a schematic structural diagram of a sound-picture synchronization apparatus according to the present application;
fig. 5 is a schematic device structure diagram of a hardware operating environment related to a sound and picture synchronization method in an embodiment of the present application.
The objectives, features, and advantages of the present application will be further described with reference to the accompanying drawings.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below. It should be apparent that the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
Example one
In a first embodiment of the sound-picture synchronization method of the present application, referring to fig. 1, the sound-picture synchronization method includes:
step S10, video data are acquired through network transmission, and audio data are acquired through Bluetooth transmission, wherein the audio data comprise white noise data and screen projection sound data;
step S20, extracting a video time stamp from the video data, analyzing the white noise data into white noise audio content, and extracting an audio time stamp according to the white noise audio content;
and S30, carrying out synchronous processing on the time point of the video data, the time point of the white noise data and the time point of the screen projection sound data according to the video time stamp and the audio time stamp, and playing the video data, the white noise data and the screen projection sound data after the synchronization.
In this embodiment, it should be noted that, this embodiment is based on a private screen projection protocol, a request screen projection end is An Zhuoduan, a screen projection implementation end is a television end, and the application is applied to a screen projection transmitter, a public number of the screen projection transmitter is CN216565435U, where a screen projection transmitting end is connected to an android device, and a screen projection receiving end is connected to a television, the currently used private screen projection protocol is a screen projection protocol implemented by a manufacturer, a corresponding application program needs to be installed on An Zhuoduan, screen projection is implemented through the application program, but most of android devices can only obtain pictures through the application program, sound needs to be obtained through bluetooth transmission, but based on the current bluetooth protocol, this transmission mode cannot stamp audio through a conventional verification mode, and therefore the sound and pictures cannot be synchronized during screen projection.
The audio data comprises white noise data and screen projection sound data, the white noise data is sound beyond the auditory range of human ears, generally 20hz to 20kHz, when the audio data is played, the white noise data cannot be heard by human ears, the white noise can be ultrasonic waves, the screen projection sound data is sound data needing to be played when the screen is projected, the white noise content is An Zhuoduan, the audio time stamp is converted into the white noise data in a mode of representing the audio time stamp in a special coding mode, then the white noise data is transmitted to a television end and is converted into the audio time stamp again, and therefore the problem that the time stamp cannot be printed is solved.
Illustratively, steps S10 to S30 include:
the video data relevant to the screen projection picture is acquired through network transmission, the audio data is acquired through Bluetooth transmission, the video time stamp is extracted from the video data, white noise data in the audio data are directly converted into white noise audio content according to coding rules, the audio time stamp is extracted from the white noise audio content, the video data and each time point in the audio data playing process are synchronously played according to the video time stamp and the audio time stamp, and therefore sound and picture synchronization is carried out when the video data and the audio data are played.
As shown in the schematic diagram of fig. 2, at An Zhuoduan, the video data is marked with a video timestamp, the video data is directly sent to the television via the application program, meanwhile, the audio timestamp is sent to the television via the screen projection sound data according to the special coding form to obtain the corresponding white noise audio content, the white noise audio content is converted into transmittable white noise data, and the white noise data and the screen projection sound data are synchronously sent to the television via the bluetooth module.
In step S30, the step of synchronizing the respective time points of the video data and the audio data according to the video time stamp and the audio time stamp includes:
step S31, determining a first time point corresponding to the video data according to the video time stamp;
step S32, determining a second time point corresponding to the white noise data and a third time point corresponding to the screen projection sound data according to the audio time stamp;
step S33, playing the video data and continuously monitoring a first time point of the video data;
step S34, determining a first matching time point matched with the first time point in the second time point, and playing white noise data corresponding to the first matching time point;
step S35, determining a second matching time point matched with the first time point in the third time point, and playing the screen projection sound data corresponding to the second matching time point.
In this embodiment, it should be noted that the white noise data and the screen projecting sound data are transmitted and played synchronously, but for human ears, the white noise is a sound that cannot be received by human ears, so even if the white noise data is played synchronously, the white noise data cannot be received by human ears, and the screen projecting sound is not scrambled, and the like.
Exemplarily, steps S31 to S35 include:
determining a first time point when video data is being played according to the video time stamp, determining a second time point when white noise data is being played and a third time point when screen projection sound data is being played according to the audio time stamp, playing the video data, continuously monitoring the first time point of the video data, determining a first matching time point matched with the first time point in the second time point, playing the white noise data corresponding to the first matching time point, determining a second matching time point matched with the first time point in the third time point, and playing the screen projection sound data corresponding to the second matching time point. If the video data and the screen projection sound data are not synchronized, the playing is rapidly carried out according to the time difference, if the time is lagged, the playing is carried out after waiting, if the time is advanced, the white noise in the embodiment can not be heard by human ears, and therefore, when the screen projection playing is carried out, a user can only receive the video data and the screen projection sound data which are synchronously played.
After step S33, after the step of playing the screen projection sound data corresponding to the second matching time point, the method further includes:
step A10, detecting whether the video data, the white noise data and the screen projection sound data which are being played are synchronously played or not at every preset detection time point according to the video time stamp and the audio time stamp;
step A20, if it is detected that the video data, the white noise data and the screen projection sound data are not synchronously played, determining whether the playing of the white noise data and the screen projection sound data is faster than the playing of the video data;
step A30, if the video data is determined to be faster than the video data, suspending the playing of the white noise data and the screen projection sound data until the video data, the white noise data and the screen projection sound data are synchronously played;
and step A40, if the video data is determined to be slower than the video data, accelerating the playing of the white noise data and the screen projection sound data until the video data, the white noise data and the screen projection sound data are synchronously played.
In this embodiment, it should be noted that the preset detection time point is a time point of detecting the sound-picture synchronization at a fixed time preset by the system.
Illustratively, steps a10 to a40 include:
detecting whether the video data, the white noise data and the screen projection sound data which are being played are synchronously played at each interval preset detection time point according to the video time stamp and the audio time stamp; if the video data, the white noise data and the screen projection sound data are detected not to be synchronously played, determining whether the white noise data and the screen projection sound data are played faster than the video data; if the white noise data is determined to be faster than the video data, pausing the playing of the white noise data and the screen projection sound data until the playing of the video data, the playing of the white noise data and the playing of the screen projection sound data are synchronous; and if the white noise data is determined to be slower than the video data, accelerating the playing of the white noise data and the screen projection sound data until the playing of the video data, the playing of the white noise data and the playing of the screen projection sound data are synchronous.
And comparing the running time points of the video data, the white noise data and the screen projection sound data, determining whether the playing of the white noise data and the screen projection sound data is faster than that of the video data, if so, waiting for the progress of the video data to catch up with the white noise data and the screen projection sound data, and if not, playing the white noise data and the screen projection sound data at double speed until the playing time points of the video data, the white noise data and the screen projection sound data are synchronous. When playing, the sound and picture are suddenly asynchronous due to transmission problems, and the synchronous time point needs to be fixedly detected so as to solve the problem of sound and picture asynchronization.
The embodiment of the application provides a sound and picture synchronization method, namely, video data are acquired through network transmission, and audio data are acquired through Bluetooth transmission, wherein the audio data comprise white noise data and screen projection sound data; extracting a video time stamp from the video data, analyzing the white noise data into white noise audio content, and extracting an audio time stamp according to the white noise audio content; the time point of the video data, the time point of the white noise data and the time point of the screen projecting sound data are synchronously processed according to the video time stamp and the audio time stamp, the video data, the white noise data and the screen projecting sound data are synchronously played, and the audio time stamp is converted into the white noise data in advance, so the white noise data is converted into the audio time stamp again according to the obtained white noise data, sound and picture synchronization can be carried out according to the audio time stamp and the video time stamp, the problem that sound and picture are not synchronous due to the fact that the sound and picture are in different transmission modes in the playing process is solved, the sound and the picture are synchronously played in the playing process, and the quality of screen projecting playing is improved.
Example two
Further, referring to fig. 3, based on the first embodiment of the present application, in another embodiment of the present application, the same or similar contents to those of the first embodiment of the present application may be referred to the above description, and are not repeated herein. On this basis, in step S20, the step of extracting an audio time stamp from the white noise audio content includes:
step B10, determining the audio characteristic change information of the white noise audio content, and recoding the white noise audio content into characteristic coded data according to the audio characteristic change information;
and B20, extracting the audio time stamp in the feature coding data.
In this embodiment, it should be noted that the white noise audio content is data in an audio format, and it also has audio features related to audio, where the audio features include loudness, tone, and the audio feature change information may be a change in loudness of the audio data, and according to a change in audio features in the white noise audio content, the white noise audio content may be re-encoded into data carrying time stamp information.
Exemplarily, the steps B10 to B20 include:
the method comprises the steps of obtaining audio characteristics of white noise audio content, determining audio characteristic change information of the white noise audio content, recoding the white noise audio content into characteristic coded data according to the audio characteristic change information, obtaining an audio time stamp in the characteristic coded data, and extracting the audio time stamp from the characteristic coded data according to a preset time stamp coding rule.
Before the audio data is acquired, an Zhuoduan needs to generate white noise data, an audio time stamp of the screen projection audio data is determined, an application program codes in the form of audio characteristic change information according to the audio time stamp to generate white noise audio content, the white noise audio content is coded into the white noise data through a Bluetooth module again, and the white noise data and the screen projection audio data are sent to a television terminal together.
In step B10, the step of re-encoding the white noise audio content into feature-encoded data according to the audio feature variation information includes:
step B11, dividing the white noise audio content into a plurality of sections of audio contents according to the audio characteristic change information;
step B12, determining audio characteristic change information of each section of audio content, and coding each audio content into a corresponding characteristic code according to a preset white noise coding rule and the audio characteristic change information;
and step B13, combining each bit of feature code into feature coded data.
In this embodiment, it should be noted that the audio characteristic variation information may be variation of loudness, timbre or tone, such as high loudness and low loudness, the preset white noise coding rule may code white noise audio content into a code containing time stamp information, and the feature code is a type of audio characteristic variation information corresponding to a feature code by the preset white noise code.
Exemplarily, the steps B11 to B13 include:
the method comprises the steps of dividing white noise audio content into a plurality of sections of audio content according to audio characteristic change information of different audios, enabling each section of audio content to correspond to one-bit characteristic code, recoding the audio content with the audio characteristic change information into the characteristic code according to a preset white noise coding rule, and combining the characteristic codes corresponding to the audio contents into complete characteristic coded data.
In a specific embodiment, the audio characteristic variation information may be, for example, loudness variation, and the audio content with high loudness is divided from the audio content with low loudness, where the audio content with high loudness is 1 in the preset white noise coding rule, and the audio content with low loudness is 0 in the preset white noise coding rule, when the loudness of the white noise audio content is high-low-high-low, a string of characteristic encoded data 101110 may be obtained from the audio characteristic variation information, and if the string of characteristic encoded data is converted into a natural number, the string of white noise audio content represents 46.
In step B20, the step of extracting the audio time stamp in the feature encoding data includes:
step B21, reading a piece of time point data from the feature coded data according to preset digits at intervals;
and step B22, converting the read time point data into an audio time stamp.
In this embodiment, it should be noted that the preset time coding rule is a rule for converting feature coding data obtained according to audio feature change information into time points, each preset number of bits in the feature coding data may be converted into a piece of time point data, and white noise audio content includes a plurality of pieces of time point data to be combined into an audio time stamp of the audio data.
Exemplarily, the steps B21 to B22 include:
according to a preset time coding rule, converting the feature coding data into a piece of time point data at intervals of preset bits, wherein the feature coding data can be completely converted into a plurality of pieces of time point data, and all the time point data sets are converted into audio time stamps of the audio data.
Specifically, the time point data includes natural numbers 0 to 9, and if the feature encoding data is represented in a binary manner, each digit of the time point data is represented in binary by at most 4 digits.
In step B21, the step of reading a piece of time point data from the feature coded data every preset number of bits includes:
step B211, taking the feature coded data of each preset digit as a group of time point coded data;
step B212, converting the time point coded data into natural number coded data conforming to a preset format;
in step B213, the respective natural number encoded data are used as time point data.
In this embodiment, it should be noted that the preset number of bits is a data bit number of a piece of time point coding data, a time point exists in a piece of time point coding, the natural number coding data conforming to the preset format is coding data composed of data of 0 to 9, and the preset time point coding rule may be 2-ary coding, 8421 coding, or the like.
Exemplarily, the steps B211 to B213 include:
dividing each preset digit of the characteristic coded data to obtain multiple groups of time point coded data, converting each group of time point coded data into natural number coded data in a preset format according to a preset time coding rule, and directly taking the natural number coded data as time point data.
In this embodiment, the time point data may include five bits of natural number data, where the first two bits are seconds, the last three bits are milliseconds, and each 4-bit binary represents a number, and the time point data may include 20 bits of binary data, and if the feature encoded data is 0000, 0010, 0001, 0010, 0011, the feature encoded data is converted into a natural number of 02123, which indicates that the read time point data is 123 milliseconds for 2 seconds.
Illustratively, at An Zhuoduan, the step of converting the time stamp to white noise audio content needs to be completed. When video data and audio data needing to be projected are obtained through an application program, the video data can be successfully printed with a video timestamp through network transmission, the audio data can be firstly determined to be the audio timestamp before transmission, the application program can convert the audio timestamp into binary data (feature coded data) according to the determined audio timestamp, audio feature change information is determined according to the binary data, white noise audio content is generated according to the audio feature change information, through the method, the audio timestamp can be converted into the white noise audio content, the white noise audio content is sent to a television end in the form of the audio data, the audio timestamp can be successfully sent to the television end on the premise that the Bluetooth cannot print the timestamp, and sound and picture synchronization is carried out through the video timestamp and the audio timestamp by the television end.
The embodiment of the application provides a method for converting white noise audio into an audio time stamp, namely, determining audio characteristic change information of white noise audio content, and recoding the white noise audio content into characteristic coded data according to the audio characteristic change information; and extracting the audio time stamp in the feature encoding data, solving the problem that the time stamp can not be printed in the Bluetooth transmission, and converting the data represented in the audio form into the time stamp.
EXAMPLE III
The embodiment of the present application further provides a sound and picture synchronization apparatus as shown in fig. 4, where the sound and picture synchronization apparatus is applied to a sound and picture synchronization device, and the sound and picture synchronization apparatus includes:
the data acquisition module C10 is used for acquiring video data through network transmission and acquiring audio data through Bluetooth transmission, wherein the audio data comprises white noise data and screen projection sound data;
a timestamp module C20, configured to extract a video timestamp from the video data, parse the white noise data into white noise audio content, and extract an audio timestamp according to the white noise audio content;
and the synchronization module C30 is used for performing synchronous processing on the time point of the video data, the time point of the white noise data and the time point of the screen projecting sound data according to the video time stamp and the audio time stamp, and playing the video data, the white noise data and the screen projecting sound data after synchronization.
Optionally, the timestamp module is further configured to:
determining audio characteristic change information of the white noise audio content, and recoding the white noise audio content into characteristic coded data according to the audio characteristic change information;
and extracting the audio time stamp in the characteristic coding data.
Optionally, the timestamp module is further configured to:
dividing the white noise audio content into a plurality of sections of audio contents according to the audio characteristic change information;
determining audio characteristic change information of each section of audio content, and coding each section of audio content into a corresponding characteristic code according to a preset white noise coding rule and the audio characteristic change information;
each bit of feature code is combined into feature encoded data.
Optionally, the timestamp module is further configured to:
reading a piece of time point data from the feature coded data according to preset digit at intervals;
converting the read time point data into an audio time stamp.
Optionally, the timestamp module is further configured to:
taking the feature coded data of each preset digit as a group of time point coded data;
converting the time point coded data into natural number coded data conforming to a preset format;
the data of each natural number is used as time point data.
Optionally, the synchronization module is further configured to:
determining a first time point corresponding to the video data according to the video timestamp;
determining a second time point corresponding to the white noise data and a third time point corresponding to the screen projection sound data according to the audio time stamp;
playing the video data and continuously monitoring a first time point of the video data;
determining a first matching time point matched with the first time point in the second time point, and playing white noise data corresponding to the first matching time point;
and determining a second matching time point matched with the first time point in the third time point, and playing the screen projection sound data corresponding to the second matching time point.
Optionally, the synchronization module is further configured to:
detecting whether the video data, the white noise data and the screen projection sound data which are being played are synchronously played at each interval preset detection time point according to the video time stamp and the audio time stamp;
if the video data, the white noise data and the screen projection sound data are detected not to be synchronously played, determining whether the white noise data and the screen projection sound data are played faster than the video data;
if the white noise data is determined to be faster than the video data, pausing the playing of the white noise data and the screen projection sound data until the playing of the video data, the playing of the white noise data and the playing of the screen projection sound data are synchronous;
and if the white noise data is determined to be slower than the video data, accelerating the playing of the white noise data and the screen projection sound data until the playing of the video data, the playing of the white noise data and the playing of the screen projection sound data are synchronous.
By adopting the sound and picture synchronization device in the first embodiment or the second embodiment, the technical problem that the sound and picture are not synchronized when a screen is projected based on a private protocol is solved. Compared with the prior art, the beneficial effects of the sound and picture synchronization device provided by the embodiment of the invention are the same as the beneficial effects of the sound and picture synchronization method provided by the embodiment, and other technical features of the sound and picture synchronization device are the same as the features disclosed by the embodiment method, which are not repeated herein.
Example four
An embodiment of the present invention provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the sound and picture synchronization method in the first embodiment or the second embodiment.
Referring now to FIG. 5, shown is a schematic diagram of an electronic device suitable for use in implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., car navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 5, the electronic device may include a processing device D10 (e.g., a central processing unit, a graphic processor, etc.) that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) D20 or a program loaded from a storage device into a Random Access Memory (RAM) D30. In the RAM, various programs and data necessary for the operation of the electronic apparatus are also stored. The processing devices D10, the ROMD20, and the RAMD30 are connected to each other by a bus. An input/output D40 (I/O) interface is also connected to the bus.
Generally, the following systems may be connected to the I/O interface: an input device D50 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, or the like; an output device D60 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; a storage device D70 including, for example, a magnetic tape, a hard disk, or the like; and a communication device D80. The communication device D80 may allow the electronic apparatus to perform wireless or wired communication with other apparatuses to exchange data. While the figures illustrate an electronic device with various systems, it is to be understood that not all illustrated systems are required to be implemented or provided. More or fewer systems may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such embodiments, the computer program may be downloaded and installed from the network through communication device D80, or installed from storage device D70, or installed from the ROMD 20. The computer program, when executed by a processing device, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
The electronic device provided by the invention adopts the sound and picture synchronization method in the first embodiment or the second embodiment, so that the technical problem that the sound and picture are not synchronized when the screen is projected based on a private protocol is solved. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiment of the present invention are the same as the beneficial effects of the sound and picture synchronization method provided by the first embodiment or the second embodiment, and other technical features of the electronic device are the same as those disclosed in the method of the above embodiment, which are not described herein again.
It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the foregoing description of embodiments, the particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or examples.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
EXAMPLE five
The present embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon for executing the method of sound-picture synchronization in the first embodiment.
Embodiments of the present invention provide a computer readable storage medium, such as a USB flash drive, but are not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or device, or any combination thereof. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present embodiment, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer-readable storage medium may be embodied in an electronic device; or may be separate and not incorporated into the electronic device.
The computer readable storage medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring video data through network transmission, and acquiring audio data through Bluetooth transmission, wherein the audio data comprises white noise data and screen projection sound data; extracting a video time stamp from the video data, analyzing the white noise data into white noise audio content, and extracting an audio time stamp according to the white noise audio content; and according to the video time stamp and the audio time stamp, carrying out synchronous processing on the time point of the video data, the time point of the white noise data and the time point of the screen projection sound data, and playing the video data, the white noise data and the screen projection sound data which are synchronized.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present disclosure may be implemented by software or hardware. Wherein the names of the modules do not in some cases constitute a limitation of the unit itself.
The computer-readable storage medium provided by the invention stores computer-readable program instructions for executing the sound and picture synchronization method, and solves the technical problem that the sound and picture are not synchronized when a screen is projected based on a private protocol. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiment of the present invention are the same as the beneficial effects of the sound and picture synchronization method provided by the first embodiment or the second embodiment, and are not described herein again.
Example six
The present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the sound-picture synchronization method as described above.
The computer program product solves the technical problem that the sound and the picture are not synchronous when the screen is projected based on the private protocol. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present invention are the same as the beneficial effects of the sound and picture synchronization method provided by the first embodiment or the second embodiment, and are not described herein again.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings, or which are directly or indirectly applied to other related technical fields, are included in the scope of the present application.

Claims (10)

1. A sound and picture synchronization method is characterized by comprising the following steps:
acquiring video data through network transmission, and acquiring audio data through Bluetooth transmission, wherein the audio data comprises white noise data and screen projection sound data;
extracting a video time stamp from the video data, analyzing the white noise data into white noise audio content, and extracting an audio time stamp according to the white noise audio content;
and carrying out synchronous processing on the time point of the video data, the time point of the white noise data and the time point of the screen projection sound data according to the video time stamp and the audio time stamp, and playing the video data, the white noise data and the screen projection sound data after the video data, the white noise data and the screen projection sound data are synchronized.
2. The sound-picture synchronization method of claim 1, wherein the step of extracting audio time stamps from the white noise audio content comprises:
determining audio characteristic change information of the white noise audio content, and recoding the white noise audio content into characteristic coded data according to the audio characteristic change information;
and extracting the audio time stamp in the characteristic coding data.
3. The sound-picture synchronization method according to claim 2, wherein the re-encoding of the white noise audio content into feature-encoded data based on the audio feature variation information comprises:
dividing the white noise audio content into a plurality of sections of audio contents according to the audio characteristic change information;
determining audio characteristic change information of each section of audio content, and coding each section of audio content into a corresponding characteristic code according to a preset white noise coding rule and the audio characteristic change information;
each bit of feature code is combined into feature encoded data.
4. The sound-picture synchronization method according to claim 2, wherein the step of extracting the audio time stamp in the feature-encoded data comprises:
reading a piece of time point data from the feature coded data according to preset digit at intervals;
converting the read time point data into an audio time stamp.
5. The sound-picture synchronization method of claim 4, wherein the step of reading a piece of time point data from the feature-encoded data according to a preset number of bits per interval comprises:
taking the feature coded data of each preset digit as a group of time point coded data;
converting the time point coded data into natural number coded data conforming to a preset format;
the data of each natural number is used as time point data.
6. The sound-picture synchronization method of claim 1, wherein the step of synchronizing the time point of the video data, the time point of the white noise data, and the time point of the screen-projected sound data according to the video time stamp and the audio time stamp, and playing the synchronized video data, the white noise data, and the screen-projected sound data comprises:
determining a first time point corresponding to the video data according to the video timestamp;
determining a second time point corresponding to the white noise data and a third time point corresponding to the screen projection sound data according to the audio time stamp;
playing the video data and continuously monitoring a first time point of the video data;
determining a first matching time point matched with the first time point in the second time point, and playing white noise data corresponding to the first matching time point;
and determining a second matching time point matched with the first time point in the third time point, and playing the screen projection sound data corresponding to the second matching time point.
7. The sound-picture synchronization method according to claim 6, wherein after the step of playing the on-screen sound data corresponding to the second matching time point, the method further comprises:
detecting whether the video data, the white noise data and the screen projection sound data which are being played are synchronously played at each interval preset detection time point according to the video time stamp and the audio time stamp;
if the video data, the white noise data and the screen projection sound data are detected not to be synchronously played, determining whether the white noise data and the screen projection sound data are played faster than the video data;
if the white noise data is determined to be faster than the video data, pausing the playing of the white noise data and the screen projection sound data until the playing of the video data, the playing of the white noise data and the playing of the screen projection sound data are synchronous;
and if the white noise data is determined to be slower than the video data, accelerating the playing of the white noise data and the screen projection sound data until the playing of the video data, the playing of the white noise data and the playing of the screen projection sound data are synchronous.
8. A sound-picture synchronization apparatus, comprising:
the data acquisition module is used for acquiring video data through network transmission and acquiring audio data through Bluetooth transmission, wherein the audio data comprises white noise data and screen projection sound data;
the time stamp module is used for extracting a video time stamp from the video data, analyzing the white noise data into white noise audio content, and extracting an audio time stamp according to the white noise audio content;
and the synchronization module is used for carrying out synchronous processing on the time point of the video data, the time point of the white noise data and the time point of the screen projection sound data according to the video time stamp and the audio time stamp, and playing the video data, the white noise data and the screen projection sound data after synchronization.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the picture and sound synchronization method of any one of claims 1 to 7.
10. A readable storage medium having stored thereon a program for implementing a sound-picture synchronization method, the program being executed by a processor to implement the steps of the sound-picture synchronization method according to any one of claims 1 to 7.
CN202211002992.2A 2022-08-22 2022-08-22 Sound and picture synchronization method and device, electronic equipment and readable storage medium Active CN115065852B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211002992.2A CN115065852B (en) 2022-08-22 2022-08-22 Sound and picture synchronization method and device, electronic equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211002992.2A CN115065852B (en) 2022-08-22 2022-08-22 Sound and picture synchronization method and device, electronic equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN115065852A CN115065852A (en) 2022-09-16
CN115065852B true CN115065852B (en) 2022-11-01

Family

ID=83208520

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211002992.2A Active CN115065852B (en) 2022-08-22 2022-08-22 Sound and picture synchronization method and device, electronic equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN115065852B (en)

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108683934A (en) * 2018-06-08 2018-10-19 四川乐望崇智科技有限公司 The mobile phone for synchronizing subsidiary audio throws screen method
CN109257641B (en) * 2018-09-05 2021-03-16 福建星网智慧科技股份有限公司 Audio and video synchronization method and system in wireless screen transmission
CN114286165B (en) * 2020-12-21 2023-04-25 海信视像科技股份有限公司 Display equipment, mobile terminal, and screen-throwing data transmission method and system
CN112423076B (en) * 2020-11-18 2023-05-05 湖南嘉加智能科技有限公司 Audio screen-throwing synchronous control method, equipment and computer readable storage medium
CN114302202B (en) * 2021-12-22 2023-03-14 深圳创维-Rgb电子有限公司 Audio and video screen projection method, device, equipment, system and storage medium
CN114666636B (en) * 2022-03-01 2024-06-14 阿里巴巴(中国)有限公司 Audio and video synchronization detection method and computer readable storage medium

Also Published As

Publication number Publication date
CN115065852A (en) 2022-09-16

Similar Documents

Publication Publication Date Title
WO2020151599A1 (en) Method and apparatus for publishing video synchronously, electronic device, and readable storage medium
CN110335615B (en) Audio data processing method and device, electronic equipment and storage medium
WO2023284437A1 (en) Media file processing method and apparatus, device, readable storage medium, and product
US20220394333A1 (en) Video processing method and apparatus, storage medium, and electronic device
CN109582274B (en) Volume adjusting method and device, electronic equipment and computer readable storage medium
KR20220144857A (en) Multimedia data publishing method and apparatus, device and recording medium
CN111325096A (en) Live stream sampling method and device and electronic equipment
US20240177374A1 (en) Video processing method, apparatus and device
CN112149168A (en) File data encryption method and device and electronic equipment
CN111385576B (en) Video coding method and device, mobile terminal and storage medium
CN113144620B (en) Method, device, platform, readable medium and equipment for detecting frame synchronous game
CN113891168B (en) Subtitle processing method, subtitle processing device, electronic equipment and storage medium
CN113391811B (en) Function compiling method, function compiling device, electronic equipment and computer readable storage medium
CN115065852B (en) Sound and picture synchronization method and device, electronic equipment and readable storage medium
CN111478916B (en) Data transmission method, device and storage medium based on video stream
WO2023165390A1 (en) Zoom special effect generating method and apparatus, device, and storage medium
CN113839829A (en) Cloud game delay testing method, device and system and electronic equipment
CN112969038A (en) Data transmission method and device, electronic equipment and computer readable storage medium
CN114760557A (en) Earphone state adjusting method and device, electronic equipment and readable storage medium
CN115134708A (en) Earphone mode switching method and device, electronic equipment and readable storage medium
CN109375892B (en) Method and apparatus for playing audio
CN111444384B (en) Audio key point determining method, device, equipment and storage medium
CN114495941A (en) Method and device for converting single-channel audio into text, electronic equipment and storage medium
CN112330996A (en) Control method, device, medium and electronic equipment for live broadcast teaching
CN111382556B (en) Data conversion method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant