WO2023185590A1 - 媒体信息的获取方法及电子设备 - Google Patents

媒体信息的获取方法及电子设备 Download PDF

Info

Publication number
WO2023185590A1
WO2023185590A1 PCT/CN2023/083116 CN2023083116W WO2023185590A1 WO 2023185590 A1 WO2023185590 A1 WO 2023185590A1 CN 2023083116 W CN2023083116 W CN 2023083116W WO 2023185590 A1 WO2023185590 A1 WO 2023185590A1
Authority
WO
WIPO (PCT)
Prior art keywords
media
media data
time
video
data
Prior art date
Application number
PCT/CN2023/083116
Other languages
English (en)
French (fr)
Inventor
张照鹏
王斌
唐吴全
秦佳美
吴大
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023185590A1 publication Critical patent/WO2023185590A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H04M1/72433—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for voice messaging, e.g. dictaphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H04M1/72439—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for image or video messaging
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4722—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content

Definitions

  • the embodiments of the present application relate to the technical field of terminal equipment, and in particular, to a method for obtaining media information and an electronic device.
  • this application provides a method and electronic device for obtaining media information.
  • the media data output by the electronic device can be forward cached for obtaining media information, which can improve the recognition efficiency and accuracy of media information.
  • embodiments of the present application provide a method for obtaining media information, which is applied to electronic devices.
  • the method includes: before outputting the first media data, cache the first media data to a first storage space; wherein the first storage space is used to cache target media in any media data output by the electronic device data, the target media data is the media data located at the time point of the current playback progress and before the time point in the arbitrary media data; wherein the target media data includes the first media data; in response to The first user operation received at the first time is to obtain the second media data from the first storage space, where the first media data is the media data cached between the second time and the first time, The second media data is media data cached between the third time and the first time, the second time is located before the first time, and the third time is located before the first time; Wherein, the third time is the same as the second time, or the second time is before the third time; based on the second media data, the first media information is output.
  • the first storage space may be a storage space accessible by the media framework module.
  • the media framework module is the output channel of system-level media data of the mobile phone.
  • the media data must be processed by the media framework module before output.
  • the media framework module can obtain the data waiting for the mobile phone. Any media data output.
  • the first storage space can be used for the time point of the current playback progress and the media data before this time point in any media data output by the mobile phone. After the mobile phone outputs the media data cached in the first storage space, the data in the first storage space will not be cleared. Therefore, no matter what type of media data is output by the mobile phone, the first storage space of the mobile phone can store the output media data.
  • the type of media data may include but is not limited to images, audio, and video.
  • the media information may be media data (such as the playback of video clips, the playback of audio clips), or information recognized from the media data (such as the names of actors, drama titles, background music names, etc. that appear in the video).
  • Methods for outputting media information may include but are not limited to: windows, floating windows, dialog boxes, etc.
  • the first user operation may be an operation on the preset icons, virtual keys, or preset gestures of the mobile phone system.
  • the first user operation may also be an operation on the output interface of the mobile phone for outputting media data. This application is for There is no restriction on this.
  • the first moment when the mobile phone receives the first user operation may be a timestamp corresponding to the playback progress of the media data output by the mobile phone. This application does not limit the specific implementation of the timestamp.
  • the timestamp of the first moment may be the playback progress time.
  • the first moment may be 0 minutes and 30 seconds.
  • the timestamp of the first moment may also be system time information corresponding to the playback progress time. For example, when a mobile phone plays a video and the playback progress is 0 minutes and 30 seconds, and the system time of the mobile phone is January 1, 2022, 22:00, the first moment can be January 1, 2022, 22:00.
  • system time information corresponding to the playback progress time. For example, when a mobile phone plays a video and the playback progress is 0 minutes and 30 seconds, and the system time of the mobile phone is January 1, 2022, 22:00, the first moment can be January 1, 2022, 22:00.
  • media data cached between two moments mentioned in various embodiments of this application may include media data cached at the two moments.
  • the first media data may be the data from the 0th minute and the 0th second to the 0th minute and 30th second in the video cached by the mobile phone.
  • the second media data is also the data from 0 minute 0 second to 0 minute 30 second in the video.
  • the second media data is data from 0 minute 5 seconds to 0 minute 30 seconds in the video.
  • the first storage space can cache media data of at least one application.
  • the mobile phone can read the second media data from the first storage space when acquiring the second media data.
  • Media data of the application corresponding to a user operation.
  • the media data can be cached before the media data is output, and the first storage space for caching the media data is the system-level media data cache space of the mobile phone.
  • the mobile phone outputs any media data before output. To cache to the first storage space.
  • the mobile phone can read a section of media data before the playback progress currently output by the mobile phone for outputting media information. Not only can the recognition time of media information be reduced, but the accuracy of the output media information is higher.
  • the method further includes The method includes: caching the third media data into the first storage space before outputting the third media data.
  • the target media data may include third media data.
  • the third media data may be data that continues to be written to the first storage space due to the update of the playback progress of the media (eg video) corresponding to the first media data.
  • the video data from the 30th second to the 31st second in the above example video.
  • the third media data may be media data that is cached in the first storage space when the playback progress is updated once and is the same as the first media data.
  • the third media data and the first media data may be media data output in different display windows of the mobile phone's display screen, or media data output in different display screens of the mobile phone.
  • the applications corresponding to the first media data and the third media data may be the same or different, and this application does not limit this.
  • this embodiment can obtain a section of media data before the current playback progress when receiving the first user input for outputting media information.
  • the first media data includes at least one first media segment
  • the third media data includes at least one second media segment
  • the said Caching the first media data to the first storage space includes: sequentially caching the at least one first media segment to the first storage space according to the output order of the at least one first media segment
  • Caching the third media data to the first storage space includes: sequentially caching the at least one second media segment to the first storage space according to the output order of the at least one second media segment.
  • the media segments can be written to the first storage space sequentially according to the output order of the media segments.
  • the media segments may include audio segments, and/or image sequences.
  • the at least one second media segment is cached in the first storage space in sequence according to the output order of the at least one second media segment. , including: detecting that there is no free space in the first storage space, sequentially deleting the cached first media segments in the first storage space in order from early to late cache time, and following the at least one The output sequence of the second media segments is to cache the at least one second media segment to the first storage space in sequence.
  • the mobile phone when the mobile phone writes media segments to be output to the first storage space and detects that there is no free space in the first storage space, the mobile phone can cache the earliest media fragments in the first storage space in a first-in, first-out manner. Delete the media segment, and then write the second media segment to the first storage space to achieve dynamic refresh of the data cached in the first storage space, so that the first storage space maintains the cache of the media data currently output by the electronic device. The time point of the progress and the media data before that time point.
  • the first storage space includes: a first space for caching audio data and a second space for caching image data; the target media Caching data into the first storage space includes: caching the audio data in the target media data into the first space; and/or caching the image data in the target media data into the second space; wherein, The target media data includes the first media data, and/or the third media data.
  • the space used to cache audio data and the space used to cache image data can be different to achieve parallel writing of audio data and image data to the first storage space and improve data caching. efficiency.
  • the target media data includes at least one target media segment; caching the at least one target media segment to the first storage space in sequence includes: sequentially caching the at least one target media segment into the first storage space. Cache at least one audio segment in the at least one target media segment to the first space; and/or cache at least one frame of image in the at least one target media segment to the second space in sequence; wherein , the target media segment includes the first media segment, and/or the second media segment.
  • audio clips and frame images can be written to corresponding storage spaces in the first storage space in parallel, and when writing the audio clips, they are written to the first space in sequence according to the output order of the audio clips. When inputting image data, it is written to the second space sequentially according to the output order of the frame images to achieve parallel writing of image and audio data.
  • the first storage space includes: a third space and a fourth space, wherein the third space is used to cache the electronic device in the first The fourth media data output by the window, the fourth space is used to cache the fifth media data output by the electronic device in the second window; the method also includes: before outputting the fourth media data, the Caching the fourth media data to the third space; displaying the fourth media data in the first window; and caching the fifth media data to the fourth space before outputting the fifth media data. space; display the fifth media data in the second window.
  • the target media data may include fourth media data and fifth media data.
  • the fourth media data and the fifth media data are different.
  • the relationship between the fourth media data, the fifth media data, and the above-mentioned first media data, second media data, and third media data is not limited.
  • the fourth media data can be related to the first media data, or Either the second media data or the third media data is the same or different.
  • the fifth media data may be the same as or different from any one of the first media data, the second media data, or the third media data.
  • the first space may include space a1 corresponding to window 1 and/or space b1 corresponding to window 2;
  • the second space may include space a2 corresponding to window 1 and/or space b2 corresponding to window 2;
  • the third space includes space a1 and/or the space a2;
  • the fourth space includes space b1 and/or space b2.
  • the number of windows corresponding to the cacheable data in the first storage space is not limited to the above two, and may be two or more.
  • the scenario may include but is not limited to: a split-screen scenario, a floating window scenario, a picture-in-picture scenario, etc.
  • the media data output in different windows of the electronic device can be cached to the storage space corresponding to the corresponding window before the media data is output, so as to realize parallel writing of the media data to be output in different windows. Enter the first storage space.
  • the method of this embodiment can also be applied to a scenario where multiple display screens output media data.
  • the scenario of multiple display screens may include but is not limited to a double-sided screen scenario, a folding screen scenario, etc. Then save first
  • the storage space may include a storage space corresponding to each display screen.
  • the method further includes: in response to receiving the , perform a second user operation on the first window to obtain sixth media data from the third space; wherein the fourth media data is media data cached between the fifth moment and the fourth moment. , the sixth media data is media data cached between the sixth time and the fourth time, the fifth time is before the fourth time, and the sixth time is before the fourth time. ; Wherein, the sixth time is the same as the fifth time, or the fifth time is before the sixth time; based on the sixth media data, the second media information is output.
  • the method further includes: in response to the fifth media data being displayed in the second window, in response to the .
  • the third user operation on the second window obtains seventh media data from the fourth space; wherein the fifth media data is the media data cached between the eighth time and the seventh time.
  • the seventh media data is media data cached between the ninth time and the seventh time
  • the eighth time is located before the seventh time
  • the ninth time is located before the seventh time;
  • the eighth time is the same as the ninth time, or the eighth time is before the ninth time; based on the seventh media data, third media information is output.
  • the electronic device can output media data in any window, and when the user triggers input, obtain the time point of the playback progress of the media data and the media data before that time point for media information output.
  • media information can be output to media data output from any window.
  • the method further includes: in response to the received fourth user operation, setting or adjusting the size of the target space; wherein the target space includes at least the following: One: the first space, the second space, the third space, and the fourth space.
  • the total size of the first storage space is configured by the electronic device when it leaves the factory.
  • the user can set or adjust the size of the first storage space, where the set or adjusted size of the first storage space needs to be less than or equal to the total size.
  • the user can set or adjust the space size of any one of the first space, the second space, the third space, and the fourth space in the first storage space to realize the user's access to the cached media information.
  • Customized setting of length so that the cached data length can be flexibly adjusted according to actual needs.
  • the media type of the arbitrary media data includes at least one of the following: image data, audio data, and video data.
  • This implementation mode can support forward caching of at least one kind of data such as image, audio, video, etc., to efficiently output media data.
  • the method further includes: in response to The received fifth user operation clears the first storage space; wherein the fifth user operation is used to represent a power-on operation, a power-off operation, or a restart operation of the electronic device.
  • the data in the first storage space can be cleared when the electronic device is turned on, shut down or restarted, so as to cache media data based on the latest media data played by the user.
  • inventions of the present application provide an electronic device.
  • the electronic device includes: a memory and a processor, the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the first aspect and a method in any implementation of the first aspect.
  • embodiments of the present application provide a computer-readable medium for storing a computer program.
  • the electronic device When the computer program is run on an electronic device, the electronic device causes the electronic device to execute the first aspect and the first aspect. method in any embodiment.
  • embodiments of the present application provide a computer program product containing instructions.
  • the computer program product When the computer program product is run on a computer, it causes the computer to execute the first aspect and any one of the embodiments of the first aspect. method in.
  • embodiments of the present application provide a computer-readable storage medium, including a computer program.
  • the computer program When the computer program is run on an electronic device, the electronic device causes the electronic device to execute the first aspect and any of the first aspects.
  • Method in one embodiment.
  • embodiments of the present application provide a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send them to the processor.
  • the processor sends the signal, and the signal includes computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device is caused to execute the first aspect and any one of the implementation manners of the first aspect.
  • Figure 1 is one of the structural schematic diagrams of an exemplary electronic device
  • Figure 2 is a schematic diagram of the software structure of an exemplary electronic device
  • Figure 3 is a schematic diagram of an application scenario for obtaining media information of an electronic device
  • Figure 4 is a schematic diagram of an application scenario for obtaining media information by an electronic device
  • Figure 5a is an exemplary schematic diagram of module interaction within an electronic device
  • Figure 5b is a schematic diagram of an exemplary internal module of the electronic device
  • Figure 5c is a schematic diagram of an exemplary internal module of the electronic device
  • Figure 6a is a schematic diagram illustrating an application scenario of an exemplary electronic device
  • Figure 6b is a schematic diagram of an exemplary playback progress bar of an application scenario of an electronic device
  • Figure 7 is a schematic diagram illustrating the changes of the playback progress bar in an application scenario of the electronic device
  • Figure 8 is a schematic diagram illustrating an exemplary process of caching data of an electronic device
  • Figure 9a is a schematic diagram illustrating an exemplary process of caching data of an electronic device
  • Figure 9b is a schematic diagram illustrating an exemplary process of caching data of an electronic device
  • Figure 10 is a schematic diagram showing an exemplary scenario in which an electronic device obtains media information
  • Figure 11 is a schematic diagram illustrating an application scenario of an exemplary electronic device
  • Figure 12 is a schematic diagram illustrating an application scenario of an exemplary electronic device
  • Figure 13 is a schematic diagram illustrating an application scenario of an exemplary electronic device
  • Figure 14 is a schematic structural diagram of a device provided by an embodiment of the present application.
  • a and/or B can mean: A exists alone, A and B exist simultaneously, and they exist alone. B these three situations.
  • first and second in the description and claims of the embodiments of this application are used to distinguish different objects, rather than to describe a specific order of objects.
  • first target object, the second target object, etc. are used to distinguish different target objects, rather than to describe a specific order of the target objects.
  • multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.
  • FIG. 1 shows a schematic structural diagram of an electronic device 100 .
  • the electronic device 100 shown in FIG. 1 is only an example of an electronic device.
  • the electronic device 100 may be a terminal, which may also be called a terminal device.
  • the terminal may be a cellular phone or a tablet computer.
  • Devices with audio and video playback functions such as (pads), wearable devices or Internet of Things devices are not limited by this application.
  • the electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations.
  • the various components shown in Figure 1 may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.
  • the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2.
  • Mobile communication module 150 wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, And subscriber identification module (subscriber identification module, SIM) card interface 195, etc.
  • SIM subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) wait.
  • application processor application processor, AP
  • modem processor graphics processing unit
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller memory
  • video codec digital signal processor
  • DSP digital signal processor
  • baseband processor baseband processor
  • NPU neural-network processing unit
  • different processing units can be independent devices or integrated in one or more processors.
  • the controller may be the nerve center and command center of the electronic device 100 .
  • the controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.
  • the processor 110 may also be provided with a memory for storing instructions and data.
  • the memory in processor 110 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 110 . If the processor 110 needs to use the instructions or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
  • processor 110 may include one or more interfaces.
  • Interfaces may include integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, pulse code modulation (pulse code modulation, PCM) interface, universal asynchronous receiver and transmitter (universal asynchronous receiver/transmitter (UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and /or universal serial bus (USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • UART universal asynchronous receiver and transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB universal serial bus
  • the I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (derail clock line, SCL).
  • processor 110 may include multiple sets of I2C bus.
  • the processor 110 can separately couple the touch sensor 180K, charger, flash, camera 193, etc. through different I2C bus interfaces.
  • the processor 110 can be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to implement the touch function of the electronic device 100 .
  • the I2S interface can be used for audio communication.
  • processor 110 may include multiple sets of I2S buses.
  • the processor 110 can be coupled with the audio module 170 through the I2S bus to implement communication between the processor 110 and the audio module 170 .
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface to implement the function of answering calls through a Bluetooth headset.
  • the PCM interface can also be used for audio communications to sample, quantize and encode analog signals.
  • the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface.
  • the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface to implement the function of answering calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • a UART interface is generally used to connect the processor 110 and the wireless communication module 160 .
  • the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function.
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.
  • the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .
  • MIPI interfaces include camera serial interface (CSI), display serial interface (DSI), etc.
  • the processor 110 and the camera 193 communicate through the CSI interface to implement the shooting function of the electronic device 100 .
  • the processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100 .
  • the GPIO interface can be configured through software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 110 with the camera 193, display screen 194, wireless communication module 160, audio module 170, sensor module 180, etc.
  • the GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the USB interface 130 is an interface that complies with the USB standard specification, and may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc.
  • the USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through them. This interface can also be used to connect other electronic devices, such as AR devices, etc.
  • the interface connection relationships between the modules illustrated in the embodiments of the present application are only schematic illustrations and do not constitute a structural limitation of the electronic device 100 .
  • the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
  • the charging management module 140 is used to receive charging input from the charger.
  • the charger can be a wireless charger or a wired charger.
  • the charging management module 140 may receive charging input from the wired charger through the USB interface 130 .
  • the charging management module 140 may receive wireless charging input through the wireless charging coil of the electronic device 100 . While the charging management module 140 charges the battery 142, it can also provide power to the electronic device through the power management module 141.
  • the power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.
  • the power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, wireless communication module 160, etc.
  • the power management module 141 can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters.
  • the power management module 141 may also be provided in the processor 110 .
  • the power management module 141 and the charging management module 140 may also be provided in the same device.
  • the wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example: Antenna 1 can be reused as a diversity antenna for a wireless LAN. In other embodiments, antennas may be used in conjunction with tuning switches.
  • the mobile communication module 150 can provide solutions for wireless communication including 2G/3G/4G/5G applied on the electronic device 100 .
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc.
  • the mobile communication module 150 can receive electromagnetic waves through the antenna 1, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna 1 for radiation.
  • at least part of the functional modules of the mobile communication module 150 may be disposed in the processor 110 .
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
  • a modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal.
  • the demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194.
  • the modem processor may be a stand-alone device.
  • the modem processor may be independent of the processor 110 and may be provided in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (bluetooth, BT), and global navigation satellites.
  • WLAN wireless local area networks
  • System global navigation satellite system, GNSS
  • frequency modulation frequency modulation, FM
  • near field communication technology near field communication, NFC
  • infrared technology infrared, IR
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
  • the wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
  • the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • the electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is an image processing microprocessor and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
  • the display screen 194 is used to display images, videos, etc.
  • Display 194 includes a display panel.
  • the display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • AMOLED organic light-emitting diode
  • FLED flexible light-emitting diode
  • Miniled MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc.
  • the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
  • the electronic device 100 can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
  • Camera 193 is used to capture still images or video.
  • the object passes through the lens to produce an optical image that is projected onto the photosensitive element.
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. Such as saving music, videos, etc. files in external memory card.
  • Internal memory 121 may be used to store computer executable program code, which includes instructions.
  • the processor 110 executes instructions stored in the internal memory 121 to execute various functional applications and data processing of the electronic device 100 .
  • the internal memory 121 may include a program storage area and a data storage area. Among them, the stored program area can store an operating system, at least one application program required for a function (such as a sound playback function, an image playback function, etc.).
  • the storage data area may store data created during use of the electronic device 100 (such as audio data, phone book, etc.).
  • the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.
  • the electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. Such as music playback, recording, etc.
  • the audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .
  • Speaker 170A also called “speaker” is used to convert audio electrical signals into sound signals.
  • the electronic device 100 can listen to music through the speaker 170A, or listen to hands-free calls.
  • Receiver 170B also called “earpiece” is used to convert audio electrical signals into sound signals.
  • the electronic device 100 answers a call or a voice message, the voice can be heard by bringing the receiver 170B close to the human ear.
  • Microphone 170C also called “microphone” or “microphone” is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak close to the microphone 170C with the human mouth and input the sound signal to the microphone 170C.
  • the electronic device 100 may be provided with at least one microphone 170C. In other embodiments, electronic device 100 Two microphones 170C can be set up, which in addition to collecting sound signals, can also achieve noise reduction functions. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions, etc.
  • the headphone interface 170D is used to connect wired headphones.
  • the headphone interface 170D may be a USB interface 130, or may be a 3.5mm open mobile terminal platform (OMTP) standard interface, or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.
  • OMTP open mobile terminal platform
  • CTIA Cellular Telecommunications Industry Association of the USA
  • the buttons 190 include a power button, a volume button, etc.
  • Key 190 may be a mechanical key. It can also be a touch button.
  • the electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .
  • the motor 191 can generate vibration prompts.
  • the motor 191 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback.
  • touch operations for different applications can correspond to different vibration feedback effects.
  • the motor 191 can also respond to different vibration feedback effects for touch operations in different areas of the display screen 194 .
  • Different application scenarios such as time reminders, receiving information, alarm clocks, games, etc.
  • the touch vibration feedback effect can also be customized.
  • the indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, or may be used to indicate messages, missed calls, notifications, etc.
  • the SIM card interface 195 is used to connect a SIM card.
  • the SIM card can be connected to or separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 .
  • the electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.
  • SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card, etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different.
  • the SIM card interface 195 is also compatible with different types of SIM cards.
  • the SIM card interface 195 is also compatible with external memory cards.
  • the electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications.
  • the electronic device 100 uses an eSIM, that is, an embedded SIM card.
  • the eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .
  • the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.
  • the embodiment of this application takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100 .
  • FIG. 2 is a software structure block diagram of the electronic device 100 according to the embodiment of the present application.
  • the layered architecture of the electronic device 100 divides the software into several layers, and each layer has clear roles and division of labor.
  • the layers communicate through software interfaces.
  • the Android system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime and system libraries, and kernel layer.
  • the application layer can include a series of application packages.
  • the application package can include camera, gallery, calendar, calling, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.
  • the application framework layer provides an application programming interface (API) and programming framework for applications in the application layer.
  • API application programming interface
  • the application framework layer includes some predefined functions.
  • the application framework layer can include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.
  • a window manager is used to manage window programs.
  • the window manager can obtain the display size, determine whether there is a status bar, lock the screen, capture the screen, etc.
  • Content providers are used to store and retrieve data and make this data accessible to applications.
  • Said data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.
  • the view system includes visual controls, such as controls that display text, controls that display pictures, etc.
  • a view system can be used to build applications.
  • the display interface can be composed of one or more views.
  • a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.
  • the phone manager is used to provide communication functions of the electronic device 100 .
  • call status management including connected, hung up, etc.
  • the resource manager provides various resources to applications, such as localized strings, icons, pictures, layout files, video files, etc.
  • the notification manager allows applications to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify download completion, message reminders, etc.
  • the notification manager can also be notifications that appear in the status bar at the top of the system in the form of charts or scroll bar text, such as notifications for applications running in the background, or notifications that appear on the screen in the form of conversation windows. For example, text information is prompted in the status bar, a beep sounds, the electronic device vibrates, the indicator light flashes, etc.
  • the system library and runtime layer include system libraries and Android Runtime.
  • System libraries can include multiple functional modules. For example: surface manager (surface manager), media libraries (Media Libraries), 3D graphics processing libraries (for example: OpenGL ES), 2D graphics engines (for example: SGL), etc.
  • the 3D graphics library is used to implement three-dimensional graphics drawing, image rendering, composition and layer processing, etc.
  • the Android runtime includes core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.
  • the core library contains two parts: one is the functional functions that need to be called by the Java language, and the other is the core library of Android.
  • the application layer and application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and application framework layer into binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection and other functions.
  • the surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, composition, and layer processing.
  • 2D Graphics Engine is a drawing engine for 2D drawing.
  • the Media Framework module can be the output channel for system-level media data (including but not limited to images, audio, video, etc.). That is, when the media data in the system is output, it must be output through the media framework module.
  • system-level media data including but not limited to images, audio, video, etc.
  • the media framework module can be used to obtain the media data to be output and output the media data before the media data is output.
  • the media data may include but is not limited to: still images, audio, video, etc.
  • the media framework module can be used to encode the media data for the last time before outputting the media data.
  • code including but not limited to audio encoding, image rendering, etc.
  • outputable media data and then output the encoded media data.
  • the media framework module can also be used to control the playback control operations of media data such as starting playback, pausing playback, fast forward playback, and rewind playback.
  • the media framework module supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc.
  • the media framework module can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the media data before the media data is transmitted to the media framework module, it can be encoded and processed by the above-mentioned three-dimensional image processing library and/or the above-mentioned two-dimensional graphics engine and other modules, and this application does not limit this.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.
  • the media framework module can output media data to the display driver of the kernel layer to display the media data on the display screen, and/or output the media data to the audio driver of the kernel layer, and the audio driver outputs the media data to the screen shown in Figure 1
  • the audio module 170 is shown to play media data.
  • the components included in the system framework layer, system library and runtime layer shown in Figure 2 do not constitute specific limitations on the electronic device 100.
  • the electronic device 100 may include more or fewer components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently.
  • FIG. 3 illustrates an application scenario diagram of an electronic device for obtaining media information.
  • the display interface 101 of the mobile phone is a display interface of a news application.
  • the display interface 101 includes one or more controls.
  • the controls include but are not limited to: " "Follow” control, "Hot Spot” control, and "Current Affairs” control. After the above three controls are clicked by the user, the mobile phone can display the corresponding news content.
  • the display interface 101 also includes a video playback window 102.
  • the video playback window 102 includes a video playback progress bar 113.
  • the playback progress bar 113 includes a playback pause control 1021, a playback progress control 1024, a current playback time control 1022, and a video duration control 1023.
  • the playback pause control 1021 is used to pause or play the video played in the video playback window 102 after being triggered.
  • the playback duration control 1023 is used to represent the total duration of the video played within the video playback window 102 (here, it is 2 minutes and 30 seconds).
  • the current playback time control 1022 is used to represent the current playback progress of the video played in the video playback window 102, and the corresponding playback time point (here is 2 minutes) in the above total duration.
  • the main interface 103 may include application icons of multiple applications.
  • the display interface 105 includes one or more controls, including but not limited to a music playback control 106 and a background song recognition control 107.
  • the display interface 108 includes one or more controls, including a music recognition control 110 and a stop recognition control 109.
  • the music recognition control 110 is used to indicate that the mobile phone is collecting the audio data being played by the mobile phone and identifying the song name from the audio data.
  • the stop recognition control 109 is clicked by the user, the mobile phone can stop collecting the audio data currently being played by the mobile phone and stop identifying the song name.
  • the audio data currently being played by the mobile phone is the background music of the video being played by the news application corresponding to Figure 3(1).
  • the music application corresponding to Figure 3(3) begins to collect the audio data being played by the mobile phone, and the current play time control 1022 of Figure 3(1).
  • the video playback progress is already at a point after 2 minutes of the video, and as shown in the playback duration control 1023 of Figure 3(1), the total duration of the video is only 2 minutes. minutes 30 seconds.
  • the audio data that can be collected by the music application is only the audio clips of less than 30s (seconds) near the end of the video, which makes the collected audio data shorter and the recognition fails.
  • the display interface of the mobile phone switches from the display interface 108 to the display interface 111.
  • the display interface 111 includes one or more controls, including a recognition result control 112 .
  • the recognition result control 112 displays the text content as "no result matched", that is, the song name recognition failed.
  • Figure 3 is only an interactive example for identifying songs, and the interface for identifying songs by listening to songs can also be brought up through other interactive methods.
  • FIG. 4 illustrates an application scenario diagram of an electronic device for obtaining media information.
  • the display interface 201 of the mobile phone is a video playback interface.
  • the display interface 201 includes one or more controls, including a playback screen 202, a playback progress bar 203, and a video information acquisition control 205.
  • the functions of each control in the playback progress bar 203 are similar to the controls in the playback progress bar 113 in Figure 3(1).
  • the total duration of the video is 50 minutes
  • the current playback progress of the video is The duration is 18 minutes.
  • the mobile phone recognizes characters, scenes and other information on the currently playing video screen at the moment when the user clicks on the video information acquisition control 205, and obtains media information such as actor names, movie titles, background music, etc., and outputs it to user.
  • the mobile phone in response to the user's operation of clicking the video information acquisition control 205, the mobile phone obtains the corresponding video picture at the moment when the user clicks the video information acquisition control 205, identifies the character in the video picture, and obtains the actor and role information of the character, As shown in Figure 4(2), the mobile phone can display a floating window control 206.
  • the floating window control 206 displays the identification result of the media information of this frame: "Actor 1 plays role 1.”
  • the mobile phone uses a video application to identify the media information of the video data being played in the video application, it can only trigger the "media information acquisition" function for the user (for example, in Figure 4(1), the user clicks the video information acquisition control 205) Acquire media information for the current frame at this moment.
  • the media information cannot be identified for the video information that has been played when the user triggers the "media information acquisition” function (for example, any video clip in the first 18 minutes of video information that has been played in Figure 4).
  • the mobile phone is playing a video about a football match.
  • the video is played until the player scores a goal.
  • the video lasts for a short time.
  • the goal scene flashes by.
  • the user triggers the "media”
  • the goal screen has been played.
  • traditional electronic devices can only obtain the video picture at the moment when the user triggers the "media information acquisition” function to identify the media information. Therefore, it is impossible to obtain the video picture that has been played and identify the related media information.
  • the electronic device of the present application is provided with a cache space in the memory accessible to the media framework module.
  • the media framework module serves as an output channel for system-level media data (including but not limited to images, audio, video, etc.).
  • system-level media data including but not limited to images, audio, video, etc.
  • the media data must be output through the media framework module. That is, as long as the electronic device is playing audio and video data, the audio and video data first need to be encoded by the media framework module before playing, and then they can be output and played. Then the media framework module can write the audio and video data to be output to the cache space before the audio and video data is output, so that the cache space can store the current playback progress (including the current playback progress) of the audio and video data currently played by the electronic device. point in time) audio and video content from a previous period of time.
  • the media framework module can take out the cached content in the above cache space for media information identification.
  • the cache content here includes a piece of audio and video content before the current playback progress of the audio and video data currently played by the electronic device.
  • the mobile phone can quickly obtain a piece of audio and video data before the user triggers the "Media Information Acquisition” function to identify the media information, enabling instant acquisition of media information. There is no need to start collecting audio and video content after the moment when the user triggers the "media information acquisition” function, which shortens the recognition time of media information.
  • Figure 5a illustrates an interactive schematic diagram of each module in the electronic device of the present application. This can be combined with Figures 5b, 5c, 6a to Figure 5a. 9a and Figure 9b to understand.
  • the media framework module receives the audio and video stream of the video application 1 and processes the audio and video stream accordingly to obtain at least one audio and video segment to be output.
  • the display interface 300 of the mobile phone is the display interface of the video application 1 of the application program layer in FIG. 5a.
  • the display interface 300 may include one or more controls.
  • the controls may include but are not limited to: "Selected" control 3031, "Small Video” control 3032, "Episodes” control 3033, "Movie” control 3034, first video window control 3035, second video window Control 3036, third video window control 3037.
  • the mobile phone can respond to the user operation and display the relevant video window of the selected video in the display interface 300.
  • the mobile phone can respond to the user operation and display the relevant video window of the short video in the display interface 300.
  • the mobile phone can respond to the user operation and display the relevant video window of the popular TV series in the display interface 300.
  • the mobile phone can display the relevant video window of the movie in the display interface 300 in response to the user operation.
  • the user clicks the third video window control 3037, and the mobile phone may display the display interface 401 shown in Figure 6a(2) in response to the user operation.
  • the video content played by the display interface 401 may be the movie content of "Movie Name 1" corresponding to the third video window control 3037.
  • the response operation of the mobile phone is similar to the response operation to the user clicking on the third video window control 3037. , we won’t go into details here.
  • the response operation of the mobile phone is also similar to the response operation of the user clicking on the third video window control 3037, which will not be described again here.
  • the mobile phone can trigger the above S101 in response to this operation, so that the video application 1 can display the audio and video stream to be displayed. Sent to the media framework module in the system library.
  • an application framework layer may also be included between the application layer and the system library, and then the audio and video streams may be sent to the media framework module through the application framework layer.
  • the system library can include at least one module for encoding media data. Then the media framework module can be the last module to encode media data. The audio and video streams received by the media framework module can be pre-passed through other modules in the system library. Module encoding processing is not limited by this application.
  • Fig. 5a Fig. 5a
  • Fig. 5b Fig. 5c
  • Fig. 5c exemplarily show a schematic diagram of the media framework module.
  • the media framework module can encode the audio and video streams to obtain at least one audio and video segment to be output.
  • the audio and video stream may be the audio and video stream currently to be output and played by the mobile phone.
  • the media framework module can process the received audio and video stream and obtain at least one audio and video segment.
  • the audio and video streams may correspond to multiple audio and video segments.
  • the media framework module can perform audio encoding processing on the audio data in the audio and video streams to obtain at least one audio segment.
  • the media framework module encodes audio data
  • the smallest unit of encoding unit may be an audio segment.
  • the media framework module can perform image encoding processing (including but not limited to rendering processing, etc.) on the image data in the audio and video stream to obtain at least one frame.
  • image encoding processing including but not limited to rendering processing, etc.
  • the media framework module encodes image data
  • the smallest unit of encoding unit may be a frame of image.
  • frame images and audio clips with the same timestamp after encoding may constitute an audio and video clip after encoding as shown in FIG. 5b.
  • the media framework module writes at least one audio and video clip to be output into the storage space A in the memory.
  • storage space A can be the storage space for media data allocated by the system to the media framework module in the memory when the mobile phone leaves the factory.
  • This storage space A can be used to cache media data to be output (such as audio and video data), that is, Audio and video data before the playback time point can be cached.
  • the storage space size of storage space A can be configured by the system at the factory.
  • the user can adjust the size of the storage space A according to the needs.
  • an entrance to adjust the size of the storage space A can be provided in the system settings interface, so that the user can flexibly adjust the size of the storage space A according to the needs.
  • the size of media data that can be cached in storage space A is controlled by the user to meet the user's personalized media data caching requirements.
  • the media framework module executes S103 and S104 to output at least one encoded audio and video clip.
  • This application does not limit the execution order of S103 and S104. That is to say, before at least one audio and video clip is output, the media framework module can cache at least one audio and video clip to be output, so that the mobile phone caches the playback progress point of the currently playing video (including The playback progress point) audio and video clips from the previous period.
  • the media framework module sends the image data in at least one audio and video clip to the display driver.
  • the media framework module sends the audio data in at least one audio and video clip to the audio driver.
  • At least one audio and video segment obtained after encoding can be written into storage space A and storage space B respectively, Then, output at least one audio and video clip in storage space B to the audio driver and display driver.
  • the media framework module can be used to perform the last encoding operation before the media data is output.
  • the media framework module can cache the encoded audio and video clips.
  • the cache space here It is storage space B (which can be located in the memory).
  • the media framework module can read all cached data from storage space B ( can include multiple encoded audio and video clips), and output the multiple encoded audio and video clips at once (i.e., the target audio and video clips shown in Figure 5b).
  • the storage space B Clear so that each audio and video clip after the next encoding can be written in sequence in storage space B, so as to cache at least one audio and video clip to be output in storage space B.
  • the space size of storage space B can be smaller than the space size of storage space A.
  • the amount of data cached by the media framework module each time is the amount of audio and video data output each time it is refreshed. If the amount of cached data is large, the cache time will be longer, which will affect the audio and video data. Frequent refresh rates can easily cause video freezes or audio freezes.
  • the audio and video data cached in storage space A is data used for media information identification. Therefore, in order to improve the accuracy of media information identification, more audio and video data can be cached than the audio and video data used for refreshing output, so that The space size of storage space A is larger than the space size of storage space B.
  • the storage space A can also be part of the storage space B allocated by the system to the media framework module in the traditional solution, so that the media framework module can reuse the traditional storage space B to cache not only the data to be output in the storage space B
  • the audio and video data can also be cached to identify the media information and the audio and video data before the current playback time point. However, each time the media framework module outputs audio and video data, it can clear the audio and video data to be output that is cached in storage space B except storage space A.
  • storage space B may include storage space A, and in storage space B, the space size used to cache audio and video data to be output is smaller than the space size used to cache audio and video data identifying media information.
  • storage space A and storage space B can be two separate storage spaces allocated by the system to the media frame module when the mobile phone is shipped from the factory, and the functions of the two storage spaces are different.
  • Storage space B is used to cache the audio and video data output each time
  • storage space A is used to cache a piece of audio and video data before the current playback time point (including the current playback time point) for media information identification.
  • the space size of storage space A may be larger than the space size of storage space B.
  • the size of storage space A is not limited by the size of storage space B.
  • each embodiment of the present application takes storage space A and storage space B as two different storage spaces as an example. In the scenario where storage space B includes storage space A, the method is the same and will not be repeated here. Repeat.
  • storage space A may include storage space A1 for storing audio, and storage space A2 for storing images.
  • Storage space B may include storage space B1 for storing audio, and storage space B2 for storing images.
  • the media framework module completes encoding of each audio segment (where the minimum encoding unit of audio data is an audio segment) for the received audio and video stream, and then the encoded audio segment is Write to storage space A1 and storage space B1 respectively; similarly, the media framework module will write the encoded frame of image to storage space A2 and storage space respectively after each pair of one frame of image encoding is completed for the received audio and video stream. B2. So that at least one audio clip is written into the storage space A1 and the storage space B1 in sequence, and at least one frame of image is written into the storage space A2 and the storage space B2 in sequence.
  • the media framework module when the media framework module detects that the storage space B1 is full of data, it reads out all the audio clips (here, the audio clip sequence) in the storage space B1.
  • the media framework module when the media framework module detects that the storage space B2 is full of data, it reads out all the frames (here, the frame sequence) in the storage space B2.
  • the audio clip sequence in the storage space B1 and the frame image sequence in the storage space B2 can constitute a target audio and video clip to be output.
  • the target audio and video segments may include at least one encoded audio and video segment.
  • storage space B1 and storage space B2 can be filled with data at the same time, and this application does not limit this.
  • the media framework module can send the audio fragment sequence read from the storage space B1 to the audio driver, and send the frame image sequence read from the storage space B2 to the display driver to achieve a goal. Output of audio and video clips.
  • the audio segment sequence in the output target audio and video segment and the frame sequence are two sets of sequence data with one-to-one time stamp correspondence, so as to achieve accurate output of a video stream.
  • the storage space B1 and the storage space B2 can be cleared, so that after updating the playback progress once, the storage space B1 and the storage space B2 can be used to continue caching for the next time.
  • the target audio and video clip to be output.
  • the display driver outputs the image data to the display screen.
  • the audio driver outputs audio data to the audio device.
  • S105 and S106 can be executed at the same time or one after another. This application does not limit the order in which S105 and S106 are executed.
  • the audio device may include the audio module 170 shown in FIG. 1 and include one of a speaker 170A or a headphone interface 170D.
  • the audio module 170 can output the processed audio data to the speaker 170A or the headphone interface 170D to achieve audio output in the video.
  • the audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .
  • the display interface of the mobile phone can be switched from the display interface 300 shown in Figure 6a(1) to the display interface 401 shown in Figure 6a(2) to realize the output and playback of video data.
  • the display interface 401 may include one or more controls, including but not limited to: the currently displayed video screen 407, the playback progress bar control 402, the playback progress control 404, the current playback Time control 403, video duration control 406, playback and pause control 405.
  • the mobile phone can respond to the user operation by pausing or starting the playback of the video played in the display interface 401.
  • the playback duration control 406 is used to represent the total duration of the video played in the display interface 401 (here, 50 minutes).
  • the current playback time control 403 is used to represent the current playback progress of the video played in the display interface 401, and the corresponding playback time point in the above total duration (referred to as the current playback time point, here is 0 minutes and 0 seconds).
  • the mobile phone can adjust the playback progress of the video played in the display interface 401 in response to the user's operation.
  • the mobile phone can trigger the execution of S101 in Figure 5a.
  • video application 1 continuously requests to refresh the video stream, as shown in Figure 5a, Figure 5b, and Figure 5c
  • the media framework module can output the target audio and video clips multiple times. Then each time the media framework module outputs the target audio and video clips, it can The playback progress of the video played in the display interface 401 shown in Figure 6a(2) (that is, the video screen 407) is refreshed once, which is equivalent to updating the playback progress once. Then, by executing the above S101 to S106 multiple times, the multiple playback progress of the video played in the display interface 401 can be refreshed.
  • the display interface of the machine can be switched from Figure 6a(2) to Figure 6a(3) after the video content is refreshed for a period of time (here 10s), so that the video playback progress (i.e. the current playback time) is as shown in Figure 6a(2)
  • the current play time control 403 shows 0 minutes and 0 seconds, which gradually changes to the current play time control 403 shown in Figure 6a (3), which shows 0 minutes and 10 seconds.
  • the position of the playback progress control 404 in the playback progress bar control 405 also gradually changes from position 0 shown in Figure 6a(2) to position 1 shown in Figure 6a(3).
  • the user operation that triggers the mobile phone to perform S101 is not limited to the user's operation of clicking the third video window control 3037 in Figure 6a(1), and may also include but is not limited to the following methods:
  • Method 1 The user clicks the playback pause control 405 shown in Figure 6a(2), causing the video in the display interface 401 to switch from the paused state to the playing state.
  • the double vertical bar icon in the playback pause control 405 shown in Figure 6a(2) indicates that the video in the display interface 401 is in the playing state.
  • the triangle icon in the control 405 can be used to indicate that the video in the display interface 401 is in a paused state. Then, when the video in the display interface 401 is in the playing state, S101 can continue to be executed, so that the video stream can continue to be received.
  • S101 may interrupt execution.
  • the steps after S101 may also interrupt execution.
  • Method 2 The user can change the playback progress of the video in the display interface 401 by dragging the playback progress control 404 in Figure 6a(2). Then the mobile phone can also trigger the execution of S101 after detecting the user operation. Among them, the playback display control 405 needs to display the double vertical bar icon shown in Figure 6a(2) to indicate that the video in the display interface 401 is in a playback state.
  • the audio and video streams in S101 may include video streams and/or audio streams.
  • the application scenario may be to perform media information identification on the played video data shown in Figure 6a, and also It can be to identify the media information of the played audio data.
  • the method is the same and will not be described again here.
  • audio data may include but is not limited to music, recordings, etc.
  • the media framework module serves as an output channel for system-level media data.
  • the electronic device of the present application configures the media framework module with a cache area (for example, storage space A) for forward storage of media data. Then the media framework The module can cache the media data to be output to the buffer area before outputting the media data, so that no matter whether the electronic device is outputting any type of media data, the media framework module can cache the current playback of the media data output by the electronic device. A period of media data before the progress time point (including this time point) is stored, which can realize forward caching of globally played media data.
  • media information identification is needed, media data only need to be read from the cache area. There is no need to collect media data in real time. This can shorten the collection time of media data, improve the recognition speed of media information, and improve the accuracy of media information identification. and success rate.
  • Figure 7 exemplarily shows a schematic diagram of changes in the playback progress bar control 402 of the movie named "Movie Name 1" played in Figure 6a.
  • Figure 7 shows a schematic diagram of the change of the playback progress bar control 402 in the process of changing from Figure 6a(2) to Figure 6a(3), wherein, from Figure 6a(2) to Figure 6a(3), the playback progress changes from 0 minutes and 0 seconds changes to 0 minutes and 10 seconds.
  • Figure 8 exemplarily shows the process when the playback progress of the movie changes from t0 to t1.
  • Schematic diagram of the changes in space A Combined with Figure 7, Figure 9a exemplarily shows a schematic diagram of the changes in storage space A during the process when the playback progress of the movie changes from t2 to t3.
  • t0 represents the starting playback time of the video, that is, t0 is 0 minutes and 0 seconds.
  • t3 represents 0 minutes and 10 seconds shown in the current playback time control 403 in Figure 6a(3).
  • t1, t2, and t3 are respectively the time points of the current playback progress of the video played in Figure 6a after the media framework module refreshes the playback progress three times.
  • the time point corresponding to the control 301 represents the time point of the current playback progress of the video in Figure 6a.
  • the Pi frame represents the i-th audio and video segment to be output obtained by the media framework module.
  • the i-th audio and video segment represents the i-th audio and video segment in at least one audio and video segment obtained after encoding processing by the media framework module.
  • Figure 6a(1) As shown in Figure 7(1) and Figure 8(1), please refer to Figure 6a(1).
  • the mobile phone When the user clicks the third video window control 3037, the mobile phone can trigger the execution of S101 in Figure 5a in response to the user operation. Then when the user clicks the third video window control 3037 to display the display interface 401 shown in Figure 6a(2), the current playback time control 403 shown in Figure 6a(2), the time point of the current playback progress of the above-mentioned movie is t0 (i.e. 0 minutes and 0 seconds).
  • the mobile phone in Figure 6a after the mobile phone in Figure 6a is turned on, it can perform an initialization operation.
  • the contents in the storage space A mentioned in S102 can be cleared, so that there is no cached content in the storage space A after the mobile phone is turned on, so as to facilitate Caching
  • the media framework module can cache the audio and video content at the current playback time after the phone is turned on and before that time in storage space A.
  • the mobile phone can respond to the user operation and trigger the execution of the process shown in Figure 5a.
  • Figure 8(2) to Figure 8(5) When the media framework module encodes the received video stream, each time it encodes an audio and video segment, it can write the audio and video segment into storage space A. For example, the media framework module writes P1 frames, P2 frames,...P100 frames to storage space A in sequence. In the same way, the 100 audio and video clips can be written to the storage space B shown in Figure 5b. It should be noted that this application does not limit the storage location of each audio and video clip in storage space A.
  • the media framework module every time the media framework module encodes 100 audio and video clips, it outputs the target audio and video clips (ie, the 100 audio and video clips here). Then the media framework module can read the above-mentioned P1 frame, P2 frame,...P100 frame in the storage space B at one time, and form a complete target audio and video clip in time sequence and output it, so that as shown in Figure 6a(2)
  • the playback progress of the playback content in the display interface 401 shown is refreshed for the first time, as shown in the figure As shown in 7(2), the time point of the playback progress is updated from t0 to t1.
  • storage space A caches the time point of the current playback progress (i.e. t1) and the video content before that time point (t0 to t1 time period), that is, the above-mentioned P1 frame , P2 frame,...P100 frame.
  • the media framework module continues to receive the video stream of the video application 1.
  • the mobile phone can refresh the playback progress of the playback content on the display interface 401 shown in Figure 6a(2) twice, as shown in Figure 6a(2) 7(2) and Figure 7(3), the time point of the playback progress is updated from t1 to t2. While the playback progress time point is updated from t1 to t2, the media framework module continues to write frames P101 to P200 to storage space A in sequence.
  • the space size of storage space A can be used to write up to 200 audio and video clips.
  • the media framework module continues to receive the video stream of video application 1, then Figure 9a(2) to Figure 9a( As shown in 4), when the media framework module encodes the received video stream, each time an audio and video segment is encoded, the audio and video segment can be written to storage space A.
  • the media framework module writes P201 frames, P202 frames,...P300 frames to storage space A in sequence.
  • the media framework module when the media framework module sequentially writes P201 frame, P202 frame, ... P300 frame to storage space A, because when P201 frame needs to be written, storage space A is full and there is no free space left.
  • the media framework module can delete the earliest written P1 frame from storage space A, and then write the P201 frame into storage space A; similarly, as shown in Figure 9a(3), the media When the framework module writes the P202 frame to storage space A, it can delete the earliest written P2 frame from storage space A, and then write the P202 frame into storage space A.
  • the media framework module can sequentially write frames P201, P202, ...P300 to storage space A, and at the same time, delete frames P1 to P100 from storage space A in sequence, as shown in Figure 9a(4) , when the time point of the current playback progress is t3, the storage content of storage space A is frame P101 to frame P300.
  • the media framework module after the media framework module refreshes the playback progress twice, it can also write frames P201 to P300 into storage space B in sequence. For example, every time the media framework module encodes 100 audio and video clips, it outputs the target audio and video clips once (That is, the P201 frame to P300 frame here). Then the media framework module can read frames P201 to P300 in storage space B at one time, and form a complete target audio and video clip in time sequence and output it, so that the display interface 401 shown in Figure 6a(2) The playback progress of the playback content is refreshed for the third time. As shown in Figure 7(3) and Figure 7(4), the time point of the playback progress is updated from t2 to t3.
  • the time point of the playback progress is t3, and the time point of the current playback progress (i.e. t3) and the time point are cached in storage space A.
  • the video content before the time point that is, the above-mentioned P101 frame to P300 frame.
  • t3 is 0 minutes and 10 seconds shown in the current playback time control 403 shown in Figure 6a(3)
  • the mobile phone can refresh the playback progress three times, so that the display interface of the mobile phone gradually changes from Figure 6a(2) to Figure 6a (3), in which the process of gradually changing from Figure 6a(2) to Figure 6a(3) is the process of dynamic video playback, which is not shown here.
  • the media framework module outputs a target audio and video clip to update the playback progress of the video
  • the update is imperceptible to the user.
  • the interval between refreshing the playback content displayed on the mobile phone is At the millisecond level and subtle level, users cannot perceive it.
  • the media framework module detects that storage space A is full of data when writing an audio and video clip to storage space A (not limited to updating the playback progress)
  • the media framework module can follow Figure 9a(2) and Figure 9a (3)
  • the schematic method is to first delete data and then write data, so that the data cached in storage space A can be dynamically updated.
  • the update of the playback progress of audio and video data is not only limited to the "backward refresh” process shown in Figure 6a(2) to Figure 6a(3), but may also include the "forward refresh” process. "Refresh” process.
  • FIG. 6b shows a schematic diagram of two playback progress bars.
  • the above-mentioned "backward refresh” can be expressed as a process of changing from Figure 6b(1) to Figure 6b(2).
  • the playback progress control 404 can be moved from position 2 shown in Figure 6b(1) manually (i.e., the user drags the playback progress control 404) or automatically (i.e., it moves automatically with the automatic update of the playback progress).
  • Position 1 shown in 6b(2) is to realize that the current playback progress of the video is directly or gradually updated from 0 minutes and 5 seconds to 0 minutes and 10 seconds.
  • forward refresh can be expressed as the process of changing from Figure 6b(2) to Figure 6b(1).
  • Figure 9b exemplarily shows that for the user's "forward refresh” operation, the media framework module The process of refreshing storage space A in response to the user operation.
  • Position 2 shown in 6b(1) is to directly update the current playback progress of the video from 0 minutes and 10 seconds to 0 minutes and 5 seconds. For example, take t3 as 0 minutes and 10 seconds here and t2 as 0 minutes and 5 seconds as shown in Figure 9b(1).
  • the time point of the playback progress is t3, which is 0 minutes and 10 seconds, the storage space
  • the data stored in A is P101 frame to P300 frame.
  • the user moves the playback progress from position 1 to position 2, so that the time point of the playback progress is updated from t3 to t2.
  • the mobile phone can respond to the user operation, and the media framework The module deletes frames P201 to P300 in storage space A to refresh the content of storage space A.
  • the playback progress is "forward refreshed" from time point t3 to time point t2.
  • the storage content of storage space A is P101 frame to P200 frame.
  • the storage content of storage space A is time Video clip from point t1 to time point t2.
  • the media framework module can "backward refresh” along with the audio and video playback progress, refreshing the audio and video content stored in storage space A in real time, specifically by deleting the earliest written audio and video clip. , and then write an audio and video clip to be output to dynamically refresh the content of storage space A.
  • the audio and video content stored in storage space A is always the time point of the current playback progress of the video currently played by the mobile phone and the audio and video data before that time point, so that the mobile phone can use the real-time audio and video data in storage space A to Perform media information acquisition and identification operations.
  • the media framework module can be based on the refreshed current playback progress. 1.
  • a "forward refresh” operation on the playback progress such as forward refresh from playback progress 2 (for example, t3) to playback progress 1 (for example, t2)
  • the media framework module can be based on the refreshed current playback progress. 1.
  • a piece of audio and video data before the time point so that the mobile phone can use the real-time audio and video data refreshed in storage space A to perform operations such as media information acquisition and identification.
  • the media framework module can not only store a piece of audio and video data before the current playback progress time point of the mobile phone in the storage space A.
  • the media framework module can also dynamically refresh the storage content of storage space A as the playback progress of the audio and video data currently played by the mobile phone is refreshed (either backward refresh or forward refresh). This allows storage space A to always store a piece of audio and video data before the time point of the current playback progress (including this time point).
  • the media framework module can always obtain the time point (including this time point) of the current playback progress of the audio and video data from the storage space A.
  • the current audio and video data can be used for media information identification, which can further improve the accuracy of the obtained media information.
  • Figures 10 and 11 exemplarily show schematic diagrams of scenarios in which an electronic device obtains media information.
  • the process of obtaining media information by an electronic device may include the following steps:
  • the media framework module monitors the preset event.
  • the media framework module can register the monitoring of the "media information acquisition” event. After the media framework module registers the monitoring of the event, the mobile phone system can be detected.
  • the ability of the "Media Information Acquisition” event means that regardless of whether the user's input operation triggers the "Media Information Acquisition” event on the application's display interface or the system's display interface, the media framework module can monitor the event.
  • the media framework module continuously or periodically monitors whether the "media information acquisition” event is triggered.
  • the time point when the event is triggered must be the current playback progress of the audio and video data. time point. Then the storage space A of the mobile phone has the audio and video data being output by the mobile phone cached. For the target audio and video clips before the time point of the current playback progress (including this time point), the mobile phone can read the cached target audio and video clips from storage space A for operations such as media information identification and acquisition, and use The operation results are fed back to the user. As a result, the target audio and video segments for media information recognition can be captured instantly and accurately, thereby improving the success rate of media information recognition.
  • the moment when the user triggers the "media information acquisition" event is when the user hears or sees interesting content from the audio and video data output by the mobile phone, and thus needs to obtain media information.
  • the mobile phone of this application can obtain the audio and video data at and before this moment through the media framework module for media information recognition, so that the recognized media information can easily meet user needs and improve the success rate of media information recognition.
  • the media framework module After monitoring the preset event, the media framework module can read audio and video data from storage space A.
  • storage space A stores audio and video data in the time period from t1 to t3.
  • the media framework module can read out all the cached content of storage space A, that is, the audio and video data in the time period from t1 to t3, for media information identification.
  • the media framework module can also store part of the cached content in space A.
  • the part of the cached content may include the time point of the current playback progress (an example of t3) and the preset time period before the time point (for example, 3s). audio and video content.
  • the preset duration can be flexibly configured according to differences in uses and scenarios after reading the audio and video data, and this application does not impose restrictions on this.
  • the mobile phone after the mobile phone reads cached forward audio and video data from storage space A, it can process the audio and video data to obtain media information. If the type of media information is different, the preset duration may be different. For example, in the scene of music recognition of song names, the preset duration can be any value within the range of 5s to 6s.
  • the mobile phone reads cached forward audio and video data from storage space A and uses it to play back the audio and video data (such as video clips of wonderful moments of scoring goals), then the preset duration can be 2s, and the specific duration can be based on Flexible configuration is required, and this application does not impose restrictions on this.
  • the media framework module can send the read audio and video data to the video application 1.
  • the media framework module can send audio and video data read from storage space A to video application 1.
  • the video application 1 has the function of identifying media information, then the video application 1 can identify the time point of the current playback progress and the audio and video data for a period of time before the time point (for example, the 8s to 8s of the video played in Figure 11 10s of video data) to identify media information.
  • video application 1 identifies the name of the background music based on the recently played video data of 3 seconds, and can determine that the name of the background music is "Music 1", and the mobile phone can respond to the user clicking on the media information in Figure 11(1)
  • the display interface 401 may include a floating window control 409, and media information may be displayed in the floating window control 409.
  • the recognition result here is "Recognition result: the background music name is "Music 1"".
  • the media data used for media information identification has been cached in storage space A before the media data is output. Then when the user triggers the media information acquisition control 408, the media framework module can immediately retrieve it from The media data used to identify media information is read from the storage space A without the need to collect the media data in real time, so that the switching time from Figure 11(1) to Figure 11(2) can be short and the user can be unaware The switching time is reached. Compared with the process of switching from Figure 3(4) to Figure 3(5) in traditional technology, which requires a longer waiting time, the user of this application waits for a shorter time for the recognition result of the media information.
  • the system of this application can also integrate the function of identifying media information.
  • the media framework module can send the read audio and video data to the media information identification module configured in the mobile phone system, and the media information identification module can
  • the audio and video data may be used for media information identification, or the audio and video data may be output, displayed and played as playback video clips (for specific methods, please refer to the subsequent description of Figure 13), which will not be described again here.
  • the media information identification module when the media information identification module identifies media information for audio and video data, it can use traditional or future developed technologies such as "audio fingerprint" to identify media such as song names, singer information, and musical instruments in the songs from the audio data. information. For example, when identifying media information from audio and video data, the media information identification module can perform image feature identification based on each frame in the video data to identify media information such as drama titles and actor names. Of course, the above media information is only used as an example.
  • the mobile phone of this application performs media information identification on the time point of the current playback progress of the mobile phone and the audio and video data before that time point, this application uses the media information that can be recognized by the mobile phone. There is no restriction on the type of media information, and more types of media information can be included. At the same time, this application does not limit the technical means of using a piece of media data (such as audio and video data) to identify media information.
  • the specific technical means and the type of media information to be identified can be flexibly configured according to needs.
  • FIG. 12 exemplarily shows a schematic diagram of the application scenario of the electronic device of the present application.
  • the display interface 600 of the mobile phone includes one or more controls, which may include but are not limited to: power icons, network icons, application icons, etc.
  • the user can click the icon 601 of the video application 1 on the display interface 600 to start the video application 1.
  • the mobile phone can start the video application 1 and display the application interface 602 of the video application 1 in response to the user's click operation.
  • the application interface 602 may include one or more controls, including the currently displayed video screen 607, the playback progress bar control 402, the playback progress control 404, the current playback time control 403, the video duration control 406, and the playback pause control 405.
  • the video displayed in the application interface 602 is in a paused state.
  • the video displayed in the application interface 602 may also be in a playing state, and this application does not limit this.
  • the mobile phone may display the sidebar 603 on the right edge (or the left edge) of the application interface 602 of the video application 1 in response to the user operation.
  • the sidebar includes one or more controls, such as an email application icon, a memo application icon, a gallery application icon, a file management application icon, a video application 2 icon 6031, etc.
  • the sidebar may also include an add option for adding the specified application's icon to the sidebar. It should be noted that the names, quantities and positions of each application in Figure 12(3) are only illustrative examples and are not limited in this application.
  • the user can drag the icon 6031 of the video application 2 in the sidebar to the lower half area of the mobile phone display window and release, for example, drag the icon 6031 in the direction of the dotted arrow.
  • the mobile phone can respond to the user operation by splitting the display window of the mobile phone into two screens, including a display window 608 and a display window 604.
  • the display of the mobile phone Window 608 is used to display the application interface of video application 1
  • displaying Window 604 is used to display the application interface of video application 2.
  • the user can click the playback pause control 405 located in the display window 608 and the display window 604, and the mobile phone can display in the display window 608 and the display window 604 respectively in response to the user operation.
  • the media framework module serves as an output channel for system-level audio and video data.
  • the media framework module can receive the video stream of the video application 1.
  • the media framework module can receive the video stream of the video application 2.
  • the media framework module can process the two sets of received video streams in parallel, and process the two sets of video streams respectively in accordance with the method of Figure 5a or related embodiments, so that the storage space A caches the time point of the current playback progress of video application 1 and the video data of the first preset duration before the time point, and the time point where the current playback progress of the video application 2 is cached and the video data of the second preset duration before the time point.
  • the mobile phone can also provide the user with an entrance to set the size of storage space A for the split-screen window on the system interface.
  • the user can set the size of the storage space A3 and the size of the storage space A4 respectively for the first window and the second window in the split-screen scenario in the system interface, where the storage space A includes the storage space A3 and the storage space A4.
  • storage space A3 (including storage space A3-1 for storing audio, and storage space A3-2 for storing images) can store up to a first preset duration (for example, 3s) of video data (including audio and images),
  • the storage space A4 (including the storage space A4-1 for storing audio and the storage space A4-2 for storing images) can store video data of a second preset duration (for example, 4 seconds) at most.
  • the first preset duration and the second preset duration respectively correspond to the space sizes of the storage space A3 and the storage space A4.
  • the first preset duration and the second preset duration can be flexibly configured by the user.
  • the first preset duration and the second preset duration can be flexibly configured by the user.
  • the first preset time period and the second preset time period may be the same or different.
  • the display window 608 formed first in Figure 12(4) is the first window
  • the display window 604 formed subsequently is the second window.
  • the media framework module can continuously update the current playback progress time point and the time written into the video application 1 in the storage space A3.
  • the video data before the point (video data with a duration of 3s), and the current playback progress time point of the video application 2 and the video data before this time point (video data with a duration of 4s) are continuously updated and written in the storage space A4.
  • the process of the media framework module writing video data to storage space A3 and storage space A4 is similar to the process of writing audio and video data of a single application to storage space A mentioned in the previous embodiment, and will not be described in detail here. Refer to the process of Figures 7 to 9a.
  • the time point t4 of the current playback progress shown by the current playback time control 403 in the display window 608 is 10 minutes and 10 seconds.
  • the mobile phone obtains time point t4 and the video clip 605 before time point t4 (duration is 3 seconds) from storage space A3.
  • the display interface of the mobile phone can be switched from Figure 13(1) to Figure 13(2).
  • the display window 609 The video clip 605 is played within the video clip 605 to realize the playback of the video clip for a period of time before the moment when the user triggers the "media information acquisition" event.
  • the video clip 605 is a 3s video clip from 10 minutes and 8 seconds to 10 minutes and 10 seconds of the video played by the video application 1 in the display window 608 .
  • the playback progress bar control 404 is located at the starting position of the playback progress bar control 402, and the time shown by the current playback time control 403 is 0 minutes and 0 seconds, and the video The length of the video clip shown in the duration control 406 is 3 seconds, and the icon shown in the playback pause control 405 indicates that the video clip 605 is in a paused state.
  • the mobile phone can play the video clip 605 in the display window 609 in response to the user operation.
  • the content played by the display window 609 is the video clip 605, that is, the video clip 605 is the complete video played by the display window 609 in Figure 13(2). Therefore, in Figure 13(2)
  • the time shown by the current playback time control 403 is 0 minutes and 0 seconds, and the video clip duration shown by the video duration control 406 is 3s, instead of the video playback duration control 406 in the display window 608 in Figure 13(1). of 50 minutes.
  • a preset operation can be triggered on the display window 609 in Figure 13(2) to switch the content of the display window 609 of the mobile phone to the one in Figure 13(1) The contents of display window 608.
  • the mobile phone can display the display window 609 shown in Figure 13(2) within the display window 608 of Figure 13(1).
  • the display window 609 may be displayed in the display window 608 in FIG. 13(1) in a floating window or a small window.
  • the video played in the display window 608 in Figure 13(1) can continue to be played without being affected by the display window 609. to avoid affecting the user's browsing of the original video due to playback of video clips.
  • the display position of the display window 609 in the display interface in Figure 13(1) is not limited to within the display window 608, and can be anywhere in the display interface in Figure 13(1). , the user can also adjust the position of the display window 608, and this application does not limit this.
  • the time point t5 of the current playback progress shown by the current playback time control 403 in the display window 604 is 10 minutes and 0 seconds.
  • the mobile phone obtains time point t5 and the video clip 606 before time point t5 (duration is 4 seconds) from storage space A4.
  • the display interface of the mobile phone can be switched from Figure 13(1) to Figure 13(2), as shown in Figure 13(2), and in the display window
  • the video clip 606 is played in 610 to realize the playback of the video clip for a period of time before the moment when the user triggers the "media information acquisition" event.
  • the video clip 606 is a 4s video clip of the video clip from 9 minutes 57 seconds to 10 minutes 0 seconds in the video played by the video application 2 in the display window 604 .
  • the playback progress bar control 404 is located at the starting position of the playback progress bar control 402, and the time shown by the current playback time control 403 is 0 minutes and 0 seconds.
  • the length of the video clip shown in the duration control 406 is 4 seconds, and the icon shown in the playback pause control 405 indicates that the video clip 606 is in a paused state.
  • the content played by the display window 610 is the video clip 606, that is, the video clip 606 is the complete video played by the display window 610 in Figure 13(2). Therefore, in Figure 13(2) while currently playing The time shown by the time control 403 is 0 minutes and 0 seconds, and the video clip duration shown by the video duration control 406 is 4s, instead of the 50 minutes shown by the video playback duration control 406 in the display window 604 in Figure 13 (1). .
  • a preset operation can be triggered on the display window 610 in Figure 13(2) to switch the content of the display window 610 of the mobile phone to the one in Figure 13(1) The contents of display window 604.
  • the mobile phone can display the display window 610 shown in Figure 13(2) within the display window 604 of Figure 13(1).
  • the display window 610 may be displayed in the display window 604 in FIG. 13(1) in a floating window or a small window.
  • the video played in the display window 604 in FIG. 13(1) can continue to be played without being affected by the display window 610. to avoid affecting the user's browsing of the original video due to playback of video clips.
  • the display position of the display window 610 in the display interface in Figure 13(1) is not limited to within the display window 604, and can be anywhere in the display interface in Figure 13(1).
  • the user can also adjust the position of the display window 610 in the display interface in Figure 13(1), and this application does not limit this.
  • the process of switching from Figure 13(1) to Figure 13(2) is similar to the process of switching from Figure 11(1) to Figure 11(2), because the media framework module can directly access the storage space from storage space A. Read media data used to identify media information, or media data used for playback, so that the screen switching time is shorter, the user does not need to wait for a long time for the recognition result or the playback result, and the user experience is better.
  • the user can also divide the storage space A into storage spaces A5, A5, and A5 for different displays in the system settings interface as needed.
  • Storage space A6, etc. the user can set the space size corresponding to the storage space of different display screens, then the media framework module can cache the audio and video data to be output to different display screens into the storage space of the corresponding display screen, so that the user can
  • the mobile phone can obtain the current playback progress time point in the audio and video data displayed on the display screen and the audio and video section before that time point from the storage space of the corresponding display screen.
  • Data is used for processing such as identification or playback of media information.
  • the mobile phone when the mobile phone simultaneously plays at least two sets of media data in a split-screen scenario or a multiple display screen scenario, the mobile phone can also cache each display window or each display screen in the cache area accessible by the media frame module.
  • a piece of audio and video data before the current playback progress so that when the user triggers the "media information acquisition" event, the media framework module can read a piece of audio and video data before the current playback progress on each display window or each display screen from the buffer area, It is used for processing media information recognition or audio and video data playback, and can achieve instant acquisition of media information in multi-window and multi-screen scenarios.
  • the media data displayed in different display windows or different display screens is not limited to the video data shown in Figure 12 and Figure 13, and different types of media data can be displayed respectively. For example, play video in one display window and play music in another display window. Or use one display screen to play music, use another display screen to play recordings, etc. This application does not limit this.
  • the execution principle of the method is similar and will not be described again here.
  • the electronic device includes corresponding hardware and/or software modules that perform each function.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed by hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions in conjunction with the embodiments for each specific application, but such implementations should not be considered to be beyond the scope of this application.
  • FIG. 14 shows a schematic block diagram of a device 300 according to an embodiment of the present application.
  • the device 300 may include: a processor 301 and a transceiver/transceiver pin 302, and optionally, a memory 303.
  • bus 304 which includes, in addition to a data bus, a power bus, a control bus, and a status signal bus.
  • bus 304 includes, in addition to a data bus, a power bus, a control bus, and a status signal bus.
  • various buses are referred to as bus 304 in the figure.
  • the memory 303 may be used for instructions in the foregoing method embodiments.
  • the processor 301 can be used to execute instructions in the memory 303, and control the receiving pin to receive signals, and control the transmitting pin to send signals.
  • the device 300 may be the electronic device or a chip of the electronic device in the above method embodiment.
  • This embodiment also provides a computer storage medium.
  • Computer instructions are stored in the computer storage medium.
  • the electronic device causes the electronic device to execute the above related method steps to achieve the acquisition of media information in the above embodiment. method.
  • This embodiment also provides a computer program product.
  • the computer program product When the computer program product is run on a computer, it causes the computer to perform the above related steps to implement the media information acquisition method in the above embodiment.
  • inventions of the present application also provide a device.
  • This device may be a chip, a component or a module.
  • the device may include a connected processor and a memory.
  • the memory is used to store computer execution instructions.
  • the processor can execute computer execution instructions stored in the memory, so that the chip executes the media information acquisition method in each of the above method embodiments.
  • the electronic equipment, computer storage media, computer program products or chips provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the corresponding methods provided above. The beneficial effects of the method will not be repeated here.
  • the disclosed devices and methods can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or units. Components may be combined or may be integrated into another device, or some features may be omitted, or not performed.
  • the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.
  • a unit described as a separate component may or may not be physically separate.
  • a component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit.
  • the above integrated units can be implemented in the form of hardware or software functional units.
  • Integrated units may be stored in a readable storage medium if they are implemented in the form of software functional units and sold or used as independent products.
  • the technical solutions of the embodiments of the present application are essentially or contribute to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium , including several instructions to cause a device (which can be a microcontroller, a chip, etc.) or a processor to execute all or part of the steps of the methods of various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program code.
  • the steps of the methods or algorithms described in connection with the disclosure of the embodiments of this application can be implemented in hardware or by a processor executing software instructions.
  • Software instructions can be composed of corresponding software modules.
  • Software modules can be stored in random access memory (Random Access Memory, RAM), flash memory, read only memory (Read Only Memory, ROM), erasable programmable read only memory ( Erasable Programmable ROM (EPROM), electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), register, hard disk, removable hard disk, compact disc (CD-ROM) or any other form of storage media well known in the art.
  • An exemplary storage medium is coupled to the processor such that the processor can read information from the storage medium and write information to the storage medium.
  • the storage medium can also be an integral part of the processor.
  • the processor and storage media may be located in an ASIC. Additionally, the ASIC can be located in a network device. Of course, the processor and storage media can also exist as discrete components in the network device.
  • Computer-readable media includes computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Storage media can be any available media that can be accessed by a general purpose or special purpose computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Marketing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Telephone Function (AREA)

Abstract

本申请实施例提供了一种媒体信息的获取方法及电子设备，涉及终端设备技术领域，该方法包括：在媒体数据输出之前，将媒体数据缓存至系统级的存储空间，以实现对电子设备输出的任意媒体数据的播放进度的时间点及该时间点之前的媒体数据的缓存。在用户触发获取媒体信息的操作时，可从缓存的媒体数据中获取播放进度的时间点及该时间点前的媒体数据，以用于输出媒体信息，可提升媒体信息的识别效率和识别准确度。

Description

媒体信息的获取方法及电子设备
本申请要求于2022年03月28日提交中国国家知识产权局、申请号为202210309845.3、申请名称为“媒体信息的获取方法及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及终端设备技术领域，尤其涉及一种媒体信息的获取方法及电子设备。
背景技术
随着电子设备的飞速发展，电子设备可提供的功能越来越丰富。目前的电子设备可对电子设备正在播放的媒体数据(例如音视频数据等)，获取媒体信息(例如背景音乐等)并输出。例如对播放的广告、新闻等媒体数据，输出媒体数据中的背景音乐的信息。再如对播放的视频，输出视频名称或视频画面中演职员等信息。
目前的电子设备的只可以在用户触发获取媒体信息的功能之后，开始采集电子设备正在输出的媒体数据，以用于媒体信息的识别。这样不仅识别媒体信息的时长较长，而且识别到的媒体信息可能不是用户所需要的媒体信息，从而导致媒体信息的准确度较低的问题。
发明内容
为了解决上述技术问题，本申请提供一种媒体信息的获取方法及电子设备。在该方法中可对电子设备输出的媒体数据进行前向缓存，以用于媒体信息的获取，可提升媒体信息的识别效率和识别准确度。
第一方面，本申请实施例提供一种媒体信息的获取方法，应用于电子设备。该方法包括：在输出第一媒体数据之前，将所述第一媒体数据缓存至第一存储空间；其中，所述第一存储空间用于缓存所述电子设备输出的任意媒体数据中的目标媒体数据，所述目标媒体数据为所述任意媒体数据中，位于当前播放进度的时间点及所述时间点之前的媒体数据；其中，所述目标媒体数据包括所述第一媒体数据；响应于在第一时刻接收到的第一用户操作，从所述第一存储空间获取第二媒体数据，其中，所述第一媒体数据为在第二时刻与所述第一时刻之间缓存的媒体数据，所述第二媒体数据为在第三时刻与所述第一时刻之间缓存的媒体数据，所述第二时刻位于所述第一时刻之前，所述第三时刻位于所述第一时刻之前；其中，所述第三时刻与所述第二时刻相同，或，所述第二时刻位于所述第三时刻之前；基于所述第二媒体数据，输出第一媒体信息。
示例性的，以电子设备为手机为例，第一存储空间可为媒体框架模块可访问的存储空间。媒体框架模块为手机的系统级媒体数据的输出通道。手机在输出任意媒体数据时，该媒体数据在输出之前要经过媒体框架模块的处理，那么媒体框架模块可获取到手机待 输出的任意媒体数据。该第一存储空间可用于手机输出的任意媒体数据中当前播放进度的时间点及该时间点之前的媒体数据。在手机输出第一存储空间中缓存的媒体数据之后，第一存储空间中的数据不会被清空，那么不论手机输出任何类型的媒体数据，手机的第一存储空间可存储有该输出的媒体数据的播放进度前的一段媒体数据，以用于媒体信息的输出。
示例性的，媒体数据的类型可包括但不限于图像、音频、视频。媒体信息可以是媒体数据(例如视频片段的回放、音频片段的回放)，也可以是从媒体数据中识别到的信息(例如视频中出现的演员的名字、剧名、背景音乐名称等)。
输出媒体信息的方式可包括但不限于：窗口、悬浮窗、对话框等。
示例性的，第一用户操作可是对手机系统的预设图标、虚拟按键的操作，或预设手势的操作，第一用户操作还可以是对手机输出媒体数据的输出界面的操作，本申请对此不做限制。
可以理解的是，手机接收到第一用户操作的第一时刻可为手机输出的媒体数据的播放进度对应的时间戳，对于时间戳的具体实现方式，本申请不做限制。
示例性的，第一时刻的时间戳可以是播放进度时间，例如手机播放的视频播放到0分30秒，则第一时刻可为0分30秒。
示例性的，第一时刻的时间戳也可以是播放进度时间对应的系统时间信息。例如手机播放视频时，播放进度为0分30秒时，手机的系统时间为2022年1月1日，22点0分，则第一时刻可为2022年1月1日，22点0分。对于本申请各个实施方式中所述的时刻，其实现原理，与这里的解释相同，后续不再一一赘述。
另外，本申请各个实施方式中提及的在两个时刻之间缓存的媒体数据，可包括在该两个时刻所缓存的媒体数据。
例如，手机的视频播放到第0分30秒，那么第一媒体数据可为手机缓存的该视频中第0分0秒至第0分30秒的数据。在第二时刻与第三时刻相同时，第二媒体数据也为该视频中第0分0秒至第0分30秒的数据。在第二时刻位于第三时刻之前时，例如第二媒体数据为视频中第0分5秒至第0分30秒的数据。换言之，在手机接收到第一用户操作后，手机可将第一存储空间缓存的全部，或部分缓存数据读取出来，以用于媒体信息的识别或回放操作，以输出媒体信息。
可选地，第一存储空间可缓存有至少一个应用的媒体数据，当第一存储空间缓存不同应用的媒体数据时，则手机在获取第二媒体数据时，可从第一存储空间读取第一用户操作对应的应用的媒体数据。
本实施方式中，可在媒体数据输出之前，对媒体数据进行缓存，且缓存该媒体数据的第一存储空间为手机的系统级媒体数据的缓存空间，手机输出任意媒体数据，在输出之前，均要缓存至第一存储空间。那么在手机接收到第一用户操作后，可响应于该第一用户操作，将手机当前输出的播放进度前的一段媒体数据读取出来，以用于输出媒体信息。不仅可降低媒体信息的识别时长，而且，输出的媒体信息的准确度更高。
根据第一方面，所述将所述第一媒体数据缓存至第一存储空间之后，所述方法还包 括：在输出第三媒体数据之前，将所述第三媒体数据缓存至所述第一存储空间。
示例性的，目标媒体数据可包括第三媒体数据。
示例性的，第三媒体数据可以是因第一媒体数据对应的媒体(例如视频)的播放进度更新而继续写入至第一存储空间的数据。例如上述举例的视频中第30秒至第31秒的的视频数据。
示例性的，第三媒体数据可以是与第一媒体数据属于更新一次播放进度时，缓存至第一存储空间中的媒体数据。
示例性的，第三媒体数据与第一媒体数据可以是手机的显示屏的不同显示窗口内输出的媒体数据，或手机的不同显示屏内输出的媒体数据。
此外，第一媒体数据，与第三媒体数据对应的应用相同或不同均可，本申请对此不做限制。
本实施方式中，只要是该电子设备待输出的媒体数据，在输出之前均要先写入至该第一存储空间，且该媒体数据输出之后，在第一存储空间中缓存的该媒体数据不会被清除。那么不论电子设备输出任何媒体数据，本实施方式均可在接收到第一用户输入时，获取到当前播放进度前的一段媒体数据，以用于输出媒体信息。
根据第一方面，或者以上第一方面的任意一种实现方式，所述第一媒体数据包括至少一个第一媒体片段，所述第三媒体数据包括至少一个第二媒体片段；所述将所述第一媒体数据缓存至到第一存储空间，包括：按照所述至少一个第一媒体片段的输出顺序，依次将所述至少一个第一媒体片段缓存至所述第一存储空间；所述将所述第三媒体数据缓存至所述第一存储空间，包括:按照所述至少一个第二媒体片段的输出顺序，依次将所述至少一个第二媒体片段缓存至所述第一存储空间。本实施方式中，可按照媒体片段的输出顺序，依次将媒体片段写入至第一存储空间。示例性的，媒体片段可包括音频片段，和/或，图像序列。
根据第一方面，或者以上第一方面的任意一种实现方式，所述按照所述至少一个第二媒体片段的输出顺序，依次将所述至少一个第二媒体片段缓存至所述第一存储空间，包括：检测到所述第一存储空间中不存在空闲空间，按照缓存时间从早到晚的顺序，依次删除所述第一存储空间中已缓存的第一媒体片段，以及按照所述至少一个第二媒体片段的输出顺序，依次将所述至少一个第二媒体片段缓存至所述第一存储空间。
示例性的，手机在向第一存储空间写入待输出的媒体片段时，在检测到第一存储空间不存在空闲空间时，可按照先进先出的方式，来将第一存储空间中最早缓存的媒体片段删除，然后，再写入第二媒体片段至第一存储空间，以实现第一存储空间中缓存的数据的动态刷新，使得第一存储空间保持缓存电子设备当前输出的媒体数据的播放进度的时间点及该时间点之前的媒体数据。
根据第一方面，或者以上第一方面的任意一种实现方式，所述第一存储空间包括：用于缓存音频数据的第一空间和用于缓存图像数据的第二空间；将所述目标媒体数据缓存至第一存储空间，包括：将所述目标媒体数据中的音频数据缓存至所述第一空间；和/或，将所述目标媒体数据中的图像数据缓存至第二空间；其中，所述目标媒体数据包括 所述第一媒体数据，和/或，所述第三媒体数据。
示例性的，在第一存储空间中，用于缓存音频数据的空间，和用于缓存图像数据的空间可不同，以实现音频数据和图像数据的并行写入至第一存储空间，提升数据缓存效率。
根据第一方面，或者以上第一方面的任意一种实现方式，所述目标媒体数据包括至少一个目标媒体片段；依次将至少一个目标媒体片段缓存至所述第一存储空间，包括：依次将所述至少一个目标媒体片段中的至少一个音频片段，缓存至所述第一空间；和/或，依次将所述至少一个目标媒体片段中的至少一帧图像，缓存至所述第二空间；其中，所述目标媒体片段包括所述第一媒体片段，和/或，所述第二媒体片段。本实施方式可将音频片段和帧图并行写入至第一存储空间中各自对应的存储空间，并且，在写入音频片段时，按照音频片段的输出顺序依次写入至第一空间，在写入图像数据时，按照帧图的输出顺序依次写入至第二空间，以实现图像和音频数据的并行写入。
根据第一方面，或者以上第一方面的任意一种实现方式，所述第一存储空间包括：第三空间和第四空间，其中，所述第三空间用于缓存所述电子设备在第一窗口输出的第四媒体数据，所述第四空间用于缓存所述电子设备在第二窗口输出的第五媒体数据；所述方法还包括：在输出所述第四媒体数据之前，将所述第四媒体数据缓存至所述第三空间；将所述第四媒体数据显示在所述第一窗口；在输出所述第五媒体数据之前，将所述第五媒体数据缓存至所述第四空间；将所述第五媒体数据显示在所述第二窗口。
示例性的，目标媒体数据可包括第四媒体数据和第五媒体数据。其中，第四媒体数据和第五媒体数据不同。
示例性的，第四媒体数据、第五媒体数据，与上述第一媒体数据、第二媒体数据、第三媒体数据之间的关系不做限制，第四媒体数据可以与第一媒体数据，或第二媒体数据，或第三媒体数据中的任意一个媒体数据相同或不同。第五媒体数据可以与第一媒体数据，或第二媒体数据，或第三媒体数据中的任意一个媒体数据相同或不同。
示例性的，第一空间可包括对应于窗口1的空间a1和/或对应于窗口2的空间b1；
第二空间可包括对应于窗口1的空间a2和/或对应于窗口2的空间b2；
所述第三空间包括空间a1和/或所述空间a2；
所述第四空间包括空间b1和/或空间b2。
此外，对于第一存储空间可缓存的数据对应的窗口的数量不限于上述两个，可以是两个以上。
本实施方式在应用到多窗口输出媒体数据的场景时，该场景可包括但不限于：分屏场景、悬浮窗场景、画中画场景等。
本实施方式中，可对电子设备的不同窗口内输出的媒体数据，在媒体数据输出之前，将媒体数据缓存至相应窗口对应的存储空间，以实现对不同窗口的待输出的媒体数据的并行写入第一存储空间。
此外，与多窗口的场景类似，本实施方式的方法还可应用到多个显示屏输出媒体数据的场景，多个显示屏的场景可包括但不限于双面屏场景、折叠屏场景等。那么第一存 储空间中可包括对应于每个显示屏的存储空间。
根据第一方面，或者以上第一方面的任意一种实现方式，所述将所述第四媒体数据显示在所述第一窗口之后，所述方法还包括：响应于在第四时刻接收到的、对所述第一窗口的第二用户操作，从所述第三空间获取第六媒体数据；其中，所述第四媒体数据为在第五时刻与所述第四时刻之间缓存的媒体数据，所述第六媒体数据为在第六时刻与所述第四时刻之间缓存的媒体数据，所述第五时刻位于所述第四时刻之前，所述第六时刻位于所述第四时刻之前；其中，所述第六时刻与所述第五时刻相同，或，所述第五时刻位于所述第六时刻之前；基于所述第六媒体数据，输出第二媒体信息。
本实施方式的原理与第一方面的实施方式的原理类似，这里不再赘述。这里是针对某个窗口内显示的媒体数据的播放进度时间点及该时间点之前的媒体数据的获取，并基于获取的媒体数据来输出媒体信息的方案。
根据第一方面，或者以上第一方面的任意一种实现方式，所述将所述第五媒体数据显示在所述第二窗口之后，所述方法还包括：响应于在第七时刻接收到的、对所述第二窗口的第三用户操作，从所述第四空间获取第七媒体数据；其中，所述第五媒体数据为在第八时刻与所述第七时刻之间缓存的媒体数据，所述第七媒体数据为在第九时刻与所述第七时刻之间缓存的媒体数据，所述第八时刻位于所述七时刻之前，所述第九时刻位于所述第七时刻之前；其中，所述第八时刻与所述第九时刻相同，或，所述第八时刻位于所述第九时刻之前；基于所述第七媒体数据，输出第三媒体信息。
本实施方式的原理与第一方面的实施方式的原理类似，这里不再赘述。这里是针对某个窗口内显示的媒体数据的播放进度时间点及该时间点之前的媒体数据的获取，并基于获取的媒体数据来输出媒体信息的方案。
在多窗口场景下，电子设备可对任意窗口输出的媒体数据，在用户触发输入时，获取该媒体数据的播放进度的时间点及该时间点之前的媒体数据，以用于媒体信息输出，可在多窗口场景下，对任意窗口输出的媒体数据实现媒体信息的输出。
根据第一方面，或者以上第一方面的任意一种实现方式，所述方法还包括：响应于接收到的第四用户操作，设置或调整目标空间的大小；其中，所述目标空间包括以下至少之一：所述第一空间、所述第二空间、所述第三空间、所述第四空间。
示例性的，第一存储空间的总大小由电子设备出厂时配置。用户可对第一存储空间的大小进行设置或调整，其中，设置或调整后的第一存储空间的大小需要小于或等于该总大小。
示例性的，用户可对第一存储空间中的第一空间、第二空间、第三空间、第四空间中的任意一个空间进行空间大小的设置或调整，以实现用户对缓存的媒体信息的长度的自定义设置，从而可根据实际需要而灵活调整缓存的数据长度。
根据第一方面，或者以上第一方面的任意一种实现方式，所述任意媒体数据的媒体类型包括以下至少之一：图像数据、音频数据、视频数据。本实施方式可支持图像、音频、视频等至少一种数据的前向缓存，以高效的输出媒体数据。
根据第一方面，或者以上第一方面的任意一种实现方式，所述方法还包括：响应于 接收到的第五用户操作，清空所述第一存储空间；其中，所述第五用户操作用于表示对所述电子设备的开机操作、或关机操作、或重启操作。本实施方式，可在电子设备开机、关键或重启操作时，清空第一存储空间的数据，以便于基于用户最新播放的媒体数据进行媒体数据的缓存。
第二方面，本申请实施例提供一种电子设备。该电子设备包括：存储器和处理器，所述存储器和所述处理器耦合；所述存储器存储有程序指令，所述程序指令由所述处理器执行时，使得所述电子设备执行如第一方面以及第一方面的任意一种实现方式中的方法。
第二方面所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第三方面，本申请实施例提供了一种计算机可读介质，用于存储计算机程序，当所述计算机程序在电子设备上运行时，使得所述电子设备执行如第一方面以及第一方面的任意一种实施方式中的方法。
第三方面所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第四方面，本申请实施例提供了一种包含指令的计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如第一方面以及第一方面的任意一种实施方式中的方法。
第四方面所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第五方面，本申请实施例提供了一种计算机可读存储介质，包括计算机程序，当所述计算机程序在电子设备上运行时，使得所述电子设备执行如第一方面以及第一方面的任意一种实施方式中的方法。
第五方面所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第六方面，本申请实施例提供了一种芯片，该芯片包括一个或多个接口电路和一个或多个处理器；所述接口电路用于从电子设备的存储器接收信号，并向所述处理器发送所述信号，所述信号包括存储器中存储的计算机指令；当所述处理器执行所述计算机指令时，使得所述电子设备执行如第一方面以及第一方面的任意一种实施方式中的方法。
第六方面所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对本申请实施例的描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为示例性示出的电子设备的结构示意图之一；
图2为示例性示出的电子设备的软件结构示意图；
图3为示例性示出的一种电子设备的获取媒体信息的应用场景示意图；
图4为示例性示出的一种电子设备的获取媒体信息的应用场景示意图；
图5a为示例性示出的电子设备内模块交互示意图；
图5b为示例性的示出的电子设备内部模块的示意图；
图5c为示例性的示出的电子设备内部模块的示意图；
图6a为示例性的示出的电子设备的一种应用场景的示意图；
图6b为示例性的示出的电子设备的一种应用场景的播放进度条的示意图；
图7为示例性的示出的电子设备的一种应用场景的播放进度条的变化示意图；
图8为示例性的示出的电子设备的缓存数据的过程示意图；
图9a为示例性的示出的电子设备的缓存数据的过程示意图；
图9b为示例性的示出的电子设备的缓存数据的过程示意图；
图10为示例性的示出的电子设备获取媒体信息的场景示意图；
图11为示例性的示出的电子设备的一种应用场景的示意图；
图12为示例性的示出的电子设备的一种应用场景的示意图；
图13为示例性的示出的电子设备的一种应用场景的示意图；
图14为本申请实施例提供的装置的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。
本申请实施例的说明书和权利要求书中的术语“第一”和“第二”等是用于区别不同的对象，而不是用于描述对象的特定顺序。例如，第一目标对象和第二目标对象等是用于区别不同的目标对象，而不是用于描述目标对象的特定顺序。
在本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
在本申请实施例的描述中，除非另有说明，“多个”的含义是指两个或两个以上。例如，多个处理单元是指两个或两个以上的处理单元；多个系统是指两个或两个以上的系统。
图1示出了电子设备100的结构示意图。应该理解的是，图1所示电子设备100仅是电子设备的一个范例，可选地，电子设备100可以为终端，也可以称为终端设备，终端可以为蜂窝电话(cellular phone)，平板电脑(pad)、可穿戴设备或物联网设备等具有音视频播放功能的设备，本申请不做限定。需要说明的是，电子设备100可以具有比图中所示的更多的或者更少的部件，可以组合两个或多个的部件，或者可以具有不同的部件配置。图1中所示出的各种部件可以在包括一个或多个信号处理和/或专用集成电路在内的硬件、软件、或硬件和软件的组合中实现。
电子设备100可以包括：处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，存储器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
其中，控制器可以是电子设备100的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。
I2C接口是一种双向同步串行总线，包括一根串行数据线(serial data line，SDA)和一根串行时钟线(derail clock line，SCL)。在一些实施例中，处理器110可以包含多组I2C 总线。处理器110可以通过不同的I2C总线接口分别耦合触摸传感器180K，充电器，闪光灯，摄像头193等。例如：处理器110可以通过I2C接口耦合触摸传感器180K，使处理器110与触摸传感器180K通过I2C总线接口通信，实现电子设备100的触摸功能。
I2S接口可以用于音频通信。在一些实施例中，处理器110可以包含多组I2S总线。处理器110可以通过I2S总线与音频模块170耦合，实现处理器110与音频模块170之间的通信。在一些实施例中，音频模块170可以通过I2S接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。
PCM接口也可以用于音频通信，将模拟信号抽样，量化和编码。在一些实施例中，音频模块170与无线通信模块160可以通过PCM总线接口耦合。在一些实施例中，音频模块170也可以通过PCM接口向无线通信模块160传递音频信号，实现通过蓝牙耳机接听电话的功能。所述I2S接口和所述PCM接口都可以用于音频通信。
UART接口是一种通用串行数据总线，用于异步通信。该总线可以为双向通信总线。它将要传输的数据在串行通信与并行通信之间转换。在一些实施例中，UART接口通常被用于连接处理器110与无线通信模块160。例如：处理器110通过UART接口与无线通信模块160中的蓝牙模块通信，实现蓝牙功能。在一些实施例中，音频模块170可以通过UART接口向无线通信模块160传递音频信号，实现通过蓝牙耳机播放音乐的功能。
MIPI接口可以被用于连接处理器110与显示屏194，摄像头193等外围器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface，DSI)等。在一些实施例中，处理器110和摄像头193通过CSI接口通信，实现电子设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信，实现电子设备100的显示功能。
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数据信号。在一些实施例中，GPIO接口可以用于连接处理器110与摄像头193，显示屏194，无线通信模块160，音频模块170，传感器模块180等。GPIO接口还可以被配置为I2C接口，I2S接口，UART接口，MIPI接口等。
USB接口130是符合USB标准规范的接口，具体可以是Mini USB接口，Micro USB接口，USB Type C接口等。USB接口130可以用于连接充电器为电子设备100充电，也可以用于电子设备100与外围设备之间传输数据。也可以用于连接耳机，通过耳机播放音频。该接口还可以用于连接其他电子设备，例如AR设备等。
可以理解的是，本申请实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
充电管理模块140用于从充电器接收充电输入。其中，充电器可以是无线充电器，也可以是有线充电器。在一些有线充电的实施例中，充电管理模块140可以通过USB接口130接收有线充电器的充电输入。在一些无线充电的实施例中，充电管理模块140可以通过电子设备100的无线充电线圈接收无线充电输入。充电管理模块140为电池142充电的同时，还可以通过电源管理模块141为电子设备供电。
电源管理模块141用于连接电池142，充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110，内部存储器121，外部存储器，显示屏194，摄像头193，和无线通信模块160等供电。电源管理模块141还可以用于监测电池容量，电池循环次数，电池健康状态(漏电，阻抗)等参数。在其他一些实施例中，电源管理模块141也可以设置于处理器110中。在另一些实施例中，电源管理模块141和充电管理模块140也可以设置于同一个器件中。
电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块150可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中，移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。
调制解调处理器可以包括调制器和解调器。其中，调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后，被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A，受话器170B等)输出声音信号，或通过显示屏194显示图像或视频。在一些实施例中，调制解调处理器可以是独立的器件。在另一些实施例中，调制解调处理器可以独立于处理器110，与移动通信模块150或其他功能模块设置在同一个器件中。
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。
在一些实施例中，电子设备100的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备100可以通过无线通信技术与网络以及其他设备通信。
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。
电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令，从而执行电子设备100的各种功能应用以及数据处理。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。
电子设备100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。
音频模块170用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中，音频模块170可以设置于处理器110中，或将音频模块170的部分功能模块设置于处理器110中。
扬声器170A，也称“喇叭”，用于将音频电信号转换为声音信号。电子设备100可以通过扬声器170A收听音乐，或收听免提通话。
受话器170B，也称“听筒”，用于将音频电信号转换成声音信号。当电子设备100接听电话或语音信息时，可以通过将受话器170B靠近人耳接听语音。
麦克风170C，也称“话筒”，“传声器”，用于将声音信号转换为电信号。当拨打电话或发送语音信息时，用户可以通过人嘴靠近麦克风170C发声，将声音信号输入到麦克风170C。电子设备100可以设置至少一个麦克风170C。在另一些实施例中，电子设备100 可以设置两个麦克风170C，除了采集声音信号，还可以实现降噪功能。在另一些实施例中，电子设备100还可以设置三个，四个或更多麦克风170C，实现采集声音信号，降噪，还可以识别声音来源，实现定向录音功能等。
耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130，也可以是3.5mm的开放移动电子设备平台(open mobile terminal platform，OMTP)标准接口，美国蜂窝电信工业协会(cellular telecommunications industry association of the USA，CTIA)标准接口。
按键190包括开机键，音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备100可以接收按键输入，产生与电子设备100的用户设置以及功能控制有关的键信号输入。
马达191可以产生振动提示。马达191可以用于来电振动提示，也可以用于触摸振动反馈。例如，作用于不同应用(例如拍照，音频播放等)的触摸操作，可以对应不同的振动反馈效果。作用于显示屏194不同区域的触摸操作，马达191也可对应不同的振动反馈效果。不同的应用场景(例如：时间提醒，接收信息，闹钟，游戏等)也可以对应不同的振动反馈效果。触摸振动反馈效果还可以支持自定义。
指示器192可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。
SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195，或从SIM卡接口195拔出，实现和电子设备100的接触和分离。电子设备100可以支持1个或N个SIM卡接口，N为大于1的正整数。SIM卡接口195可以支持Nano SIM卡，Micro SIM卡，SIM卡等。同一个SIM卡接口195可以同时插入多张卡。所述多张卡的类型可以相同，也可以不同。SIM卡接口195也可以兼容不同类型的SIM卡。SIM卡接口195也可以兼容外部存储卡。电子设备100通过SIM卡和网络交互，实现通话以及数据通信等功能。在一些实施例中，电子设备100采用eSIM，即：嵌入式SIM卡。eSIM卡可以嵌在电子设备100中，不能和电子设备100分离。
电子设备100的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本申请实施例以分层架构的Android系统为例，示例性说明电子设备100的软件结构。
图2是本申请实施例的电子设备100的软件结构框图。
电子设备100的分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为四层，从上至下分别为应用程序层，应用程序框架层，安卓运行时(Android runtime)和系统库，以及内核层。
应用程序层可以包括一系列应用程序包。
如图2所示，应用程序包可以包括相机，图库，日历，通话，地图，导航，WLAN，蓝牙，音乐，视频，短信息等应用程序。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。
如图2所示，应用程序框架层可以包括窗口管理器，内容提供器，视图系统，电话管理器，资源管理器，通知管理器等。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。
内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。所述数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。
视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。
电话管理器用于提供电子设备100的通信功能。例如通话状态的管理(包括接通，挂断等)。
资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。
通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。
系统库与运行时层包括系统库和安卓运行时(Android Runtime)。系统库可以包括多个功能模块。例如：表面管理器(surface manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，2D图形引擎(例如：SGL)等。3D图形库用于实现三维图形绘图，图像渲染，合成和图层处理等。安卓运行时包括核心库和虚拟机。安卓运行时负责安卓系统的调度和管理。核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。
三维图形处理库用于实现三维图形绘图，图像渲染，合成，和图层处理等。
二维图形引擎是二维绘图的绘图引擎。
媒体框架模块(Media Framework)，可为系统级的媒体数据(包括但不限于图像、音频、视频等)的输出通道。即系统中的媒体数据在输出时，均要通过媒体框架模块输出。
媒体框架模块，可用于在媒体数据输出之前，获取待输出的媒体数据，并将该媒体数据输出。
其中，该媒体数据可包括但不限于：静态图像、音频、视频等。
可选地，媒体框架模块可用于在媒体数据输出之前，对媒体数据进行最后一次的编 码(包括但不限于音频编码、图像渲染等)处理，得到可输出的媒体数据，然后，将编码处理后的媒体数据输出。
可选地，媒体框架模块，还可用于控制媒体数据的启动播放、暂停播放、快进播放、快退播放等媒体数据的播放控制操作。
媒体框架模块支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体框架模块可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。
此外，媒体数据在传递至媒体框架模块之前，可经由上述三维图像处理库,和/或，上述二维图形引擎等模块的编码处理，本申请对此不做限制。
内核层是硬件和软件之间的层。内核层至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动。
媒体框架模块可将媒体数据输出到内核层的显示驱动，以将媒体数据显示在显示屏，和/或，将媒体数据输出到内核层的音频驱动，由音频驱动将媒体数据输出到图1所示的音频模块170，以播放媒体数据。
可以理解的是，图2示出的系统框架层、系统库与运行时层包含的部件，并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。
目前，随着电子设备的飞速发展，电子设备可提供的功能越来越丰富。目前的电子设备可对电子设备正在播放的媒体数据(例如音视频数据)，获取媒体信息并输出。例如对播放的广告、新闻等媒体数据，输出媒体数据中的背景音乐的信息。再如对播放的视频，输出视频名称或视频画面中演职员等信息。
示例性的，图3示例性的示出了一种电子设备的获取媒体信息的应用场景示意图。
示例性的，如图3(1)所示，以电子设备为手机为例，手机的显示界面101为新闻应用的显示界面，显示界面101包括一个或多个控件，控件包括但不限于：“关注”控件、“热点”控件、“时事”控件，上述三个控件被用户点击后，手机可显示相应的新闻内容。此外，显示界面101还包括视频播放窗口102。该视频播放窗口102包括视频的播放进度条113，播放进度条113包括播放暂停控件1021，播放进度控件1024、当前播放时间控件1022、视频时长控件1023。示例性的，播放暂停控件1021在触发后用于对视频播放窗口102内播放的视频进行暂停或播放操作。示例性的，播放时长控件1023用于表示视频播放窗口102内播放的视频的总时长(这里为2分30秒)。当前播放时间控件1022用于表示视频播放窗口102内播放的视频的当前播放进度，在上述总时长中对应的播放时间点(这里为2分钟)。
如图3(1)所示，在视频播放窗口102内的视频播放至2分钟时，用户对该视频的背景音乐产生了兴趣，希望获取到音乐的名称。但是图3(1)所示的显示界面101对应的新闻应用并不具备“后台听歌识曲”的功能，用户可在显示界面101的底部，从下向上按照黑色箭头方向进行滑动，手机可响应于该用户操作，将手机的显示界面从图3(1)的显示界面101切换为图3(2)所示的主界面103，并将上述新闻应用切换到后台运行。示例性 的，在新闻应用处于后台运行状态后，视频播放窗口102内的视频继续播放。
如图3(2)所示，主界面103可包括多个应用的应用图标，用户点击音乐应用图标104，手机可响应于该用户操作，将显示界面切换为图3(3)所示的显示界面105，显示界面105包括一个或多个控件，该控件包括但不限于音乐播放控件106，后台听歌识曲控件107。如图3(3)，用户点击后台听歌识曲控件107，手机响应于该用户操作，将显示界面切换为图3(4)所示的显示界面108。
如图3(4)所示，显示界面108包括一个或多个控件，该控件包括识曲控件110，停止识别控件109。其中，识曲控件110用于表示手机正在采集手机正在播放的音频数据，并对音频数据识别歌曲名称的操作。停止识别控件109在被用户点击后，手机可停止对手机当前正在播放的音频数据的采集，并停止识别歌曲名称的操作。
示例性的，手机当前正在播放的音频数据，为图3(1)对应的新闻应用正在播放的视频的背景音乐。如图3(3)所示，在用户点击识曲控件110后，图3(3)对应的音乐应用才开始采集手机正在播放的音频数据，而如图3(1)的当前播放时间控件1022所示，在用户点击识曲控件110时，视频播放进度已经在视频的2分钟之后的某个时间点，而如图3(1)的播放时长控件1023所示，该视频的总时长只有2分钟30秒。那么音乐应用可采集的音频数据只有接近视频结束的不到30s(秒)的音频片段，使得采集的音频数据的时长较短而识别失败。从图3(4)到图3(5)，在识别一段时间后，手机的显示界面从显示界面108切换为显示界面111。显示界面111包括一个或多个控件，该控件包括识别结果控件112。识别结果控件112显示文本内容为“未匹配到结果”，即歌曲名称识别失败。
通过图3的过程可以发现，在用户触发“后台听歌识曲”功能(这里为点击图3(3)中的听歌识曲控件107)之后，手机才开始采集音频数据以用于歌曲名称的识别。如上述示例所述，用户触发“后台听歌识曲”功能时，音频已经接近播放结束，那么手机采集的音频数据的时长较短，容易造成识别歌曲名称失败。
此外，用户触发“后台听歌识曲”功能时，必然是用户听到了想要识别的媒体内容才会去主动触发(例如用户听到了图3(1)所示的视频的2分钟这一时刻的音频)，但是，手机在采集音频时只可以采集到2分钟之后的音频，从而错过了用户请求进行歌曲名称识别的音频片段，也易造成对歌曲名称的识别错误或识别失败的问题。
另外，从图3(4)变化到图3(5),需要经过一段时间，即手机需要采集足够时长的音频才可进行歌曲名称的识别，而给出识别结果，容易造成媒体信息的识别时间较长，用户体验较差的问题。需要说明的是，图3只是一种识别歌曲的交互示例，还可通过其他的交互方式来调出听歌识曲的界面。
示例性的，图4示例性的示出了一种电子设备的获取媒体信息的应用场景示意图。
如图4(1)所示，手机的显示界面201为视频播放界面，显示界面201包括一个或多个控件，该控件包括播放画面202、播放进度条203、视频信息获取控件205。示例性的，播放进度条203内的各控件与图3(1)中的播放进度条113内的控件的功能类似，具体可参照图3的介绍，这里不再赘述。
如图4(1)中的视频时长控件1023所示，视频总时长为50分钟，视频的当前播放进 度为18分钟，在视频播放至18分钟时，用户点击视频信息获取控件205。手机响应于该用户操作，对用户点击视频信息获取控件205这一时刻，对应的当前正在播放的视频画面进行人物、场景等信息识别，得到演员名、电影名、背景音乐等媒体信息，输出给用户。示例性的，手机响应于用户点击视频信息获取控件205的操作，获取用户点击视频信息获取控件205这一时刻，对应的视频画面，对该视频画面识别人物，得到该人物的演员和角色信息，如图4(2)所示，手机可显示悬浮窗控件206，悬浮窗控件206内显示有对这一帧画面的媒体信息的识别结果：“演员1饰演角色1”。
手机在通过视频应用，来对该视频应用中正在播放的视频数据，进行媒体信息的识别时，只可以对用户触发“媒体信息获取”功能(例如图4(1)中用户点击视频信息获取控件205)这一时刻的当前帧画面，进行媒体信息的获取。而无法对在用户触发“媒体信息获取”功能时已经播放的视频信息(例如图4中已经播放的前18分钟的视频信息中的任意视频片段)，来识别媒体信息。
例如手机播放的是关于一场足球比赛的视频，在视频播放至球员进球的视频画面，但该视频画面在视频中持续的时长较短，进球画面一闪而过，在用户触发“媒体信息获取”功能这一时刻时，进球画面已经播放完成。那么传统的电子设备只可以获取到用户触发“媒体信息获取”功能这一时刻的视频画面，来进行媒体信息识别，从而无法实现对已经播放过的视频画面进行获取以及相关媒体信息的识别。
需要说明的是，在图4(1)与图4(2)中，相同的附图标记表示相同的对象，因此，未对各附图的附图标记做逐一解释说明，相互参考即可，这里不再赘述。
传统的电子设备在对播放的音视频数据获取媒体信息时，只可以在用户触发“获取媒体信息”功能时，开始采集触发该功能的时刻以及该时刻之后的音视频信息，来进行媒体信息的识别。那么手机采集音视频信息的时长较长，进而使得对媒体信息的识别时长较长，用户体验较差。此外，目前的电子设备无法即时、准确地捕捉到需要进行媒体信息识别的音视频片段，使得对媒体信息的识别准确率较低。
为此，本申请的电子设备在媒体框架模块可访问的内存中设置有一段缓存空间。媒体框架模块作为系统级的媒体数据(包括但不限于图像、音频、视频等)的输出通道，在电子设备输出任意媒体数据时，该媒体数据均要通过媒体框架模块输出。即，只要电子设备在播放音视频数据，那么音视频数据在播放之前首先需要经过该媒体框架模块进行编码处理，然后，才可以输出播放。那么媒体框架模块可在音视频数据输出之前，将待输出的音视频数据写入至该缓存空间，使得缓存空间可存储有电子设备当前播放的音视频数据的当前播放进度(包括当前播放进度的时间点)前一段时间的音视频内容。
那么当用户触发“媒体信息获取”功能后，媒体框架模块可将上述缓存空间中的缓存内容取出，以用于媒体信息识别。这里的缓存内容包括电子设备当前播放的音视频数据的当前播放进度前的一段音视频内容。手机可快速的获取到用户触发“媒体信息获取”功能时刻之前的一段音视频数据进行媒体信息的识别，能够实现对媒体信息的即时获取。且无需在用户触发“媒体信息获取”功能的时刻之后才开始采集该时刻之后的音视频内容，缩短了对媒体信息的识别时长。
下面结合场景，来对本申请的技术方案做详细阐述，示例性的，图5a示例性的示出了本申请的电子设备内各模块的交互示意图，可结合图5b、图5c、图6a至图9a、图9b来理解。
S101，媒体框架模块接收视频应用1的音视频流，并对音视频流进行相应处理，得到待输出的至少一个音视频片段。
示例性的，可参照图6a，以电子设备为手机为例，手机的显示界面300为图5a中应用程序层的视频应用1的显示界面。示例性的，该显示界面300可包括一个或多个控件。示例性的，该控件可包括但不限于：“精选”控件3031、“小视频”控件3032、“剧集”控件3033、“电影”控件3034、第一视频窗口控件3035、第二视频窗口控件3036、第三视频窗口控件3037。
示例性的，在用户点击“精选”控件3031后，手机可响应于该用户操作，在显示界面300内显示精选视频的相关视频窗口。在用户点击“小视频”控件3032后，手机可响应于该用户操作，在显示界面300内显示小视频的相关视频窗口。在用户点击“剧集”控件3033后，手机可响应于该用户操作，在显示界面300内显示热门的电视剧的相关视频窗口。在用户点击“电影”控件3034后，手机可响应于该用户操作，在显示界面300内显示电影的相关视频窗口。
示例性的，用户点击第三视频窗口控件3037，手机可响应于该用户操作，显示图6a(2)所示的显示界面401。显示界面401播放的视频内容可为第三视频窗口控件3037对应的《影片名1》的电影内容。
可选地，在用户点击图6a(1)中的上述第一视频窗口控件3035、或上述第二视频窗口控件3036时，手机的响应操作与对用户点击第三视频窗口控件3037的响应操作类似，这里不再赘述。同理，用户在继续点击上述相关视频窗口后，手机的响应操作也与用户点击第三视频窗口控件3037的响应操作类似，这里不再赘述。
示例性的，继续回到图5a，在用户点击图6a(1)中的第三视频窗口控件3037后，手机可响应于该操作触发上述S101，使得视频应用1可将待显示的音视频流发送至系统库中的媒体框架模块。
需要说明的是，如图2所示，应用程序层和系统库之间还可包括应用程序框架层，那么音视频流可经应用程序框架层发送至媒体框架模块。此外，系统库可包括至少一种对媒体数据进行编码处理的模块，那么媒体框架模块可作为最后一个对媒体数据进行编码的模块，媒体框架模块接收到的音视频流可预先经过系统库中其他模块的编码处理，本申请对此不做限制。
示例性的，结合图5a，图5b、图5c示例性的示出了媒体框架模块的示意图。
如图5b所示，媒体框架模块在接收到音视频流后，可对音视频流进行编码处理，得到待输出的至少一个音视频片段。
示例性的，该音视频流可为该手机当前待输出播放的音视频流。示例性的，媒体框架模块可对接收到的音视频流进行处理，可得到至少一个音视频片段。示例性的，音视频流经编码处理后可对应有多个音视频片段。
示例性的，如图5c所示，媒体框架模块在接收到音视频流后，可对音视频流中的音频数据进行音频编码处理，可得到至少一个音频片段。示例性的，媒体框架模块对音频数据进行编码时，编码单位的最小单位可为一个音频片段。
示例性的，如图5c所示，媒体框架模块可对音视频流中的图像数据进行图像编码处理(包括但不限于渲染处理等)，可得到至少一帧图。示例性的，媒体框架模块对图像数据进行编码时，编码单位的最小单位可为一帧图像。
示例性的，编码处理后的时间戳相同的帧图与音频片段可构成图5b所示的编码处理后的一个音视频片段。
S102，媒体框架模块将待输出的至少一个音视频片段写入内存中的存储空间A。
示例性的，存储空间A可为手机出厂时，系统在内存中对媒体框架模块分配的媒体数据的存储空间，该存储空间A可用于对待输出的媒体数据(例如音视频数据)进行缓存，即可缓存播放时间点之前的音视频数据。
示例性的，存储空间A的存储空间大小可在出厂时由系统配置。在实际使用中，用户可根据需求对该存储空间A的大小进行调整，例如可在系统设置界面内提供调整该存储空间A的大小的入口，以便于用户根据需求灵活调整存储空间A的大小，使得存储空间A可缓存的媒体数据的大小由用户控制，以满足用户的个性化媒体数据的缓存需求。
在S102之后，媒体框架模块执行S103和S104，以将编码后的至少一个音视频片段输出，其中，本申请对于S103和S104的执行顺序不做限制。也就是说，在至少一个音视频片段输出之前，媒体框架模块可对待输出的至少一个音视频片段进行缓存，使得手机缓存有图6a所示的显示界面内，当前播放视频的播放进度点(包括该播放进度点)前一段时间的音视频片段。
S103，媒体框架模块将至少一个音视频片段中的图像数据发送至显示驱动。
S104，媒体框架模块将至少一个音视频片段中的音频数据发送至音频驱动。
可选地，对于S103和S104，可参照图5b来理解，媒体框架模块对音视频流编码完成之后，可将编码处理后得到的至少一个音视频片段分别写入存储空间A和存储空间B，然后，将存储空间B中的至少一个音视频片段输出到音频驱动和显示驱动。
示例性的，媒体框架模块可用于在媒体数据输出之前，进行最后一次编码操作，那么为了提升音视频数据输出的流畅性，媒体框架模块可将编码后的音视频片段进行缓存，这里的缓存空间为存储空间B(可位于内存中)，待存储空间B中的缓存内容满足输出条件(例如存储空间B存满内容)时，则媒体框架模块可从存储空间B中读取全部的缓存数据(可包括多个编码后的音视频片段)，并一次性输出该多个编码后的音视频片段(即图5b所示的目标音视频片段)，在输出该缓存数据之后，则将存储空间B清空，以便于在存储空间B中，依次写入下一次编码后的每个音视频片段，以在存储空间B中缓存待输出的至少一个音视频片段。
需要说明的是，为了提升音视频数据的流畅性，避免音视频数据播放卡顿，存储空间B的空间大小可小于存储空间A的空间大小。其中，媒体框架模块每次缓存的数据量为每次刷新输出的音视频数据量，如果缓存的数据量较大，则缓存时长较长，影响音视 频的刷新率，容易造成视频播放画面卡顿，或音频卡顿的问题。而存储空间A中缓存的音视频数据为用于媒体信息识别的数据，那么为了媒体信息识别的准确率，可缓存相较于用于刷新输出的音视频数据，更多的音视频数据，使得存储空间A的空间大小大于存储空间B的空间大小。
示例性地，存储空间A也可以是传统方案中系统为媒体框架模块分配的存储空间B中的一部分，使得媒体框架模块可以复用传统的存储空间B，以在存储空间B中不仅缓存待输出的音视频数据，还可以缓存用于识别媒体信息的、位于当前播放时间点之前的音视频数据。但是，媒体框架模块每次输出音视频数据后，可将存储空间B中除存储空间A之外缓存的待输出的音视频数据清空。本实施例中，存储空间B可包括存储空间A，而在存储空间B中，用于缓存待输出的音视频数据的空间大小，小于，用于缓存识别媒体信息的音视频数据的空间大小。
示例性的，存储空间A和存储空间B可为手机在出厂时，系统为媒体框架模块分配两个单独的存储空间，两个存储空间的功能不同。存储空间B用于缓存每次输出的音视频数据，存储空间A用于缓存当前播放时间点(包括该当前播放时间点)之前的一段音视频数据，以用于媒体信息识别。存储空间A的空间大小可大于存储空间B的空间大小。通过单独设置存储空间A和存储空间B，可使存储空间A的空间大小不受存储空间B的大小的限制。为了便于说明，本申请各个实施例均以存储空间A和存储空间B为不同的两个存储空间为例进行说明，而在存储空间B包括存储空间A的场景下，方法同理，这里不再赘述。
可选地，对于S102至S104，可继续结合图5c来理解：
如图5c所示，存储空间A可包括用于存储音频的存储空间A1，和用于存储图像的存储空间A2。存储空间B可包括用于存储音频的存储空间B1，和用于存储图像的存储空间B2。
示例性的，如图5c所示，媒体框架模块对于接收到的音视频流，每对一个音频片段(其中，音频数据的最小编码单位为音频片段)编码完成，则将编码后的该音频片段分别写入存储空间A1和存储空间B1；同理，媒体框架模块对于接收到的音视频流，每对一帧图像编码完成，则将编码后的该帧图像分别写入存储空间A2和存储空间B2。使得存储空间A1和存储空间B1被依次写入至少一个音频片段，存储空间A2和存储空间B2被依次写入至少一帧图像。
示例性的，如图5c所示，当媒体框架模块检测到存储空间B1存满数据时，则将存储空间B1中的全部音频片段(这里为音频片段序列)读取出来。同理，当媒体框架模块检测到存储空间B2存满数据时，则将存储空间B2中的全部帧图(这里为帧图序列)读取出来。示例性的，当存储空间B1和存储空间B2均存满数据时，说明存储空间B的缓存内容满足预设条件。那么存储空间B1中的音频片段序列，与存储空间B2中的帧图序列，可构成一个待输出的目标音视频片段。示例性的，目标音视频片段可包括编码后的至少一个音视频片段。可选地，存储空间B1和存储空间B2可同时存满数据，本申请对此不做限制。
示例性的，如图5a所示，媒体框架模块可将从存储空间B1读取的音频片段序列发送至音频驱动，将从存储空间B2读取的帧图序列发送至显示驱动，以实现一个目标音视频片段的输出。
需要说明的是，输出的目标音视频片段中的音频片段序列与所述帧图序列是时间戳一一对应的两组序列数据，以实现一段视频流的准确输出。
示例性的，如图5c所示，在将目标音视频片段输出之后，则可清空存储空间B1和存储空间B2，从而在更新一次播放进度之后，利用存储空间B1和存储空间B2继续缓存下次待输出的目标音视频片段。
S105，显示驱动将图像数据输出至显示屏。
S106，音频驱动将音频数据输出至音频设备。
可以理解，S105和S106可以同时执行，也可以先后执行，本申请对S105和S106执行的先后顺序不做限定。
示例性的，音频设备可包括图1所示的音频模块170，以及包括扬声器170A或耳机接口170D中的一种。其中，音频模块170可将处理后的音频数据输出到扬声器170A或耳机接口170D，以实现视频中的音频输出。
示例性的，音频模块170用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中，音频模块170可以设置于处理器110中，或将音频模块170的部分功能模块设置于处理器110中。
这样，在S105和S106之后，手机的显示界面可从图6a(1)所示的显示界面300切换为图6a(2)所示的显示界面401，以实现视频数据的输出播放。
示例性的，如图6a(2)所示，显示界面401可包括一个或多个控件，控件包括但不限于：当前显示的视频画面407、播放进度条控件402、播放进度控件404、当前播放时间控件403，视频时长控件406、播放暂停控件405。
示例性的，用户在点击播放暂停控件405后，手机可响应于该用户操作，对显示界面401内播放的视频进行暂停播放或启动播放。示例性的，播放时长控件406用于表示显示界面401内播放的视频的总时长(这里为50分钟)。当前播放时间控件403用于表示显示界面401内播放的视频的当前播放进度，在上述总时长中对应的播放时间点(简称当前播放时间点，这里为0分0秒)。示例性的，用户在沿播放进度条控件402，拖动播放进度控件404时，手机可响应于用户操作，来调整显示界面401内播放的视频的播放进度。
在图6a的场景下，在用户点击第三视频窗口控件3037后，手机即可触发图5a中S101的执行。随着视频应用1不断的请求刷新视频流，如图5a、图5b、图5c所示，媒体框架模块可多次输出目标音视频片段，那么媒体框架模块每输出一次目标音视频片段，可对图6a(2)所示的显示界面401中播放的视频的播放进度(也即视频画面407)进行一次刷新，相当于更新一次播放进度。那么通过多次执行上述S101至S106，可实现对显示界面401内播放视频的多次播放进度的刷新。示例性的，经过多次视频内容的刷新，手 机的显示界面从图6a(2)经过一段时间(这里为10s)的视频内容刷新可切换为图6a(3)，使得视频播放进度(即当前播放时间)，从图6a(2)所示的当前播放时间控件403示出的0分0秒，逐渐变化为图6a(3)所示的当前播放时间控件403示出的0分10秒。同时，播放进度控件404在播放进度条控件405中所处的位置，也从图6a(2)所示的位置0逐渐变化为图6a(3)所示的位置1。
需要说明的是，触发手机执行S101的用户操作并不限于图6a(1)中用户点击第三视频窗口控件3037的操作，还可包括但不限于以下方式：
方式1：用户点击图6a(2)所示的播放暂停控件405，使得显示界面401中的视频从暂停状态切换为播放状态。示例性的，图6a(2)所示的播放暂停控件405中的双竖条图标，表示显示界面401内的视频处于播放状态，示例性的，可参考图12(2)所示的播放暂停控件405中的三角图标，可用于表示显示界面401内的视频处于暂停状态。那么在显示界面401中的视频处于播放状态时，则S101可继续执行，从而可继续接收到视频流。
可选地，在显示界面401中的视频处于暂停状态时，则S101可中断执行，同理，S101之后的步骤也中断执行。
方式2：用户可通过拖动图6a(2)中的播放进度控件404，来改变显示界面401内视频的播放进度，那么手机在检测到该用户操作后，同样可触发S101的执行。其中，播放显示控件405需显示图6a(2)所示的双竖条图标，以表示显示界面401内的视频处于播放状态。
需要说明的是，在上述实施例中，S101中的音视频流可包括视频流，和/或，音频流，那么应用场景可以是对图6a所示的播放的视频数据进行媒体信息识别，还可以是对播放的音频数据进行媒体信息识别，方法同理，这里不再赘述。示例性的，音频数据可包括但不限于音乐、录音等。
在本申请实施例中，媒体框架模块作为系统级的媒体数据的输出通道，本申请的电子设备对媒体框架模块配置有用于前向存储媒体数据的缓存区(例如存储空间A)，那么媒体框架模块可在输出媒体数据之前，将待输出的媒体数据缓存至该缓存区，从而不论电子设备在输出任意类型的媒体数据，媒体框架模块均可将该电子设备所输出的该媒体数据的当前播放进度的时间点(包括该时间点)前的一段媒体数据存储起来，可以实现对全局播放的媒体数据的前向缓存。在需要进行媒体信息识别时，只需要从该缓存区读取媒体数据即可，无需实时采集媒体数据，可缩短媒体数据的采集时长，提升媒体信息的识别速度，以及提升媒体信息的识别准确率和成功率。
在一种可能的实施方式中，下面结合图6a至图9a，来对图5a中S102提及的媒体框架模块将待输出的音视频片段写入内存中的存储空间A的具体过程做详细阐述。
图7示例性的示出了图6a中播放的名称为《影片名1》的电影的播放进度条控件402的变化示意图。图7示出了从图6a(2)变化为图6a(3)的过程中，播放进度条控件402的变化示意图，其中，从图6a(2)变化为图6a(3)，播放进度从0分0秒，变化为0分10秒。
结合于图7，图8示例性的示出了该电影的播放进度从t0变化为t1的过程中，存储 空间A的变化示意图。结合于图7，图9a示例性的示出了该电影的播放进度从t2变化为t3的过程中，存储空间A的变化示意图。
在介绍图7至图9a之前，对图7中涉及的t0、t1、t2、t3、tn、控件301，以及图8和图9a中的Pi帧进行解释：
t0表示视频的起始播放时间，即t0为0分0秒。
tn表示视频的总时长，即图6a所示的视频时长控件406示出的总时长为50分钟，那么tn＝50分钟。
t3表示图6a(3)中当前播放时间控件403示出的0分10秒。
t1、t2、t3分别为媒体框架模块三次刷新播放进度后，图6a中播放的视频的当前播放进度的时间点。
控件301对应的时间点表示图6a中视频的当前播放进度的时间点。
Pi帧表示媒体框架模块获取的待输出的第i个音视频片段。其中，如图5b所示，第i个音视频片段表示媒体框架模块编码处理后得到的至少一个音视频片段中的第i个音视频片段。如图5c所示，第i个音视频片段可包括编码处理后的时间戳相同的帧图以及音频片段。示例性的，i＝1,2,3,4,……300，需要说明的是，这只是举例，并不用于限制本申请。
请参照图6a,在手机的显示界面从图6a(2)所示的界面变化为图6a(3)所示的界面的过程中，播放进度条控件402发生的变化可参照图7，存储空间A发生的变化可参照图8和图9a。
如图7(1)和图8(1)所示，请参照图6a(1)，在用户点击第三视频窗口控件3037时，手机可响应于该用户操作，触发执行图5a中的S101，那么在用户点击第三视频窗口控件3037来显示图6a(2)所示的显示界面401时，如图6a(2)所示的当前播放时间控件403，上述电影的当前播放进度的时间点为t0(即0分0秒)。
可选地，图6a中的手机在开机后，可进行初始化操作，手机在初始化时，可将S102提及的存储空间A中的内容清空，使得手机开机后存储空间A无缓存内容，以便于缓存手机开机后，媒体框架模块可在存储空间A缓存本次开机后当前播放时间点及该时间点之前的音视频内容。
请参照图6a(1)，在用户点击第三视频窗口控件3037之后，手机可响应于该用户操作，触发执行图5a所示的流程，那么如图8(2)至图8(5),媒体框架模块对接收的视频流进行编码时，每编码得到一个音视频片段，就可以将该音视频片段写入至存储空间A。示例性的，媒体框架模块依次写入P1帧、P2帧、……P100帧至存储空间A。同理，可将该100个音视频片段写入至图5b所示的存储空间B。需要说明的是，本申请对于各音视频片段在存储空间A中所处的存储位置不做限制。
例如媒体框架模块每编码得到100个音视频片段，则输出一次目标音视频片段(即这里的100个音视频片段)。那么媒体框架模块可将存储空间B中的上述P1帧、P2帧、……P100帧一次性读取出来，并按照时间顺序构成一个完整的目标音视频片段并输出，使得图6a(2)所示的显示界面401中的播放内容的播放进度得到首次刷新，如图 7(2)所示，播放进度的时间点从t0更新为t1。那么在手机当前播放的视频的播放进度首次刷新后，存储空间A中缓存了当前播放进度的时间点(即t1)以及该时间点之前(t0至t1时间段)的视频内容，即上述P1帧、P2帧、……P100帧。
然后，如图5a所示，媒体框架模块继续接收视频应用1的视频流，同理，手机可对图6a(2)所示的显示界面401的播放内容的播放进度进行二次刷新，如图7(2)和图7(3)所示，播放进度的时间点从t1更新为t2。在播放进度的时间点从t1更新为t2的过程中，媒体框架模块向存储空间A继续依次写入了P101帧至P200帧。那么如图9a(1)所示，在播放进度的时间点为t2时，存储空间A已缓存当前播放进度的时间点(即t2)以及该时间点之前(t0至t2时间段)的视频内容，即P1帧至P200帧。
示例性的，例如存储空间A的空间大小最多可写入200个音视频片段，如图5a所示，媒体框架模块继续接收视频应用1的视频流，那么如图9a(2)至图9a(4)所示,媒体框架模块对接收的视频流进行编码时，每编码得到一个音视频片段，就可以将该音视频片段写入至存储空间A。示例性的，媒体框架模块依次写入P201帧、P202帧、……P300帧至存储空间A。
示例性的，媒体框架模块在依次写入P201帧、P202帧、……P300帧至存储空间A时，由于在需要写入P201帧时，存储空间A已满，无剩余空闲空间。如图9a(2)所示，媒体框架模块可将最早写入的P1帧从存储空间A删除，然后，将P201帧写入存储空间A；同理，如图9a(3)所示，媒体框架模块在写入P202帧至存储空间A时，可将最早写入的P2帧从存储空间A删除，然后，将P202帧写入存储空间A。以此类推，媒体框架模块可依次写入P201帧、P202帧、……P300帧至存储空间A，同时，将P1帧至P100帧从存储空间A中依次删除，如图9a(4)所示，在当前播放进度的时间点为t3时，存储空间A的存储内容为P101帧至P300帧。
同理，媒体框架模块在二次刷新播放进度后，还可依次将P201帧至P300帧写入存储空间B，例如媒体框架模块每编码得到100个音视频片段，则输出一次目标音视频片段(即这里的P201帧至P300帧)。那么媒体框架模块可将存储空间B中的P201帧至P300帧一次性读取出来，并按照时间顺序构成一个完整的目标音视频片段并输出，使得图6a(2)所示的显示界面401中的播放内容的播放进度得到第三次刷新，如图7(3)和图7(4)所示，播放进度的时间点从t2更新为t3。如图9a(4)所示，在手机当前播放的视频的播放进度第三次刷新后，播放进度的时间点为t3，存储空间A中缓存了当前播放进度的时间点(即t3)以及该时间点之前(t1至t3时间段)的视频内容，即上述P101帧至P300帧。例如t3为图6a(3)所示的当前播放时间控件403示出的0分10s，则手机可经过3次播放进度的刷新，使得手机的显示界面从图6a(2)逐渐变化为图6a(3)，其中，在图6a(2)逐渐变化为图6a(3)的过程中，是视频动态播放的过程，这里图示未示出。
需要说明的是，媒体框架模块每次输出一个目标音视频片段，来实现一次视频的播放进度的更新时，该更新对用户而言是无感知的，每次刷新手机显示的播放内容的间隔是毫秒级、微妙级的，用户并无法感知。
还需要说明的是，上述示例，以存储空间A的空间大小最多可写入200个音视频片 段为例，以每次更新播放进度时，手机输出100个音视频片段为例进行说明，但是，本申请对于存储空间A的空间大小最多可缓存的音视频片段的数量不做限制，同理，对每次更新播放进度时，手机输出的音视频片段的数量也不做限制。具体可根据应用场景和实际需要而灵活设置。
那么只要在写入一个音视频片段至存储空间A时(不限于更新播放进度时)，媒体框架模块检测到存储空间A已存满数据，则媒体框架模块可按照图9a(2)和图9a(3)示意的方式先进行数据删除，再进行数据写入，以使存储空间A内缓存的数据得到动态更新。
可选地，在一些应用场景下，对于音视频数据的播放进度的更新不仅仅限于从图6a(2)变化到图6a(3)所示的“后向刷新”过程，还可包括“前向刷新”过程。
示例性的，可结合图6a，参考图6b,图6b示出了两个播放进度条的示意图。示例性的，上述“后向刷新”可表示为从图6b(1)变化为图6b(2)的过程。示例性的，播放进度控件404可从图6b(1)所示的位置2手动(即用户拖动播放进度控件404)或自动(即随着播放进度的自动更新而自动移动)的移动到图6b(2)所示的位置1，以实现该视频的当前播放进度从0分5秒直接或逐渐更新为0分10秒。
示例性的，上述“前向刷新”可表示为从图6b(2)变化为图6b(1)的过程，图9b示例性的示出了对于用户的“前向刷新”操作，媒体框架模块响应于该用户操作对存储空间A进行刷新的过程。
具体而言，参照图6b(2),用户可在播放进度条控件402上，按照箭头方向拖动播放进度控件404，使得播放进度控件404从图6b(2)所示的位置1移动到图6b(1)所示的位置2，以实现该视频的当前播放进度从0分10秒直接更新为0分5秒。示例性的，以t3为这里的0分10秒，t2为0分5秒为例，如图9b(1)所示，在播放进度的时间点为t3，即0分10秒时，存储空间A存储的数据为P101帧至P300帧。如图6b所示，用户将播放进度从位置1移动到位置2，使得播放进度的时间点从t3更新为t2，那么如图9b(2)所示，手机可响应于该用户操作，媒体框架模块将存储空间A中的P201帧至P300帧删除，以对存储空间A的内容进行刷新，如图9b(3)所示，播放进度从时间点t3“前向刷新”为时间点t2后，存储空间A的存储内容为P101帧至P200帧。如图9b(4)所示，在经过从图6b(2)到图6b(1)的“前向刷新”操作后，在播放进度的时间点为t2时，存储空间A的存储内容为时间点t1至时间点t2的视频片段。
在本申请实施例中，考虑到存储空间A的空间大小是有限的，为了确保存储空间A可即时地存储手机正在播放的音视频数据的当前播放进度的时间点，及该时间点之前的一段音视频数据。在存储空间A存满时，媒体框架模块可随着音视频的播放进度的“后向刷新”，实时地刷新存储空间A中存储的音视频内容，具体为删除一个最早写入的音视频片段，然后写入一个待输出的音视频片段，以实现存储空间A的内容的动态刷新。那么存储空间A中存储的音视频内容，始终为手机当前播放的视频的当前播放进度的时间点及该时间点之前的音视频数据，以便于手机可利用存储空间A中即时的音视频数据来进行媒体信息获取及识别等操作。
此外，在用户触发对播放进度的“前向刷新”操作时，例如从播放进度2(例如t3)前向刷新为播放进度1(例如t2)，则媒体框架模块可基于刷新后的当前播放进度1，来对存储空间A中对应于播放进度1至播放进度2之间的时间段内的音视频数据进行删除，使得存储空间A中始终存储当前播放进度的时间点(这里为t2)及该时间点之前的一段音视频数据，以便于手机可利用存储空间A中实时刷新后的即时的音视频数据来进行媒体信息获取及识别等操作。
那么本申请实施例，通过对媒体框架模块可访问的缓存区中设置前向存储的存储空间A，媒体框架模块不仅可在存储空间A存储手机的当前播放进度的时间点前的一段音视频数据，而且，媒体框架模块还可随着手机当前播放的音视频数据的播放进度的刷新(可以是后向刷新，也可以是前向刷新)，而动态刷新存储空间A的存储内容。使得存储空间A始终存储有当前播放进度的时间点(包括该时间点)前的一段音视频数据。这样，不论用户何时对当前播放的音视频数据触发“媒体信息获取”事件，媒体框架模块始终可以从该存储空间A获取到该音视频数据的当前播放进度的时间点(包括该时间点)前的音视频数据，以用于媒体信息识别，可进一步提高获取的媒体信息的准确率。
需要说明的是，上述图7至图9a、图9b中相同的附图标记表示相同的对象，因此，未对各附图的附图标记做逐一解释说明，上述各附图中未提及的附图标记可参照图7至图9a、图9b中已提及的同一附图标记的解释说明，这里不再赘述。
示例性的，可结合于图5a至图5c，以及图6a至图9a，图10和图11示例性的示出了电子设备获取媒体信息的场景示意图。
如图10所示，电子设备获取媒体信息的过程可包括如下步骤：
S601，媒体框架模块监听到预设事件。
示例性的，在手机启动之后，媒体框架模块完成初始化操作后，媒体框架模块可注册关于“媒体信息获取”事件的监听，在媒体框架模块注册该事件的监听后，可使手机系统具有检测到该“媒体信息获取”事件的能力，即不论用户的输入操作，是在应用的显示界面还是系统的显示界面触发了“媒体信息获取”事件，媒体框架模块都可监听到该事件。
示例性的，媒体框架模块在注册关于“媒体信息获取”事件的监听后，可持续或周期性地监听“媒体信息获取”事件是否被触发。
示例性的，如图6a(3)和图11所示，在手机显示图6a(3)所示的显示界面401后，如图11(1)所示，用户通过操作，手机可在显示界面401中显示有媒体信息获取控件408。用户点击媒体信息获取控件408，可触发“媒体信息获取”事件，那么媒体框架模块可监听到该“媒体信息获取”事件。如图11(1)所示，用户触发“媒体信息获取”事件时，视频的当前播放进度为当前播放时间控件403所示的0分10秒(即图7(4)的t3时刻)，那么存储空间A的存储内容可参照图9a(4)。
需要说明的是，不论手机的显示界面显示的音视频数据处于播放状态，还是暂停状态，当用户触发“媒体信息获取”事件时，触发该事件的时间点必然是该音视频数据的当前播放进度的时间点。那么手机的存储空间A内缓存有手机正在输出的音视频数据的 当前播放进度的时间点(包括该时间点)之前的目标音视频片段，手机可从存储空间A中读取已缓存的该目标音视频片段，来用于媒体信息识别和获取等操作，并将操作结果反馈给用户。从而可以即时、准确地捕捉到用于媒体信息识别的目标音视频段，从而提升媒体信息的识别成功率。
示例性的，用户触发“媒体信息获取”事件的时刻是用户从手机已输出的音视频数据中听到或看到感兴趣的内容，从而需要获取媒体信息。那么本申请的手机可通过媒体框架模块获取将该时刻及该时刻之前的音视频数据，以用于媒体信息识别，使得识别的媒体信息容易符合用户需求，并提升媒体信息的识别成功率。
S602，媒体框架模块在监听到预设事件后，可从存储空间A中读取音视频数据。
示例性的，如图7(4)所示，在触发该预设事件的t3时刻，存储空间A存储有t1至t3时间段内的音视频数据。
可选地，媒体框架模块可将存储空间A的全部缓存内容，即t1至t3时间段内的音视频数据读取出来，以用于媒体信息识别。
可选地，媒体框架模块还可将存储空间A中的部分缓存内容，例如该部分缓存内容可包括当前播放进度的时间点(t3的一个示例)以及该时间点之前预设时长(例如3s)的音视频内容。
需要说明的是，该预设时长可根据读取该音视频数据后的用途和场景的差异而灵活配置，本申请对此不做限制。
例如手机从存储空间A读取缓存的前向音视频数据后，可对该音视频数据处理来获取媒体信息，那么媒体信息的类型不同，则该预设时长可存在差异。例如对音乐识别歌曲名称的场景，则该预设时长可为5s至6s范围内的任意一个取值。
再如手机从存储空间A读取缓存的前向音视频数据，用于对该音视频数据进行回放(例如进球的精彩瞬间的视频片段)，则该预设时长可为2s，具体可根据需要灵活配置，本申请对此不做限制。
S603，媒体框架模块可将读取的音视频数据发送至视频应用1。
示例性的，媒体框架模块可将从存储空间A读取的音视频数据发送至视频应用1。
示例性的，视频应用1具备识别媒体信息的功能，那么视频应用1可对当前播放进进度的时间点及该时间点之前一段时间的音视频数据(例如图11中播放的视频的第8s至第10s的视频数据)，进行媒体信息识别。示例性的，视频应用1基于该最近播放的3s时长的视频数据进行背景音乐的名称的识别，可确定背景音乐名称为《乐曲1》，手机可响应于图11(1)中用户点击媒体信息获取控件408的操作，而立刻显示图11(2)所示的显示界面401，如图11(2)所示，显示界面401可包括悬浮窗控件409，悬浮窗控件409内可显示有媒体信息的识别结果，这里为“识别结果：背景音乐名称为《乐曲1》”。
在本申请实施例中，如上文所述，用于媒体信息识别的媒体数据已经在媒体数据输出之前，缓存至存储空间A，那么在用户触发媒体信息获取控件408时，媒体框架模块可立刻从存储空间A中读取到用于识别媒体信息的媒体数据，而无需实时采集该媒体数据，从而能够在图11(1)至图11(2)的切换上，时长较短，用户可无感知到该切换时长。 相比于传统技术中从图3(4)切换到图3(5)的过程中，需要等待较长时间，本申请用户等候对媒体信息的识别结果的时长更短。
需要说明的是，本申请的系统也可以集成识别媒体信息的功能，那么媒体框架模块可将读取的音视频数据发送至手机系统中配置的媒体信息识别模块，由该媒体信息识别模块对该音视频数据进行媒体信息识别，或者将该音视频数据作为回放的视频片段输出显示播放(具体方式可参照后续的图13的描述)，这里不再赘述。
示例性的，媒体信息识别模块在对音视频数据识别媒体信息时，可基于“音频指纹”等传统或未来开发的技术，来对音频数据识别歌曲名称、歌唱者信息、歌曲中的乐器等媒体信息。示例性的，媒体信息识别模块在对音视频数据识别媒体信息时，可基于视频数据中各帧图进行图像特征识别，来对剧名、演员名等媒体信息进行识别。当然，上述媒体信息仅仅作为举例，本申请的手机在对获取到的手机当前播放进度的时间点及该时间点之前的音视频数据，进行媒体信息识别时，本申请对于手机可识别的媒体信息的类型不做限制，可包括更多类型的媒体信息。同时，本申请对于利用一段媒体数据(例如音视频数据)来识别媒体信息的技术手段也不做限制，具体技术手段和所需要识别的媒体信息的类型可根据需要而灵活配置。
可选地，结合于上述任意实施方式，图12示例性的示出了本申请的电子设备的应用场景示意图。
参照图12(1)，手机的显示界面600包括一个或多个控件，该控件可包括但不限于：电量图标、网络图标、应用图标等。用户可点击显示界面600上的视频应用1的图标601，以启动视频应用1。图12(2)所示，手机可响应于用户的点击操作，启动视频应用1并显示视频应用1的应用界面602。应用界面602可包括一个或多个控件，该控件包括当前显示的视频画面607、播放进度条控件402、播放进度控件404、当前播放时间控件403，视频时长控件406、播放暂停控件405。
示例性的，如图12(2)中的播放暂停控件405所示，应用界面602中显示的视频处于暂停状态。在其他实施方式中，应用界面602中显示的视频也可以处于播放状态，本申请对此不做限制。
参照图12(2)，用户从右侧边缘(也可以是左侧边缘)向显示窗口中心滑动并停留。如图12(3)所示，手机可响应于该用户操作，在视频应用1的应用界面602的右边缘(也可以是左边缘)显示侧边栏603。参照图12(3)，侧边栏中包括一个或多个控件，例如包括电子邮件应用的图标、备忘录应用的图标、图库应用的图标、文件管理应用的图标、视频应用2的图标6031等，侧边栏还可以包括添加选项，该选项用于将指定的应用的图标添加至侧边栏。需要说明的是，图12(3)中的各应用的名称、数量及位置仅为示意性举例，本申请不做限定。
继续参照图12(3)，示例性的，用户可拖动侧边栏中的视频应用2的图标6031至手机显示窗口的下半部区域并松开，例如沿虚线箭头方向拖动图标6031。如图12(4)所示，手机可响应于该用户操作，将手机的显示窗口分屏，包括显示窗口608和显示窗口604，如图12(4)所示，示例性的，手机的显示窗口608用于显示视频应用1的应用界面，显示 窗口604用于显示视频应用2的应用界面。
示例性的，如图12(4)所示，用户可点击位于显示窗口608和显示窗口604内的播放暂停控件405，手机可响应于该用户操作，在显示窗口608和显示窗口604，分别显示视频应用1播放的视频和视频应用2播放的视频。
示例性的，媒体框架模块作为系统级的音视频数据的输出通道，在用户点击图12(4)中显示窗口608内的播放暂停控件405后，媒体框架模块可接收到视频应用1的视频流。在用户点击图12(4)中显示窗口604内的播放暂停控件405后，媒体框架模块可接收到视频应用2的视频流。媒体框架模块可对接收到的两组视频流进行并行处理，对两组视频流分别按照图5a或相关实施例的方式进行处理，使得存储空间A缓存有视频应用1的当前播放进度的时间点及该时间点之前第一预设时长的视频数据，以及缓存有视频应用2的当前播放进度的时间点及该时间点之前第二预设时长的视频数据。
示例性的，针对分屏的场景，手机同样可在系统界面向用户提供对分屏窗口设置存储空间A的大小的入口。示例性的，用户可在系统界面内对分屏场景下的第一窗口、第二窗口分别设置存储空间A3的大小和存储空间A4的大小，其中存储空间A包括存储空间A3和存储空间A4。示例性的，存储空间A3(包括存储音频的存储空间A3-1，和存储图像的存储空间A3-2)最多可存储第一预设时长(例如3s)的视频数据(包括音频和图像)，存储空间A4(包括存储音频的存储空间A4-1，和存储图像的存储空间A4-2)最多可存储第二预设时长(例如4s)的视频数据。第一预设时长和第二预设时长分别对应存储空间A3和存储空间A4的空间大小，那么第一预设时长和第二预设时长可由用户灵活配置，本申请不做限制，此外，第一预设时长与第二预设时长可相同或不同。
示例性的，在图12的场景下，图12(4)中首先形成的显示窗口608为第一窗口，后续形成的显示窗口604为第二窗口。那么在用户对图12(4)中两个显示窗口内的播放暂停控件405分别进行点击之后，媒体框架模块可在存储空间A3中不断更新写入视频应用1的当前播放进度时间点及该时间点之前的视频数据(3s时长的视频数据)，以及在存储空间A4中不断更新写入视频应用2的当前播放进度时间点及该时间点之前的视频数据(4s时长的的视频数据)。媒体框架模块写入视频数据到存储空间A3和存储空间A4的过程，与前述实施方式提及的对单个应用的音视频数据写入存储空间A的过程的原理类似，这里不再赘述，具体可参考图7至图9a的过程。
示例性的，在手机显示如图12(4)所示的界面之后的一段时间，显示窗口608和显示窗口604各自显示的视频的播放进度得到多次更新，手机的显示界面更新为图13(1)所示的界面。参照图13(1)，显示窗口608内当前播放时间控件403示出的当前播放进度的时间点t4为10分10秒，用户在所述时间点t4点击显示窗口608中的媒体信息获取控件408。如图13(2)所示，手机响应于该用户操作，从存储空间A3获取时间点t4及时间点t4之前的视频片段605(时长为3s)。
在一种可能的实施方式中，手机在显示该视频片段605时，手机的显示界面可从图13(1)切换为图13(2)，如图13(2)所示，在显示窗口609内对视频片段605进行播放，以实现对用户触发“媒体信息获取”事件的时刻之前的一段时长的视频片段的回放。
示例性的，视频片段605为视频应用1在显示窗口608内已播放的视频中第10分8秒至10分10秒的3s时长的视频片段。示例性的，如图13(2)中的显示窗口609所示，播放进度条控件404位于播放进度条控件402的起始位置，当前播放时间控件403示出的时间为0分0秒，视频时长控件406示出的视频片段时长为3s，播放暂停控件405示出的图标表示该视频片段605处于暂停状态。用户通过点击显示窗口609内的播放暂停控件405，手机可响应于该用户操作在显示窗口609内对视频片段605进行播放。
其中，在图13(2)中，显示窗口609播放的内容为视频片段605，即视频片段605作为图13(2)中的显示窗口609所播放的完整视频，因此，在图13(2)中当前播放时间控件403示出的时间为0分0秒，视频时长控件406示出的视频片段时长为3s，而非图13(1)中显示窗口608内的视频播放时长控件406所示出的50分钟。
示例性的，当用户需要退出对视频片段605的播放时，可对图13(2)中的显示窗口609触发预设操作，以将手机的显示窗口609的内容切换为图13(1)中的显示窗口608的内容。
在另一种可能的实施方式中，手机在从存储空间A3获取到视频片段605之后，可在图13(1)的显示窗口608内显示图13(2)所示的显示窗口609。示例性的，显示窗口609可以悬浮窗或小窗口等方式显示在图13(1)中的显示窗口608内。并且，在图13(1)的显示窗口608内显示图13(2)所示的显示窗口609后，图13(1)中显示窗口608中播放的视频可继续播放，而不受显示窗口609的影响，以避免因回放视频片段而影响用户对原视频的浏览。
需要说明的是，本申请对于显示窗口609在图13(1)中的显示界面中的显示位置并不限制为在显示窗口608内，可在图13(1)中的显示界面中的任意位置，用户也可以对显示窗口608的位置进行调节，本申请对此不做限制。
同理，参照图13(1)，显示窗口604内当前播放时间控件403示出的当前播放进度的时间点t5为10分0秒，用户在所述时间点t5点击显示窗口604中的媒体信息获取控件408。如图13(2)所示，手机响应于该用户操作，从存储空间A4获取时间点t5及时间点t5之前的视频片段606(时长为4s)。
在一种可能的实施方式中，手机在显示该视频片段606时，手机的显示界面可从图13(1)切换为图13(2)，如图13(2)所示，并在显示窗口610内对视频片段606进行播放，以实现对用户触发“媒体信息获取”事件的时刻之前的一段时长的视频片段的回放。
示例性的，视频片段606为视频应用2在显示窗口604内已播放的视频中第9分57秒至10分0秒的视频片段的4s时长的视频片段。示例性的，如图13(2)中的显示窗口610所示，播放进度条控件404位于播放进度条控件402的起始位置，当前播放时间控件403示出的时间为0分0秒，视频时长控件406示出的视频片段时长为4s，播放暂停控件405示出的图标表示该视频片段606处于暂停状态。用户通过点击显示窗口610内的播放暂停控件405，手机可响应于该用户操作在显示窗口610内对视频片段606进行播放。
其中，在图13(2)中，显示窗口610播放的内容为视频片段606，即视频片段606作为图13(2)中的显示窗口610所播放的完整视频，因此，在图13(2)中当前播放时 间控件403示出的时间为0分0秒，视频时长控件406示出的视频片段时长为4s，而非图13(1)中显示窗口604内的视频播放时长控件406所示出的50分钟。
示例性的，当用户需要退出对视频片段606的播放时，可对图13(2)中的显示窗口610触发预设操作，以将手机的显示窗口610的内容切换为图13(1)中的显示窗口604的内容。
在另一种可能的实施方式中，手机在从存储空间A4获取到视频片段606之后，可在图13(1)的显示窗口604内显示图13(2)所示的显示窗口610。示例性的，显示窗口610可以悬浮窗或小窗口等方式显示在图13(1)中的显示窗口604内。并且，在图13(1)的显示窗口604内显示图13(2)所示的显示窗口610后，图13(1)中显示窗口604中播放的视频可继续播放，而不受显示窗口610的影响，以避免因回放视频片段而影响用户对原视频的浏览。
需要说明的是，本申请对于显示窗口610在图13(1)中的显示界面中的显示位置并不限制为在显示窗口604内，可在图13(1)中的显示界面中的任意位置，用户也可以对显示窗口610在图13(1)中显示界面内的位置进行调节，本申请对此不做限制。
在本实施例中，从图13(1)切换到图13(2)的过程，与从图11(1)切换到图11(2)的过程类似，由于媒体框架模块可从存储空间A直接读取用于识别媒体信息的媒体数据，或用于回放的媒体数据，从而画面切换时长较短，用户无需等待较长时间来等待识别结果或回放结果，用户体验更好。
与分屏场景类似，在电子设备包括多个显示屏(至少两个显示屏)的场景下，用户同样可根据需要在系统设置界面，对存储空间A划分为针对不同显示屏的存储空间A5、存储空间A6等，用户可对应于不同显示屏的存储空间分别设置空间大小，那么媒体框架模块可将待输出至不同显示屏的音视频数据缓存至相应显示屏的存储空间中，以在用户在相应的显示屏上触发“媒体信息获取”事件时，手机可从相应显示屏的存储空间中，获取该显示屏显示的音视频数据中当前播放进度的时间点及该时间点之前的一段音视频数据，以用于媒体信息的识别或回放等处理。
在本申请实施例中，手机在分屏场景或多个显示屏场景下来同时播放至少两组媒体数据时，手机同样可在媒体框架模块可访问的缓存区内，缓存各显示窗口或各显示屏上当前播放进度前的一段音视频数据，从而在用户触发“媒体信息获取”事件时，媒体框架模块可从缓存区读取各显示窗口或各显示屏上当前播放进度前的一段音视频数据，以用于媒体信息识别或音视频数据回放等处理，能够在多窗口和多屏场景下，实现媒体信息的即时获取。
此外，在分屏和多个显示屏的场景下，不同显示窗口，或不同显示屏显示的媒体数据并不限于图12和图13所示出的视频数据，可以分别显示不同类型的媒体数据，例如在一个显示窗口内播放视频，在另一个显示窗口播放音乐。或者利用一个显示屏播放音乐，利用另一个显示屏播放录音等，本申请对此不做限制，方法的执行原理类似，这里不再赘述。
需要说明的是，上述图6a、图6b、图11至图13中相同的附图标记表示相同的对象， 因此，未对各附图的附图标记做逐一解释说明，上述各附图中未提及的附图标记可参照图6a、图6b、图11至图13中已提及的同一附图标记的解释说明，这里不再赘述。
可以理解的是，电子设备为了实现上述功能，其包含了执行各个功能相应的硬件和/或软件模块。结合本文中所公开的实施例描述的各示例的算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以结合实施例对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
一个示例中，图14示出了本申请实施例的一种装置300的示意性框图装置300可包括：处理器301和收发器/收发管脚302，可选地，还包括存储器303。
装置300的各个组件通过总线304耦合在一起，其中总线304除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图中将各种总线都称为总线304。
可选地，存储器303可以用于前述方法实施例中的指令。该处理器301可用于执行存储器303中的指令，并控制接收管脚接收信号，以及控制发送管脚发送信号。
装置300可以是上述方法实施例中的电子设备或电子设备的芯片。
其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
本实施例还提供一种计算机存储介质，该计算机存储介质中存储有计算机指令，当该计算机指令在电子设备上运行时，使得电子设备执行上述相关方法步骤实现上述实施例中的媒体信息的获取方法。
本实施例还提供了一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得计算机执行上述相关步骤，以实现上述实施例中的媒体信息的获取方法。
另外，本申请的实施例还提供一种装置，这个装置具体可以是芯片，组件或模块，该装置可包括相连的处理器和存储器；其中，存储器用于存储计算机执行指令，当装置运行时，处理器可执行存储器存储的计算机执行指令，以使芯片执行上述各方法实施例中的媒体信息的获取方法。
其中，本实施例提供的电子设备、计算机存储介质、计算机程序产品或芯片均用于执行上文所提供的对应的方法，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
通过以上实施方式的描述，所属领域的技术人员可以了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或 组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
本申请各个实施例的任意内容，以及同一实施例的任意内容，均可以自由组合。对上述内容的任意组合均在本申请的范围之内。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。
结合本申请实施例公开内容所描述的方法或者算法的步骤可以硬件的方式来实现，也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于随机存取存储器(Random Access Memory，RAM)、闪存、只读存储器(Read Only Memory，ROM)、可擦除可编程只读存储器(Erasable Programmable ROM，EPROM)、电可擦可编程只读存储器(Electrically EPROM，EEPROM)、寄存器、硬盘、移动硬盘、只读光盘(CD-ROM)或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。另外，该ASIC可以位于网络设备中。当然，处理器和存储介质也可以作为分立组件存在于网络设备中。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本申请实施例所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码 进行传输。计算机可读介质包括计算机存储介质和通信介质，其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。

Claims (16)

  1. 一种媒体信息的获取方法，其特征在于，应用于电子设备，所述方法包括：
    在输出第一媒体数据之前，将所述第一媒体数据缓存至第一存储空间；
    其中，所述第一存储空间用于缓存所述电子设备输出的任意媒体数据中的目标媒体数据，所述目标媒体数据为所述任意媒体数据中，位于当前播放进度的时间点及所述时间点之前的媒体数据；其中，所述目标媒体数据包括所述第一媒体数据；
    响应于在第一时刻接收到的第一用户操作，从所述第一存储空间获取第二媒体数据，其中，所述第一媒体数据为在第二时刻与所述第一时刻之间缓存的媒体数据，所述第二媒体数据为在第三时刻与所述第一时刻之间缓存的媒体数据，所述第二时刻位于所述第一时刻之前，所述第三时刻位于所述第一时刻之前；
    其中，所述第三时刻与所述第二时刻相同，或，所述第二时刻位于所述第三时刻之前；
    基于所述第二媒体数据，输出第一媒体信息。
  2. 根据权利要求1所述的方法，其特征在于，所述将所述第一媒体数据缓存至第一存储空间之后，所述方法还包括：
    在输出第三媒体数据之前，将所述第三媒体数据缓存至所述第一存储空间。
  3. 根据权利要求1或2所述的方法，其特征在于，所述第一媒体数据包括至少一个第一媒体片段，所述第三媒体数据包括至少一个第二媒体片段；
    所述将所述第一媒体数据缓存至到第一存储空间，包括：
    按照所述至少一个第一媒体片段的输出顺序，依次将所述至少一个第一媒体片段缓存至所述第一存储空间；
    所述将所述第三媒体数据缓存至所述第一存储空间，包括:
    按照所述至少一个第二媒体片段的输出顺序，依次将所述至少一个第二媒体片段缓存至所述第一存储空间。
  4. 根据权利要求3所述的方法，其特征在于，所述按照所述至少一个第二媒体片段的输出顺序，依次将所述至少一个第二媒体片段缓存至所述第一存储空间，包括：
    检测到所述第一存储空间中不存在空闲空间，按照缓存时间从早到晚的顺序，依次删除所述第一存储空间中已缓存的第一媒体片段，以及按照所述至少一个第二媒体片段的输出顺序，依次将所述至少一个第二媒体片段缓存至所述第一存储空间。
  5. 根据权利要求1至4中任意一项所述的方法，其特征在于，所述第一存储空间包括：用于缓存音频数据的第一空间和用于缓存图像数据的第二空间；
    将所述目标媒体数据缓存至第一存储空间，包括：
    将所述目标媒体数据中的音频数据缓存至所述第一空间；
    和/或，
    将所述目标媒体数据中的图像数据缓存至第二空间；
    其中，所述目标媒体数据包括所述第一媒体数据，和/或，所述第三媒体数据。
  6. 根据权利要求1至5中任意一项所述的方法，其特征在于，所述目标媒体数据包括至少一个目标媒体片段；
    依次将至少一个目标媒体片段缓存至所述第一存储空间，包括：
    依次将所述至少一个目标媒体片段中的至少一个音频片段，缓存至所述第一空间；
    和/或，
    依次将所述至少一个目标媒体片段中的至少一帧图像，缓存至所述第二空间；
    其中，所述目标媒体片段包括所述第一媒体片段，和/或，所述第二媒体片段。
  7. 根据权利要求1至6中任意一项所述的方法，其特征在于，所述第一存储空间包括：第三空间和第四空间，其中，所述第三空间用于缓存所述电子设备在第一窗口输出的第四媒体数据，所述第四空间用于缓存所述电子设备在第二窗口输出的第五媒体数据；
    所述方法还包括：
    在输出所述第四媒体数据之前，将所述第四媒体数据缓存至所述第三空间；
    将所述第四媒体数据显示在所述第一窗口；
    在输出所述第五媒体数据之前，将所述第五媒体数据缓存至所述第四空间；
    将所述第五媒体数据显示在所述第二窗口。
  8. 根据权利要求7所述的方法，其特征在于，
    所述将所述第四媒体数据显示在所述第一窗口之后，所述方法还包括：
    响应于在第四时刻接收到的、对所述第一窗口的第二用户操作，从所述第三空间获取第六媒体数据；
    其中，所述第四媒体数据为在第五时刻与所述第四时刻之间缓存的媒体数据，所述第六媒体数据为在第六时刻与所述第四时刻之间缓存的媒体数据，所述第五时刻位于所述第四时刻之前，所述第六时刻位于所述第四时刻之前；其中，所述第六时刻与所述第五时刻相同，或，所述第五时刻位于所述第六时刻之前；
    基于所述第六媒体数据，输出第二媒体信息。
  9. 根据权利要求7或8所述的方法，其特征在于，
    所述将所述第五媒体数据显示在所述第二窗口之后，所述方法还包括：
    响应于在第七时刻接收到的、对所述第二窗口的第三用户操作，从所述第四空间获取第七媒体数据；
    其中，所述第五媒体数据为在第八时刻与所述第七时刻之间缓存的媒体数据，所述第七媒体数据为在第九时刻与所述第七时刻之间缓存的媒体数据，所述第八时刻位于所述七时刻之前，所述第九时刻位于所述第七时刻之前；其中，所述第八时刻与所述第九 时刻相同，或，所述第八时刻位于所述第九时刻之前；
    基于所述第七媒体数据，输出第三媒体信息。
  10. 根据权利要求1至9中任意一项所述的方法，其特征在于，所述方法还包括：
    响应于接收到的第四用户操作，设置或调整目标空间的大小；
    其中，所述目标空间包括以下至少之一：所述第一空间、所述第二空间、所述第三空间、所述第四空间。
  11. 根据权利要求1至10中任意一项所述的方法，其特征在于，所述任意媒体数据的媒体类型包括以下至少之一：
    图像数据、音频数据、视频数据。
  12. 根据权利要求1至11中任意一项所述的方法，其特征在于，所述方法还包括：
    响应于接收到的第五用户操作，清空所述第一存储空间；
    其中，所述第五用户操作用于表示对所述电子设备的开机操作、或关机操作、或重启操作。
  13. 一种电子设备，其特征在于，包括：存储器和处理器，所述存储器和所述处理器耦合；所述存储器存储有程序指令，所述程序指令由所述处理器执行时，使得所述电子设备执行如权利要求1至12中任意一项所述的媒体信息的获取方法。
  14. 一种计算机可读存储介质，其特征在于，包括计算机程序，当所述计算机程序在电子设备上运行时，使得所述电子设备执行如权利要求1至12中任意一项所述的媒体信息的获取方法。
  15. 一种包含指令的计算机程序产品，其特征在于，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如权利要求1至12中任意一项所述的媒体信息的获取方法。
  16. 一种芯片，其特征在于，包括一个或多个接口电路和一个或多个处理器；所述接口电路用于从电子设备的存储器接收信号，并向所述处理器发送所述信号，所述信号包括存储器中存储的计算机指令；当所述处理器执行所述计算机指令时，使得所述电子设备执行权利要求1至12中任意一项所述的媒体信息的获取方法。
PCT/CN2023/083116 2022-03-28 2023-03-22 媒体信息的获取方法及电子设备 WO2023185590A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210309845.3 2022-03-28
CN202210309845.3A CN116866470A (zh) 2022-03-28 2022-03-28 媒体信息的获取方法及电子设备

Publications (1)

Publication Number Publication Date
WO2023185590A1 true WO2023185590A1 (zh) 2023-10-05

Family

ID=88199207

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/083116 WO2023185590A1 (zh) 2022-03-28 2023-03-22 媒体信息的获取方法及电子设备

Country Status (2)

Country Link
CN (1) CN116866470A (zh)
WO (1) WO2023185590A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100215336A1 (en) * 2009-02-26 2010-08-26 International Business Machines Corporation Administering a time-shifting cache in a media playback device
CN104661041A (zh) * 2013-11-25 2015-05-27 乐视网信息技术（北京）股份有限公司 一种直播时移定位的方法、服务器、终端及通信系统
CN106385619A (zh) * 2016-09-14 2017-02-08 乐视控股（北京）有限公司 一种直播视频流的回看方法及装置
CN112019905A (zh) * 2019-05-30 2020-12-01 上海哔哩哔哩科技有限公司 直播回放方法、计算机设备及可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100215336A1 (en) * 2009-02-26 2010-08-26 International Business Machines Corporation Administering a time-shifting cache in a media playback device
CN104661041A (zh) * 2013-11-25 2015-05-27 乐视网信息技术（北京）股份有限公司 一种直播时移定位的方法、服务器、终端及通信系统
CN106385619A (zh) * 2016-09-14 2017-02-08 乐视控股（北京）有限公司 一种直播视频流的回看方法及装置
CN112019905A (zh) * 2019-05-30 2020-12-01 上海哔哩哔哩科技有限公司 直播回放方法、计算机设备及可读存储介质

Also Published As

Publication number Publication date
CN116866470A (zh) 2023-10-10

Similar Documents

Publication Publication Date Title
WO2020221039A1 (zh) 投屏方法、电子设备以及系统
US11818420B2 (en) Cross-device content projection method and electronic device
CN112394895B (zh) 画面跨设备显示方法与装置、电子设备
US11934352B2 (en) Card rendering method and electronic device
US8606183B2 (en) Method and apparatus for remote controlling bluetooth device
WO2022100304A1 (zh) 应用内容跨设备流转方法与装置、电子设备
WO2022267733A1 (zh) 一种动态调节丢帧阈值的方法及相关设备
WO2022105445A1 (zh) 基于浏览器的应用投屏方法及相关装置
WO2022042769A2 (zh) 多屏交互的系统、方法、装置和介质
WO2021213451A1 (zh) 轨迹回放方法及相关装置
CN109819306B (zh) 一种媒体文件裁剪的方法、电子设备和服务器
CN112004041B (zh) 视频录制方法、装置、终端及存储介质
WO2021190524A1 (zh) 截屏处理的方法、图形用户接口及终端
WO2021185352A1 (zh) 一种版本升级方法及相关装置
WO2024041047A1 (zh) 一种屏幕刷新率切换方法及电子设备
WO2023179123A1 (zh) 蓝牙音频播放方法、电子设备及存储介质
WO2022135157A1 (zh) 页面显示的方法、装置、电子设备以及可读存储介质
CN114281440B (zh) 一种双系统中用户界面的显示方法及电子设备
WO2023185590A1 (zh) 媒体信息的获取方法及电子设备
CN114489469B (zh) 一种数据读取方法、电子设备及存储介质
CN116939559A (zh) 蓝牙音频编码数据分发方法、电子设备及存储介质
WO2022160999A1 (zh) 显示方法和电子设备
CN116915896B (zh) 防止蓝牙音频Track音轨抖动的方法及相关设备
CN115529379B (zh) 防止蓝牙音频Track音轨抖动的方法、电子设备及存储介质
WO2023174322A1 (zh) 图层处理方法和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23777961

Country of ref document: EP

Kind code of ref document: A1