WO2025001808A1 - 解码数据获取方法、装置、设备与存储介质 - Google Patents

解码数据获取方法、装置、设备与存储介质 Download PDF

Info

Publication number
WO2025001808A1
WO2025001808A1 PCT/CN2024/097854 CN2024097854W WO2025001808A1 WO 2025001808 A1 WO2025001808 A1 WO 2025001808A1 CN 2024097854 W CN2024097854 W CN 2024097854W WO 2025001808 A1 WO2025001808 A1 WO 2025001808A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage space
decoded data
frame image
target frame
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/097854
Other languages
English (en)
French (fr)
Inventor
邓文尧
赖守波
刘建平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025001808A1 publication Critical patent/WO2025001808A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4335—Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content

Definitions

  • the present disclosure relates to the field of computer technology, and in particular to a method and device for obtaining decoded data, an electronic device, and a computer-readable storage medium.
  • NLE non-linear editing
  • Performing a seek operation on the material loaded on the video editing track refers to dragging the timeline on the video editing track, or dragging the material loaded on the video editing track, so that the timeline position and the material change relative positions along the time axis of the video editing track, locating the target playback position of the material according to the timeline position, and then rendering and presenting at least the image frame located at the target playback position in the material.
  • the material loaded on the video editing track can be called a material track clip.
  • the NLE system will decode the material currently loaded into the memory in real time according to the located target playback position, so as to obtain the decoded data of the image frame corresponding to the target playback position, and then render and present the image frame based on the decoded data.
  • the method of real-time decoding of video files is used to obtain decoded data of image frames matching the target playback position.
  • the problem is that the efficiency of obtaining decoded data is low, thereby reducing the overall efficiency of NLE.
  • the disclosed embodiments provide a method for obtaining decoded data, which is used to solve the problem in the prior art that, in response to a preview operation triggered on a video editing track, real-time decoding of materials is used to obtain decoded data of image frames matching a target playback position, which results in low efficiency in obtaining decoded data and further leads to low overall efficiency of NLE.
  • the embodiments of the present disclosure also provide a decoding data acquisition device, an electronic device, and a computer-readable storage medium.
  • a method for obtaining decoded data comprising:
  • the decoded data of the target frame image is stored in the multi-level cache space; wherein the multi-level cache space at least includes a first storage space and a second storage space, and the read and write speed of the first storage space is greater than the read and write speed of the second storage space; the multi-level cache strategy includes: if the decoded data of the frame image cached in the first storage space does not reach the upper limit of the first storage space, then the decoded data of the target frame image is cached in the first storage space; if the decoded data of the frame image cached in the first storage space has reached the upper limit of the first storage space, then the decoded data of the target frame image is cached in the second storage space;
  • a target storage space is determined from the multi-level cache space, and the target storage space is used to store decoded data of the target frame image;
  • the decoded data of the target frame image is stored in the multi-level cache space, the decoded data of the target frame image is read from the target storage space, and the target frame image is rendered based on the decoded data and a preview image is displayed.
  • a decoding data acquisition device comprising:
  • a decoding unit configured to decode a target frame image in a material track segment presented on a video editing track, and generate decoded data of the target frame image
  • a storage unit for storing the decoded data of the target frame image in a multi-level cache space according to a multi-level cache strategy; wherein the multi-level cache space at least includes a first storage space and a second storage space, and the read and write speed of the first storage space is greater than the read and write speed of the second storage space; the multi-level cache strategy includes: if the decoded data of the frame image cached in the first storage space does not reach the upper limit of the first storage space, then the decoded data of the target frame image is cached in the first storage space; if the decoded data of the frame image cached in the first storage space has reached the upper limit of the first storage space, then the decoded data of the target frame image is cached in the second storage space;
  • a storage space determination unit configured to respond to a preview operation triggered on a video editing track, and if the timeline position of the target frame image in the material track segment is previewed, determine a target storage space from the multi-level cache space, wherein the target storage space is used to store decoded data of the target frame image;
  • a data acquisition and display unit is used to read the decoded data of the target frame image from the target storage space if the decoded data of the target frame image is stored in the multi-level cache space, and render the target frame image based on the decoded data and display a preview image.
  • An electronic device comprising: a memory and a processor, wherein:
  • the memory is used to store programs; the processor is coupled to the memory and is used to execute the programs stored in the memory to execute the above-mentioned decoding data acquisition method.
  • a computer-readable storage medium storing a computer program, wherein the computer program can implement the above-mentioned decoding data acquisition method when executed by a computer.
  • a multi-level cache space including at least a first storage space and a second storage space is used to store the decoded data generated by decoding the target frame image in the material track segment, based on this, in response to the preview operation triggered on the video editing track, when the timeline position of the target frame image in the material track segment is previewed, if the decoded data of the target frame image is stored in the multi-level cache space, then the decoded data of the target frame image can be obtained from the target storage space of the multi-level cache space to render and preview the target frame, without the need to use real-time decoding of video files to obtain the decoded data of the target frame image as in the prior art, thereby improving the efficiency of obtaining the decoded data, and further improving The overall efficiency of the NLE.
  • FIG. 1a is a specific implementation flow chart of a method for obtaining decoded data provided by an embodiment of the present disclosure
  • FIG1b is a schematic diagram of an interface of a non-linear editing system
  • FIG2 is a schematic diagram of a specific implementation process of the decoding data acquisition method provided by an embodiment of the present disclosure in a non-linear editing system
  • FIG3 is a schematic diagram of a specific implementation process of the decoding data acquisition method provided by an embodiment of the present disclosure in a non-linear editing system
  • FIG4 is a schematic diagram of a specific structure of a decoding data acquisition device provided by an embodiment of the present disclosure.
  • FIG5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure.
  • an embodiment of the present disclosure provides a decoding data acquisition method.
  • the execution subject of the method may be, for example, a computing device such as a mobile phone, a personal computer, a wearable device, or software installed on the computing device, such as a player or decoder of a video file, etc.
  • a computing device such as a mobile phone, a personal computer, a wearable device, or software installed on the computing device, such as a player or decoder of a video file, etc.
  • the embodiment of the present disclosure does not limit what computing device or software the execution subject is.
  • FIG. 1a of the specification is a specific implementation flow chart of a method for obtaining decoded data provided by an embodiment of the present disclosure, including the following steps:
  • Step 11 For the material track segment presented on the video editing track, the non-linear editing system decodes the target frame image in the material track segment to generate decoded data of the target frame image;
  • the non-linear editing system may, for example, decode the target image frame contained in the material track segment in response to a user's preview instruction for the material track segment presented on the video editing track, thereby generating decoded data of the target frame image.
  • the "target frame image” may not specifically refer to a certain frame image, but generally refers to an image in a material track segment that is decoded by a linear editing system.
  • Step 12 The nonlinear editing system stores the decoded data of the target frame image in a multi-level cache strategy. Stored in multi-level cache space;
  • the multi-level cache space mentioned here at least includes: a first storage space and a second storage space.
  • the read and write speed of the first storage space is greater than the read and write speed of the second storage space.
  • the first storage space may be, for example, a dedicated storage space; the second storage space may be, for example, a shared storage space.
  • the dedicated storage space mentioned here refers to a storage space that is specifically used to store the decoded data obtained by decoding the target frame image in the material track segment, and is not used to store other data.
  • the shared storage space refers to a storage space that can be used to store the decoded data obtained by decoding the target frame image in the material track segment, but can also be used to store other data.
  • an identifier for the storage space when setting an identifier for the storage space to indicate whether the storage space is a dedicated storage space or a shared storage space, it can be indicated whether the storage space is a dedicated storage space or a shared storage space.
  • a dedicated storage space or a shared storage space
  • dedicated storage space refers to the storage space that is used by the graphics processing unit (GPU) of a computing device itself and is not shared with other applications; while shared storage space refers to the storage space that can be used by all applications but is used preferentially by the GPU.
  • GPU graphics processing unit
  • the first storage space may be, for example, a video memory; and the second storage space may be, for example, a memory.
  • the multi-level cache strategy may include but is not limited to: if the first storage space If the decoded data of the frame image cached in the first storage space does not reach the upper limit of the first storage space, the decoded data of the target frame image is cached in the first storage space; if the decoded data of the frame image cached in the first storage space has reached the upper limit of the first storage space, the decoded data of the target frame image is cached in the second storage space.
  • the decoded data generated by executing step 11 may be first stored in the local memory of the computing device on which the non-linear editing system is installed, so that the non-linear editing system may obtain the decoded data from the memory and then store the decoded data in the multi-level cache space.
  • the non-linear editing system may not perform the operation of obtaining the decoded data from the memory and storing it in the second storage space.
  • the first storage space may include: a first sub-storage space and a second sub-storage space.
  • the first sub-storage space is a dedicated video memory
  • the second sub-storage space is a shared video memory.
  • dedicated video memory refers to the memory used by the GPU of the computing device itself and not shared with other applications; while shared video memory refers to the memory that can be used by all applications but is used by the GPU first.
  • the multi-level caching strategy described in the embodiment of the present disclosure may also include: if the decoded data of the frame image cached in the first sub-storage space has reached the upper limit of the first sub-storage space, then the decoded data of the target frame image is cached in the second sub-storage space.
  • the multi-level cache space in the embodiment of the present disclosure is equivalent to being composed of at least three types of storage spaces, and the three types of storage spaces are: the first sub-storage space, the second sub-storage space, and the second storage space.
  • the first sub-storage space, the second sub-storage space, and the second storage space can be: dedicated video memory, shared video memory, and internal memory, respectively.
  • the non-linear editing system before storing the decoded data in the multi-level cache space, can implement the designation of the storage space as the multi-level cache space through the following steps:
  • the nonlinear editing system obtains the size of the currently available volatile storage space (including memory and video memory) of the computing device - for example, the size of the currently available volatile storage space can be obtained from the global resource management center (Global Resource Center) of the computing device, that is, the size of the currently free volatile storage space; wherein the global resource management center can be a component in the computing device that has the authority to obtain information such as the usage of the volatile storage space;
  • the global resource management center Global Resource Center
  • the non-linear editing system can determine the multi-level cache space based on the size of the currently available volatile storage space (for example, 100M for memory and 50M for video memory) and the preset resource usage ratio (for example, 60%) - specifically 60% of the currently available 100M for memory, that is, 60M for memory; and 60% of the currently available 50M for video memory, that is, 30M for video memory.
  • the size of the currently available volatile storage space for example, 100M for memory and 50M for video memory
  • the preset resource usage ratio for example, 60%
  • the decoded data can be stored in the multi-level cache space for subsequent query. It should be noted that the decoded data stored in the multi-level cache space can have an identifier of the decoded data, such as the frame number of the image frame corresponding to the decoded data; or the decoded data can be stored in the multi-level cache space corresponding to the frame number of the corresponding image frame for subsequent query.
  • the purpose of storing the decoded data obtained by decoding the material track fragment in the multi-level cache space is mainly to take into account that if the non-linear editing system detects that a preview operation for the material track fragment is triggered on the video editing track, then when there is decoded data corresponding to the image to be previewed in the multi-level cache space, the decoded data can be directly obtained from the multi-level cache space without the need to decode the material track fragment in real time to obtain the decoded data, thereby ensuring the data acquisition efficiency of the video editing process.
  • the nonlinear editing system stores the decoded data obtained by decoding the images in the "material track fragments presented on the video editing track", rather than the decoded data obtained by decoding the materials that have not yet been loaded on the video editing track;
  • the decoded data stored in the multi-level cache space is the decoded data obtained by decoding during the video editing process, not the decoded data obtained by decoding in other processes, nor the decoded data obtained by decoding the image that has not been previewed during the video editing process. Therefore, the embodiment of the present disclosure is adopted.
  • the decoded data storage method provided does not require deliberate advance decoding of images to obtain decoded data, but rather stores the decoded data naturally during the image preview process.
  • the above two features are both based on the consideration that in the "non-linear editing" scenario, users are likely to reposition and preview the material track segments at any time, for example, performing a seek operation as described in the background technology.
  • the decoded data that has been decoded before the seek operation is stored, and when the seek operation occurs, the stored decoded data can be directly obtained for rendering and previewing the image, thereby improving the efficiency of non-linear editing.
  • Step 13 The non-linear editing system responds to the preview operation triggered on the video editing track, and if the timeline position of the target frame image in the material track segment is previewed, the target storage space is determined from the multi-level cache space;
  • the preview operation triggered on the video editing track may include but is not limited to the seek operation triggered on the video editing track, that is: by dragging the timeline on the video editing track, or by dragging the material loaded on the video editing track, the timeline position and the material change relative positions along the time axis direction of the video editing track, and then locating the target playback position of the material according to the timeline position, and then rendering and at least presenting the image frame in the material located at the target playback position.
  • Figure 1b is a schematic diagram of presenting a material track segment on the video editing track of a non-linear editing system. As shown in the figure, the image frames contained in the material track segment can be previewed in the preview window. There are control buttons below the playback window to support playback control of the video file.
  • the user may directly drag the timeline as shown in Figure 1b.
  • This dragging operation is This is the preview operation described in step 13; alternatively, the user can also long press the "rewind play" button in the control button, and the long press operation of the button is also the preview operation described in step 13.
  • the user can perform the preview operation in any manner, and the embodiments of the present disclosure do not limit how to perform the preview operation.
  • the target frame image does not specifically refer to a certain frame image, but generally refers to the image decoded by the linear editing system in the material track segment.
  • the linear editing system if the preview operation makes the timeline position coincide with the position of a certain image in the material track segment, then the linear editing system generally sequentially obtains the decoded data of the image, the decoded data of other images that belong to the same material track segment as the image but are arranged after the image, and the decoded data of images in other material track segments that are located on other video editing tracks and are arranged after the material track segment to which the image belongs, and sequentially renders the preview images of each image based on the obtained decoded data.
  • the linear editing system can determine the target storage space from the multi-level cache space, and then execute the subsequent step 14; if, triggered by the preview operation, the decoded data to be acquired by the linear editing system does not include the decoded data of the target image described in step 1, or, although it includes the decoded data of the target image described in step 1, the decoded data does not yet exist in the multi-level cache space, then the linear editing system no longer executes the subsequent step 14, but acquires the decoded data according to the conventional decoded data acquisition method - for example, acquiring the image frame from the image frame currently loaded into the memory for decoding, and obtaining the decoded data.
  • whether the decoded data of the target image exists in the multi-level cache space can be determined by the linear editing system by querying the multi-level cache space according to the identifier of the target image.
  • the identifier of the target image may refer to the number of the target image in the material track segment to which the target image belongs, for example, it may be the frame number of the image frame.
  • any of the following two situations can trigger the non-linear editing system to determine the target storage space where the target frame image is stored from the multi-level cache space: 1.
  • the timeline position just overlaps with the position of the target frame image; 2.
  • the timeline position overlaps with the position of the image arranged before the target frame image in the material track segment.
  • the above-mentioned target storage space refers to a storage space used to store decoded data of a target frame image, which may be a first storage space or a second storage space.
  • a multi-level cache strategy is adopted when storing decoded data.
  • the decoded data will be preferentially stored in the first storage space with a relatively high read and write speed until the first storage space reaches the storage limit, and then the decoded data will be stored in the second storage space.
  • the decoded data stored in the first storage space is consumed, the decoded data stored in the second storage space is transferred to the free storage position released in the first storage space in sequence according to the First In First Out (FIFO) rule.
  • FIFO First In First Out
  • Such a data transfer method is adopted in consideration of the process of previewing the image frames of the material track fragments on the video editing track, which conforms to the feature of "previewing in sequence starting from the timeline position". Based on this feature, if the decoded data of the previewed image frame can be moved from the second storage space to the first storage space in sequence in a FIFO manner, then, in theory, the decoded data can always be read (i.e., consumed) from the first storage space with a higher reading and writing speed, thereby ensuring the efficiency of decoding data acquisition and further ensuring the overall efficiency of the NLE.
  • an optional implementation mode of determining the target storage space from the multi-level cache space in step 13 may include the following sub-steps 1 and 2:
  • Sub-step 1 query the decoded data of the target frame image from the target storage space according to the data query strategy
  • Sub-step 2 Determine the storage space where the decoded data of the queried target frame image is located as the target frame image. Label storage space.
  • the data query strategy may include: querying different storage spaces included in the multi-level cache space in order of the read and write speed of the storage space from large to small.
  • step 14 is executed.
  • Step 14 If the decoded data of the target frame image is stored in the multi-level cache space, the non-linear editing system reads the decoded data of the target frame image from the target storage space, and renders the target frame image based on the decoded data of the target frame image and displays a preview image.
  • the decoded data of the target frame image when the decoded data of the target frame image is stored in step 12, the identifier of the target frame image and the decoded data may be stored correspondingly.
  • the decoded data stored corresponding to the identifier may be read from the target storage space as the decoded data of the target frame image.
  • the target frame image can be rendered based on the decoded data and a preview image can be displayed.
  • the non-linear editing system may read the decoded data in the target storage space into the graphics processor memory (referred to as video memory), so that the graphics processor may render a preview image corresponding to the target frame image based on the decoded data.
  • the graphics processor memory referred to as video memory
  • the decoded data stored in the multi-level cache space may contain decoded data that has not been acquired and used (acquisition and use can be called consumption) by the non-linear editing system from the multi-level cache space in the recent period of time.
  • decoded data is "cold data", and cold data continues to occupy storage resources, which is equivalent to causing a waste of storage resources. Therefore, in an optional implementation, the non-linear editing system can clear the decoded data that meets the data elimination rule from the multi-level cache space according to the data elimination rule set based on the Least Recently Used (LRU) algorithm.
  • LRU Least Recently Used
  • the LRU algorithm refers to replacing the least used data.
  • the data elimination rule set based on the LRU algorithm may include: eliminating the least used data relatively, and replacing the least used data with the decoded data obtained by the most recent decoding.
  • the non-linear editing system when it obtains decoded data for rendering, it is specifically implemented by the non-linear editing system triggering the graphics processor.
  • the efficiency of obtaining decoded data from the video memory for rendering is higher than the efficiency of obtaining decoded data from the memory for rendering.
  • the graphics processor obtains decoded data from the memory of the computing device, it will read the obtained decoded data from the memory to the video memory, and then obtain the decoded data from the video memory for rendering processing - therefore, from the perspective of the graphics processor, when both the memory and the video memory can be used to store decoded data, the decoded data is stored in the video memory first, which is more conducive to the graphics processor to efficiently obtain the decoded data that has been decoded.
  • the video memory is determined as a storage space with a high consumption priority in the multi-level cache space
  • the internal memory is determined as a secondary storage space with a low consumption priority in the multi-level cache space.
  • the "consumption priority” mentioned here can be equivalent to the "reading and writing speed” mentioned above, specifically, for example, it can refer to the reading and writing speed when the graphics processor reads and writes data.
  • the dedicated video memory can be further determined as the storage space with the highest consumption priority in the multi-level cache space
  • the shared video memory can be determined as the storage space with the second highest consumption priority in the multi-level cache space.
  • the consumption priority of the multi-level cache space is equivalent to the query priority followed by the non-linear editing system when querying the decoded data stored in the multi-level cache space.
  • the storage space with a higher consumption priority will be queried more preferentially when the non-linear editing system queries the decoded data.
  • the non-linear editing system can select the target storage space according to the data query strategy.
  • the decoded data of the target frame image is searched in the storage space; and the storage space where the decoded data of the target frame image is searched is determined as the target storage space.
  • the data query strategy includes: querying different storage spaces included in the multi-level cache space in order from large to small according to the reading and writing speed (ie, consumption priority) of the storage space.
  • a multi-level cache space including at least a first storage space and a second storage space is adopted to store the decoded data generated by decoding the target frame image in the material track segment.
  • the decoded data of the target frame image is stored in the multi-level cache space, then the decoded data of the target frame image can be obtained from the target storage space of the multi-level cache space to render the target frame and display the preview image, without the need to obtain the decoded data of the target frame image by real-time decoding of the video file as in the prior art, thereby improving the efficiency of obtaining the decoded data, and further improving the overall efficiency of the NLE.
  • FIG2 of the description is a schematic diagram of the specific implementation process of the decoding data acquisition method in practice.
  • the execution subject of each step in the process may be, for example, a non-linear editing system.
  • the process mainly includes the following steps:
  • Step 31 The non-linear editing system obtains the size of the currently available dedicated video memory, the size of the shared video memory, and the size of the internal memory from the global resource management center of the local device (such as the user's personal computer) where the non-linear editing system is installed;
  • Step 32 The nonlinear editing system sets/designates the dedicated video memory, the shared video memory, and the internal memory as a decoding buffer pool, and sets 60% of the currently available storage space sizes of the dedicated video memory, the shared video memory, and the internal memory as the storage space size of the decoding buffer pool according to the default resource usage ratio of 60%;
  • the consumption priority of dedicated video memory, shared video memory, and internal memory may be in the order of: dedicated video memory>shared video memory>internal memory.
  • Step 33 the non-linear editing system stores the decoded data of the video frame obtained by decoding the decoder in the process of non-linear editing the video file into the decoding buffer pool;
  • the decoded data of the video frames can be stored in the decoding cache pool in the order of consumption priority from high to low, and in accordance with the rule of filling up different types of storage space (including dedicated video memory, shared video memory, and internal memory) contained in the decoding cache pool in sequence.
  • full storage refers to filling up 60% of the currently available storage space of the dedicated video memory, using the above example.
  • Step 34 The non-linear editing system receives a fast-rewind instruction during the non-linear editing of the video file
  • Step 35 the non-linear editing system responds to the instruction and queries the decoding buffer pool according to the frame number of the video frame to be decoded corresponding to the instruction;
  • the decoded data of the video frame with the frame number is directly obtained from the decoded buffer pool, and then based on the decoded data of the video frame, the operation of rendering and presenting the corresponding preview image is performed; wherein, the decoded data of the video frame obtained from the decoded buffer pool by the non-linear editing system will be deleted from the decoded buffer pool as the decoded data of the consumed video frame, and then, the decoded data of the video frame stored in the decoded buffer pool will be transferred in a FIFO manner - the details can be seen in the relevant description in the previous embodiment, which will not be repeated here.
  • step 36 is executed;
  • Step 36 the non-linear editing system decodes the video file loaded in the memory of the computing device to obtain decoded data of the video frame with the frame number, and then performs an operation of rendering and presenting a corresponding image based on the video frame; in addition, jump to step 37;
  • Step 37 The non-linear editing system stores the decoded data of the video frame with the frame number obtained by performing the decoding in step 36 into the decoding buffer pool.
  • the non-linear editing system can run a thread such as "Recyle Thread" which is used to monitor the usage of the decoded data of the video frames stored in the decode buffer pool; If the thread monitors that the decoded data of a certain video frame/frames stored in the decoding cache pool has been consumed the least number of times from the storage time to the current time compared with the decoded data of other video frames - for example, it is less than the preset threshold Cache Threshold, then the non-linear editing system can clear the decoded data of the video frames that meet the data elimination rules from the decoding cache pool according to the data elimination rules determined based on the LRU algorithm, that is, for example, eliminate the decoded data of the video frames that have been consumed the number of times less than the preset threshold Cache Threshold.
  • the thread monitors that the decoded data of a certain video frame/frames stored in the decoding cache pool has been consumed the least number of times from the storage time to the current time compared with the decode
  • a decoding buffer pool is used to realize the storage of decoded data of video frames decoded in the process of decoding and presenting video files by a non-linear editing system. Based on this, when an instruction is received during the process to trigger decoding of a video file to obtain decoded data of a specified video frame, the decoded data of the specified video frame can be obtained from the decoding buffer pool, without the need to use real-time decoding of video files to obtain decoded data of the specified video frame as in the prior art, thereby improving the efficiency of obtaining decoded data of the specified video frame in the process.
  • the disclosed embodiments further provide a decoding data acquisition device.
  • the decoding unit 41 is used to decode the target frame image in the material track segment presented on the video editing track to generate decoded data of the target frame image;
  • the storage unit 42 is used to store the decoded data of the target frame image in a multi-level cache space according to a multi-level cache strategy; wherein the multi-level cache space at least includes a first storage space and a second storage space, and the read and write speed of the first storage space is greater than the read and write speed of the second storage space; the multi-level cache strategy includes: if the decoded data of the frame image cached in the first storage space does not reach the If the decoded data of the frame image cached in the first storage space reaches the upper limit of the first storage space, the decoded data of the target frame image is cached in the first storage space; if the decoded data of the frame image cached in the first storage space reaches the upper limit of the first storage space, the decoded data of the target frame image is cached in the second storage space;
  • a storage space determination unit 43 is used to respond to a preview operation triggered on a video editing track, and if the timeline position of the target frame image in the material track segment is previewed, determine a target storage space from the multi-level cache space, wherein the target storage space is used to store decoded data of the target frame image;
  • the data acquisition and display unit 44 is used to read the decoded data of the target frame image from the target storage space if the decoded data of the target frame image is stored in the multi-level cache space, and render the target frame image based on the decoded data and display a preview image.
  • the storage space determining unit 43 may be configured to:
  • the storage space where the decoded data of the target frame image is found is determined as the target storage space.
  • the data query strategy includes: querying different storage spaces included in the multi-level cache space in sequence according to the order of the read and write speed of the storage space from large to small.
  • the device provided by the embodiment of the present disclosure may further include:
  • the data transfer unit is used to transfer the decoded data stored in the second storage space to the free storage positions released in the first storage space in sequence according to the first-in-first-out (FIFO) rule if the decoded data stored in the first storage space is consumed.
  • FIFO first-in-first-out
  • the first storage space is a dedicated storage space; the second storage space is a shared storage space; or, the first storage space is a video memory; and the second storage space is a memory.
  • the first storage space includes: a first sub-storage space and a second sub-storage space;
  • the first storage space is a video memory
  • the first sub-storage space is a dedicated video memory
  • the second sub-storage space is a shared video memory
  • the multi-level cache strategy further includes: if the decoded data of the frame image cached in the first sub-storage space has reached the upper limit of the first sub-storage space, then caching the decoded data of the target frame image in the second sub-storage space.
  • the device provided by the embodiment of the present disclosure may further include:
  • the data clearing unit is used to clear the decoded data that meets the data elimination rule from the multi-level cache space according to the data elimination rule set based on the least recently used LRU algorithm.
  • the above-mentioned decoding data acquisition device adopts a multi-level cache space including at least a first storage space and a second storage space to store the decoding data generated by decoding the target frame image in the material track segment.
  • a multi-level cache space including at least a first storage space and a second storage space to store the decoding data generated by decoding the target frame image in the material track segment.
  • the embodiments of the present disclosure also provide an electronic device to solve the problem in the prior art that, in response to a preview operation triggered on a video editing track, real-time decoding of materials is used to obtain decoding data of image frames matching a target playback position, which results in low efficiency in obtaining decoding data and further leads to low overall efficiency of the NLE.
  • the electronic device includes: a memory 51 and a processor 52.
  • the memory 51 can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device.
  • the memory 51 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), or a combination thereof.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM erasable programmable read-only memory
  • EPROM Programmable Read-Only Memory
  • ROM Read-Only Memory
  • Magnetic Storage Flash Memory
  • Magnetic or Optical Disk any type of volatile or non-volatile storage device or a combination thereof.
  • the processor 52 is coupled to the memory 51 and is used to execute the program stored in the memory 51 to execute the decoding data acquisition method described in the embodiment of the present disclosure, or to execute the multimedia data storage method described in the embodiment of the present disclosure.
  • the processor 52 executes the program in the memory 51, in addition to the above functions, it can also realize other functions, and the details can be referred to the description of the previous embodiments.
  • the electronic device also includes other components such as a display 54, a communication component 53, a power component 55, and an audio component 56.
  • Fig. 5 only schematically shows some components, which does not mean that the electronic device only includes the components shown in Fig. 5 .
  • an embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a computer, the steps or functions of the methods provided in the above embodiments can be implemented.
  • the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
  • each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware.
  • the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

本公开公开一种解码数据获取方法、装置、设备与存储介质，解决现有技术存在的解码数据获取效率较低，从而导致NLE的整体效率较低的问题。方法包括：对素材轨道片段中的目标帧图像进行解码，生成目标帧图像的解码数据；按照多级缓存策略，将解码数据存储到多级缓存空间；多级缓存策略包括：若第一存储空间中已缓存的解码数据未达到第一存储空间的上限则在第一存储空间中缓存目标帧图像的解码数据，若第一存储空间中已缓存的解码数据已达到第一存储空间的上限则在第二存储空间中缓存目标帧图像的解码数据；从多级缓存空间中确定目标存储空间；从目标存储空间中读取目标帧图像的解码数据，基于解码数据对目标帧图像进行渲染并显示预览图像。

Description

解码数据获取方法、装置、设备与存储介质
相关申请的交叉引用
本申请要求申请号为202310776768.7，题为“解码数据获取方法、装置、设备与存储介质”、申请日为2023年6月28日的中国发明专利申请的优先权，通过引用的方式将该申请整本并入本文。
技术领域
本公开涉及计算机技术领域，尤其涉及一种解码数据获取方法、装置、电子设备与计算机可读存储介质。
背景技术
用户在采用非线性编辑(Non-linear edit，NLE)系统对视频文件等素材进行编辑的过程中，有可能会对加载到视频编辑轨道上的素材执行seek操作。
对加载到视频编辑轨道上的素材执行seek操作，是指通过拖动视频编辑轨道上的时间线，或者通过拖动加载到视频编辑轨道上的素材，使得时间线位置和素材沿视频编辑轨道的时间轴方向发生相对位置变化后，根据时间线位置定位素材的目标播放位置，进而渲染并至少呈现素材中位于目标播放位置处的图像帧的操作。其中，加载到视频编辑轨道上的素材，可称为素材轨道片段。
按照现有技术，在执行seek操作后，NLE系统会根据定位出的目标播放位置，对当前加载到内存中的素材进行实时解码，以便得到与目标播放位置相对应的图像帧的解码数据，进而基于解码数据进行图像帧的渲染与呈现。
采用实时解码视频文件的方式来获取与目标播放位置相匹配的图像帧的解码数据，存在的问题在于对于解码数据的获取效率较低，进而降低了NLE的整体效率。
发明内容
本公开实施例提供一种解码数据获取方法，用以解决现有技术中响应于视频编辑轨道上触发的预览操作，采用实时解码素材的方式来获取与目标播放位置相匹配的图像帧的解码数据，会导致解码数据获取效率较低，进而导致NLE的整体效率较低的问题。
本公开实施例还提供一种解码数据获取装置、电子设备和计算机可读存储介质。
本公开实施例采用下述技术方案：
一种解码数据获取方法，包括：
对于在视频编辑轨道上呈现的素材轨道片段，对所述素材轨道片段中的目标帧图像进行解码，生成所述目标帧图像的解码数据；
按照多级缓存策略，将所述目标帧图像的解码数据存储到多级缓存空间；其中，所述多级缓存空间至少包括第一存储空间和第二存储空间，所述第一存储空间的读写速度大于所述第二存储空间的读写速度；所述多级缓存策略包括：若第一存储空间中已缓存的帧图像的解码数据未达到所述第一存储空间的上限则在所述第一存储空间中缓存所述目标帧图像的解码数据，若第一存储空间中已缓存的帧图像的解码数据已达到所述第一存储空间的上限则在所述第二存储空间中缓存所述目标帧图像的解码数据；
响应于在视频编辑轨道上触发的预览操作，若预览到所述素材轨道片段中所述目标帧图像的时间线位置，从所述多级缓存空间中确定目标存储空间，所述目标存储空间用于存储所述目标帧图像的解码数据；
若所述多级缓存空间中存储有所述目标帧图像的解码数据，从所述目标存储空间中读取所述目标帧图像的解码数据，并基于所述解码数据对所述目标帧图像进行渲染并显示预览图像。
一种解码数据获取装置，包括：
解码单元，用于对于在视频编辑轨道上呈现的素材轨道片段，对所述素材轨道片段中的目标帧图像进行解码，生成所述目标帧图像的解码数据；
存储单元，用于按照多级缓存策略，将所述目标帧图像的解码数据存储到多级缓存空间；其中，所述多级缓存空间至少包括第一存储空间和第二存储空间，所述第一存储空间的读写速度大于所述第二存储空间的读写速度；所述多级缓存策略包括：若第一存储空间中已缓存的帧图像的解码数据未达到所述第一存储空间的上限则在所述第一存储空间中缓存所述目标帧图像的解码数据，若第一存储空间中已缓存的帧图像的解码数据已达到所述第一存储空间的上限则在所述第二存储空间中缓存所述目标帧图像的解码数据；
存储空间确定单元，用于响应于在视频编辑轨道上触发的预览操作，若预览到所述素材轨道片段中所述目标帧图像的时间线位置，从所述多级缓存空间中确定目标存储空间，所述目标存储空间用于存储所述目标帧图像的解码数据；
数据获取与显示单元，用于若所述多级缓存空间中存储有所述目标帧图像的解码数据，从所述目标存储空间中读取所述目标帧图像的解码数据，并基于所述解码数据对所述目标帧图像进行渲染并显示预览图像。
一种电子设备，包括：存储器及处理器，其中，
所述存储器，用于存储程序；所述处理器，与所述存储器耦合，用于执行所述存储器中存储的所述程序，以用于执行上述解码数据获取方法。
一种存储有计算机程序的计算机可读存储介质，所述计算机程序被计算机执行时能够实现上述解码数据获取方法。
本公开实施例采用的上述至少一个技术方案能够达到以下有益效果：
由于采用至少包括第一存储空间和第二存储空间的多级缓存空间，存储对素材轨道片段中的目标帧图像进行解码生成的解码数据，基于此，响应于视频编辑轨道上触发的预览操作，在预览到素材轨道片段中目标帧图像的时间线位置的情况下，若多级缓存空间中存储有目标帧图像的解码数据，那么就可以从多级缓存空间的目标存储空间中，获取目标帧图像的解码数据进行目标帧的渲染与预览图像显示，而无需像现有技术采用实时解码视频文件的方式来获取目标帧图像的解码数据，从而提高了对于解码数据的获取效率，进而也就提高了 NLE的整体效率。
附图说明
此处所说明的附图用来提供对本公开的进一步理解，构成本公开的一部分，本公开的示意性实施例及其说明用于解释本公开，并不构成对本公开的不当限定。在附图中：
图1a为本公开实施例提供的一种解码数据获取方法的具体实现流程图；
图1b为一种非线性编辑系统的界面示意图；
图2为本公开实施例提供的解码数据获取方法在非线性编辑系统中的一种具体实现过程的示意图；
图3为本公开实施例提供的解码数据获取方法在非线性编辑系统中的一种具体实现过程的示意图；
图4为本公开实施例提供的一种解码数据获取装置的具体结构示意图；
图5为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
为使本公开的目的、技术方案和优点更加清楚，下面将结合本公开具体实施例及相应的附图对本公开技术方案进行清楚、完整地描述。显然，所描述的实施例仅是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
本领域普通技术人员可知，随着技术的发展和新场景的出现，本公开实施例提供的技术方案对于类似的技术问题，同样适用。
本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，这仅仅是描述本公开的实施例中对相同属性 的对象在描述时所采用的区分方式。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，以便包含一系列单元的过程、方法、系统、产品或设备不必限于那些单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它单元。
为解决现有技术中响应于视频编辑轨道上触发的预览操作，采用实时解码素材的方式来获取与目标播放位置相匹配的图像帧的解码数据，会导致解码数据获取效率较低，进而导致NLE的整体效率较低的问题，本公开实施例提供一种解码数据获取方法。
该方法的执行主体，比如可以是手机、个人电脑、可穿戴设备等计算设备，或者，也可以是计算设备上安装的软件，比如可以是视频文件的播放器或者解码器等等。本公开实施例对执行主体是何种计算设备或软件不做限定。
此外，该方法的不同步骤可以由不同的计算设备或软件实施，本公开实施例对具体采用何种计算设备或软件实施哪个步骤，不做限定。
为便于描述，以下以本方法的执行主体为安装于计算设备的非线性编辑系统为例，对本公开实施例提供的该方法进行详细说明。
请参考说明书附图1a，为本公开实施例提供的一种解码数据获取方法的具体实现流程图，包括如下步骤：
步骤11：对于在视频编辑轨道上呈现的素材轨道片段，非线性编辑系统对素材轨道片段中的目标帧图像进行解码，生成目标帧图像的解码数据；
对于步骤11，在一种可选的实施方式中，非线性编辑系统比如可以是响应于用户对于视频编辑轨道上呈现的素材轨道片段的预览指令，对素材轨道片段所包含的目标图像帧进行解码，从而生成目标帧图像的解码数据。
在步骤11中，所述的“目标帧图像”可以并不具体指代某一帧图像，而是泛指素材轨道片段中被线性编辑系统解码的图像。
步骤12：非线性编辑系统按照多级缓存策略，将目标帧图像的解码数据存 储到多级缓存空间；
其中，这里所说的多级缓存空间，至少包括：第一存储空间和第二存储空间。
在本公开实施例中，第一存储空间的读写速度大于第二存储空间的读写速度。
在一个具体的示例中，第一存储空间，比如可以是专用存储空间；第二存储空间，比如可以是共享存储空间。
这里所说的专用存储空间，是指专门用于存储对素材轨道片段中的目标帧图像进行解码得到的解码数据，而不用于存储其他数据的存储空间。而相对的，共享存储空间，是指虽可以用于存储对素材轨道片段中的目标帧图像进行解码得到的解码数据，但也可以用于存储其他数据的存储空间。
在一种可选的实施方式中，通过为存储空间设置用以表征存储空间为专用存储空间或者为共享存储空间的标识，可以指明存储空间是专用存储空间还是共享存储空间。在存储数据至专用存储空间(或共享存储空间)时，可以根据该标识，判断存储空间为专用存储空间还是共享存储空间，进而可以进一步根据预先设置的“专用存储空间专门用于存储对素材轨道片段中的目标帧图像进行解码得到的解码数据，而不用于存储其他数据；以及，共享存储空间可以用于存储对素材轨道片段中的目标帧图像进行解码得到的解码数据，也可以用于存储其他数据”这样的存储规则，判断当前待存储的数据是否可以存入存储至专用存储空间(或共享存储空间)。
或者，专用存储空间，是指计算设备的图形处理器(Ggraphics processing unit，GPU)自己使用而不会共享给其它应用程序使用的存储空间；而共享存储空间，是指所有应用程序都可以使用，但是GPU优先使用的存储空间。
在一个具体的示例中，第一存储空间，比如可以是显存；第二存储空间，比如可以是内存。
在本公开实施例中，多级缓存策略，可以包括但不限于：若第一存储空间 中已缓存的帧图像的解码数据未达到第一存储空间的上限，则在第一存储空间中缓存目标帧图像的解码数据；若第一存储空间中已缓存的帧图像的解码数据已达到第一存储空间的上限则在第二存储空间中缓存目标帧图像的解码数据。
对于步骤12的具体实现方式而言，在一种可选的实施方式中，通过执行步骤11所生成的解码数据，可以先存储至安装有该非线性编辑系统的计算设备本地的内存中，从而，非线性编辑系统可以从该内存中获取解码数据，进而将解码数据存储至多级缓存空间。当然，当第二存储空间是内存，且第一存储空间中已存储的帧图像的解码数据已经达到第一存储空间的上限时，非线性编辑系统可以不执行从内存中获取解码数据存储至第二存储空间的操作。
在一种可选的实施方式中，为实现对数据的灵活协同存储，第一存储空间可以包括：第一子存储空间和第二子存储空间。在一个具体的示例中，当第一存储空间为显存时，第一子存储空间为专用显存，第二子存储空间为共享显存。
其中，专用显存，是指计算设备的GPU自己使用而不会共享给其它应用程序使用的内存；而共享显存，是指所有应用程序都可以使用，但是GPU优先使用的内存。
在一种可选的实施方式中，当第一存储空间包括第一子存储空间和第二子存储空间时，本公开实施例中所述的多级缓存策略，还可以包括：若第一子存储空间中已缓存的帧图像的解码数据已达到第一子存储空间的上限，则在第二子存储空间中缓存目标帧图像的解码数据。
当第一存储空间包括第一子存储空间和第二子存储空间时，本公开实施例中的多级缓存空间相当于至少由三类存储空间构成，三类存储空间分别为：第一子存储空间、第二子存储空间和第二存储空间。在一个具体的示例中，第一子存储空间、第二子存储空间和第二存储空间可以分别为：专用显存、共享显存和内存。
本公开实施例中，在执行将解码数据存储至多级缓存空间之前，可以由非线性编辑系统通过下述步骤，实现指定存储空间作为多级缓存空间：
首先，非线性编辑系统获取计算设备当前可用的易失性存储空间(包括内存和显存)的大小——比如，可以从计算设备的全局的资源管理中心(Global Resource Center)处，获取当前可用的易失性存储空间的大小，也即获取当前空闲的易失性存储空间的大小；其中，全局的资源管理中心，可以是计算设备中具备可获取易失性存储空间使用情况等信息的权限的一个组件；
然后，非线性编辑系统根据获取到的当前可用的易失性存储空间的大小(比如内存为100M，显存为50M)，以及预先设置的资源使用比例(比如60％)，可以确定出多级缓存空间——具体为当前可用的100M的内存的60％，即60M的内存；以及，当前可用的50M的显存的60％，即30M的显存。
在确定出多级缓存空间后，后续，就可以将解码数据存储至多级缓存空间以便后续查询。需说明的是，存储至多级缓存空间中的解码数据，可以具备解码数据的标识，比如解码数据对应的图像帧的帧编号；或者，解码数据可以与对应的图像帧的帧编号，对应存储至多级缓存空间，以便后续查询。
需要说明的是，本公开实施例中，将对素材轨道片段解码得到的解码数据存储至多级缓存空间的目的，主要是考虑到后续若非线性编辑系统检测到在视频编辑轨道上触发针对该素材轨道片段的预览操作，那么，在多级缓存空间中存在待预览的图像对应的解码数据时，就可以直接从多级缓存空间中获取解码数据，而无需实时对该素材轨道片段进行解码来获取解码数据，从而保证视频编辑过程的数据获取效率。
本公开实施例中对于解码数据的存储有这样的特点：
1、非线性编辑系统是针对“在视频编辑轨道上呈现的素材轨道片段”中的图像进行解码得到的解码数据进行存储，而非是对尚未加载到视频编辑轨道上的素材进行解码得到的解码数据进行存储；
2、存储至多级缓存空间的解码数据，是视频编辑过程中已解码获得的解码数据，而非其他过程中解码获得的解码数据，也并非是对视频编辑过程中尚未进行过预览的图像进行解码而得到的解码数据——从而，采用本公开实施例 提供的解码数据存储方式，无需刻意提前解码图像得到解码数据，而是伴随图像的预览过程，顺其自然进行解码数据的存储。
上述两个特点，均是考虑到“非线性编辑”场景下，用户很有可能会随时对素材轨道片段重新进行定位预览，比如，像背景技术所描述的那样进行seek操作——在这样的场景和需求背景下，对seek操作前已经解码得到的解码数据进行存储，可以在seek操作发生时，直接获取存储的解码数据进行渲染和预览图像的呈现，提升非线性编辑的效率。
步骤13：非线性编辑系统响应于在视频编辑轨道上触发的预览操作，若预览到素材轨道片段中目标帧图像的时间线位置，从多级缓存空间中确定目标存储空间；
其中，在视频编辑轨道上触发的预览操作，比如可以包括但不限于是在视频编辑轨道上触发的seek操作，也就是：通过拖动视频编辑轨道上的时间线，或者通过拖动加载到视频编辑轨道上的素材，使得时间线位置和素材沿视频编辑轨道的时间轴方向发生相对位置变化后，根据时间线位置定位素材的目标播放位置，进而渲染并至少呈现素材中位于目标播放位置处的图像帧的操作。
当然，除了通过拖动时间线或者拖动素材的操作，其他可以使得时间线位置和素材沿视频编辑轨道的时间轴方向发生相对位置变化，从而会重新根据时间线位置定位素材的目标播放位置，进而渲染并至少呈现素材中位于目标播放位置处的图像帧的操作的操作，都可以归为本公开实施例所述的“预览操作”的范畴。
为便于读者理解在视频编辑轨道上触发的预览操作和时间线位置的关系，可参见说明书附图1b。该附图1b，为在非线性编辑系统的视频编辑轨道上呈现素材轨道片段的示意图，如该图所示，素材轨道片段所包含的图像帧可在预览窗口中进行预览。该播放窗口下方具有支持对视频文件进行播放控制的控制按钮。
在实际场景中，用户可能会直接拖动如图1b中的时间线，该拖动操作即 为步骤13中所述的预览操作；或者，用户也可以执行长按控制按钮中的“后退播放”按钮，长按该按钮的操作也是步骤13中所述的预览操作。
在实际场景中，用户可以以任何方式来执行预览操作，本公开实施例对如何执行预览操作不做限定。
对于步骤13中所述的目标帧图像，前文已经说明，所述的“目标帧图像”并不具体指代某一帧图像，而是泛指素材轨道片段中被线性编辑系统解码的图像。在步骤13中，若预览操作使得时间线位置与素材轨道片段中的某个图像所在位置重合，那么，线性编辑系统一般会依次获取该图像的解码数据、与该图像同属同一素材轨道片段但排列于该图像后的其他图像的解码数据，以及，位于其他视频编辑轨道上、且排列于该图像所属素材轨道片段之后的其他素材轨道片段中的图像的解码数据，并依次基于获取到的解码数据渲染得到各个图像的预览图像。
若在预览操作的触发下，线性编辑系统将要获取的上述解码数据中包含步骤1中所述的目标图像的解码数据，且目标图像的解码数据存在于多级缓存空间中，那么线性编辑系统可以从多级缓存空间中确定出目标存储空间，进而执行后续的步骤14；而若在预览操作的触发下，线性编辑系统将要获取的上述解码数据中并不包含步骤1中所述的目标图像的解码数据，或，虽然包含步骤1中所述的目标图像的解码数据，但是该解码数据尚不存在于多级缓存空间中，那么线性编辑系统不再执行后续的步骤14，而是按照常规的解码数据获取途径来获取解码数据——比如从当前加载到内存中的图像帧中获取图像帧进行解码，而获得解码数据。
在本公开实施例中，目标图像的解码数据是否存在于多级缓存空间，可以由线性编辑系统根据目标图像的标识，通过查询多级缓存空间来确定。
本公开实施例中，目标图像的标识，可以是指目标图像在目标图像所属的素材轨道片段中的编号，比如可以是图像帧的帧编号。
本公开实施例中，假设目标帧图像的解码数据已经通过执行步骤12存储 至多级缓存空间，并且当前尚未被消费，那么，这两种情况发生任意一种，都可以触发非线性编辑系统从多级缓存空间中确定出存储有目标帧图像的目标存储空间：1、时间线位置刚好与目标帧图像的位置重叠；2、时间线位置与素材轨道片段中排列在目标帧图像之前的图像的位置重叠。
上述目标存储空间，是指用于存储目标帧图像的解码数据的存储空间，它可能是第一存储空间，也可能是第二存储空间。
如前文所述，本公开实施例中，在对解码数据进行存储时，采用的是多级缓存策略。按照该多级缓存策略，会优先将解码数据存储至读写速度相对较大的第一存储空间，直至第一存储空间达到存储上限，而后才会将解码数据存储至第二存储空间。
可以理解，从保证解码数据获取效率的角度来说，如若能够总是从读写速度较大的第一存储空间进行读取解码数据，能够最大程度上保证解码数据的获取效率。因此，本公开实施例中，若第一存储空间中存储的解码数据被消费，则，将第二存储空间中存储的解码数据，按照先进先出(First In First Out，FIFO)的规则，依次转移至第一存储空间中经释放得到的空闲存储位置。
采用这样的数据转移方式，也是考虑到对视频编辑轨道上的素材轨道片段的图像帧进行预览的过程，符合“从时间线位置开始依次预览”的特点，从该特点出发，如果被预览的图像帧的解码数据，能够按照FIFO的方式依次从第二存储空间移动至第一存储空间，那么，理论上可以总是从读写速度较大的第一存储空间读取(即消费)解码数据，从而保证解码数据获取效率，进而保证NLE的整体效率。
对应于上述解码数据的依次转移方式，步骤13中从多级缓存空间中确定目标存储空间的一种可选的实施方式，可以包括下述子步骤1和子步骤2：
子步骤1：按照数据查询策略，从目标存储空间中查询目标帧图像的解码数据；
子步骤2：将查询到的目标帧图像的解码数据所在的存储空间，确定为目 标存储空间。
其中，数据查询策略，可以包括：按照存储空间的读写速度由大至小的顺序，依次查询多级缓存空间包含的不同存储空间。
按照该数据查询策略，若查询到多级缓存空间中存在目标帧图像的解码数据，那么，可以确定该解码数据具体在哪个存储空间中——也就是确定出目标存储空间，进而执行步骤14。
步骤14：若多级缓存空间中存储有目标帧图像的解码数据，非线性编辑系统从目标存储空间中读取目标帧图像的解码数据，并基于目标帧图像的解码数据，对目标帧图像进行渲染并显示预览图像。
在一种可选的实施方式中，在执行步骤12中的存储目标帧图像的解码数据时，可以对目标帧图像的标识和解码数据进行对应存储。从而，在步骤14中，可以根据目标帧图像的标识，从目标存储空间中读取与该标识对应存储的解码数据，作为目标帧图像的解码数据。
在读取到目标帧图像的解码数据后，就可以基于解码数据目标帧图像进行渲染并显示预览图像。
在一种可选的实施方式中，非线性编辑系统可以将目标存储空间中的解码数据读取到图形处理器内存(简称显存)中，从而使得图形处理器可以基于该解码数据渲染得到目标帧图像对应的预览图像。
考虑到存储至多级缓存空间的解码数据中，可能会存在在最近一段时间内，一直没有被非线性编辑系统从该多级缓存空间中获取并使用(获取并使用可称为消费)过的解码数据。这样的解码数据属于“冷数据”，冷数据持续占用存储资源，相当于是导致存储资源浪费。因此，在一种可选的实施方式中，非线性编辑系统可以按照基于最近最少使用(Least Recently Used，LRU)算法设置的数据淘汰规则，从所述多级缓存空间中清除满足所述数据淘汰规则的解码数据。
其中，LRU算法，是指替换掉最少使用的数据。本公开实施例中，基于LRU算法设置的数据淘汰规则，可以包括：淘汰相对而言最少使用的数据，并以最近解码得到的解码数据替代所述最少使用的数据。
采用上述淘汰方式，可以保证多级缓存空间的存储资源得到充分利用，避免不经常使用的“冷数据”占用存储资源而导致资源浪费的问题。
在一种可选的实施方式中，考虑到当非线性编辑系统获取解码数据进行渲染时，具体是由非线性编辑系统触发图形处理器来实现的，而对于图形处理器而言，从显存获取解码数据进行渲染的效率会高于从内存获取解码数据进行渲染的效率。这是由于若图形处理器从计算设备的内存中获取解码数据，会将获取到的解码数据从内存读取至显存，然后再从显存获取解码数据进行渲染处理——因此，从图形处理器的角度来看，当内存和显存均可用于存储解码数据，将解码数据优先存储至显存，更有利于图形处理器高效地获取已解码得到的解码数据。
基于上述考虑，本公开实施例中，将显存确定为多级缓存空间中具备高消费优先级的存储空间，而将内存确定为多级缓存空间中具备低消费优先级的目第二存储空间。这里所说的“消费优先级”，可以相当于前文所说的“读写速度”，具体而言比如可以是指由图形处理器进行数据读写时的读写速度。
进一步地，若第一存储空间包括专用显存(即第一子存储空间)和共享显存(即第二子存储空间)，则可以进一步地将专用显存确定为多级缓存空间中具备最高消费优先级的存储空间，而将共享显存确定为多级缓存空间中具备次高消费优先级的存储空间。
需要说明的是，多级缓存空间的消费优先级，相当于是非线性编辑系统对于多级缓存空间中存储的解码数据进行查询所遵循的查询优先级。消费优先级越高的存储空间，在非线性编辑系统查询解码数据时，会越优先被查询。
基于多级缓存空间所包含的不同类型的存储空间具备不同的消费优先级，在一种可选的实施方式中，非线性编辑系统可以按照数据查询策略，从目标存 储空间中查询目标帧图像的解码数据；将查询到的目标帧图像的解码数据所在的存储空间，确定为目标存储空间。
其中，数据查询策略包括：按照存储空间的读写速度(即消费优先级)由大至小的顺序，依次查询多级缓存空间包含的不同存储空间。
采用本公开实施例提供的上述方法，由于采用至少包括第一存储空间和第二存储空间的多级缓存空间，存储对素材轨道片段中的目标帧图像进行解码生成的解码数据，基于此，响应于视频编辑轨道上触发的预览操作，在预览到素材轨道片段中目标帧图像的时间线位置的情况下，若多级缓存空间中存储有目标帧图像的解码数据，那么就可以从多级缓存空间的目标存储空间中，获取目标帧图像的解码数据进行目标帧的渲染与预览图像显示，而无需像现有技术采用实时解码视频文件的方式来获取目标帧图像的解码数据，从而提高了对于解码数据的获取效率，进而也就提高了NLE的整体效率。
为使读者更好地理解本公开实施例提供的方法，以下结合实际场景，对上文所述的解码数据获取方法在非线性编辑系统中的具体实现过程进行介绍。
请参照说明附图2，为该解码数据获取方法在实际中的具体实现过程的示意图，该过程中各步骤的执行主体比如可以是非线性编辑系统，该过程主要包括如下步骤：
步骤31：非线性编辑系统从安装该非线性编辑系统的本地设备(比如可以是用户的个人电脑)的全局的资源管理中心，获取当前可用的专用显存的大小、共享显存的大小、内存大小；
步骤32：非线性编辑系统将专用显存、共享显存和内存设置为/指定为解码缓存池，并按照默认的资源使用比例60％，将专用显存、共享显存、内存各自当前可用的存储空间大小的60％，共同设置为解码缓存池的存储空间大小；
在本公开实施例中，专用显存、共享显存、内存的消费优先级的大小关系可以为：专用显存>共享显存>内存。
步骤33：非线性编辑系统在对视频文件进行非线性编辑的过程中，将该过程中由解码器解码得到的视频帧的解码数据，存储至解码缓存池；
其中，在将视频帧的解码数据存储至解码缓存池时，可按照消费优先级由高至低的顺序，以及依次存满解码缓存池所包含的不同类型的存储空间(包括专用显存、共享显存、内存)的规则，将视频帧的解码数据存储至解码缓存池。
其中，对于这里所说的“存满”，沿用上例，比如是指将专用显存当前可用的存储空间大小的60％存满。
步骤34：非线性编辑系统在对视频文件进行非线性编辑的过程中，接收到快退指令；
步骤35：非线性编辑系统响应于该指令，根据该指令所对应的待解码的视频帧的帧编号，对解码缓存池进行查询；
若查询到解码缓存池中存在具备所述帧编号的视频帧的解码数据，则直接从解码缓存池中获取具备所述帧编号的视频帧的解码数据，进而基于视频帧的解码数据，执行渲染出相应的预览图像并呈现的操作；其中，非线性编辑系统从解码缓存池中获取的视频帧的解码数据，作为被消费的视频帧的解码数据，将从解码缓存池中删除，然后，按照FIFO的方式，对解码缓存池中存储的视频帧的解码数据进行转移——具体可见前文实施例中相关描述，此处不再赘述。
若未查询到解码缓存池中存在具备所述帧编号的视频帧的解码数据，则执行步骤36；
步骤36：非线性编辑系统对计算设备内存中加载的视频文件进行解码，以获得具备所述帧编号的视频帧的解码数据，进而基于视频帧执行渲染出相应的图像并呈现的操作；此外，跳转到执行步骤37；
步骤37：非线性编辑系统将通过执行步骤36解码得到的具备所述帧编号的视频帧的解码数据，存储至解码缓存池。
需要说明的是，非线性编辑系统可以运行“Recyle Thread”这样的线程，该线程用于对于解码缓存池中存储的视频帧的解码数据的使用情况进行监控； 若该线程监控到解码缓存池中存储的某个/某些视频帧的解码数据自存储时刻到当前时刻内，被消费过的次数相较其他视频帧的解码数据来说最小——比如小于预设阈值Cache Threshold，那么，非线性编辑系统可以按照基于LRU算法确定出的数据淘汰规则，从解码缓存池中清除满足所述数据淘汰规则的视频帧的解码数据，即，比如淘汰被消费过的次数小于预设阈值Cache Threshold的视频帧的解码数据。
采用本公开实施例提供的上述方案，由于采用解码缓存池，来实现在非线性编辑系统对视频文件的解码与呈现过程中，对该过程中已解码得到的视频帧的解码数据进行存储，基于此，在该过程中当接收到触发解码视频文件以获得指定的视频帧的解码数据的指令时，就可以从解码缓存池中获取指定的视频帧的解码数据，而无需像现有技术必须采用实时解码视频文件的方式来获取指定的视频帧的解码数据，从而提高了该过程中对于指定的视频帧的解码数据的获取效率。
上述方案的具体实现过程，也可以进一步参考如图3所示的示意图。
出于与前文各实施例相同的发明构思，为解决现有技术中响应于视频编辑轨道上触发的预览操作，采用实时解码素材的方式来获取与目标播放位置相匹配的图像帧的解码数据，会导致解码数据获取效率较低，进而导致NLE的整体效率较低的问题，本公开实施例还提供一种解码数据获取装置。
该装置的具体结构示意图如图4所示，包括如下功能单元：
解码单元41，用于对于在视频编辑轨道上呈现的素材轨道片段，对所述素材轨道片段中的目标帧图像进行解码，生成所述目标帧图像的解码数据；
存储单元42，用于按照多级缓存策略，将所述目标帧图像的解码数据存储到多级缓存空间；其中，所述多级缓存空间至少包括第一存储空间和第二存储空间，所述第一存储空间的读写速度大于所述第二存储空间的读写速度；所述多级缓存策略包括：若第一存储空间中已缓存的帧图像的解码数据未达到所述 第一存储空间的上限则在所述第一存储空间中缓存所述目标帧图像的解码数据，若第一存储空间中已缓存的帧图像的解码数据已达到所述第一存储空间的上限则在所述第二存储空间中缓存所述目标帧图像的解码数据；
存储空间确定单元43，用于响应于在视频编辑轨道上触发的预览操作，若预览到所述素材轨道片段中所述目标帧图像的时间线位置，从所述多级缓存空间中确定目标存储空间，所述目标存储空间用于存储所述目标帧图像的解码数据；
数据获取与显示单元44，用于若所述多级缓存空间中存储有所述目标帧图像的解码数据，从所述目标存储空间中读取所述目标帧图像的解码数据，并基于所述解码数据对所述目标帧图像进行渲染并显示预览图像。
在一种可选的实施方式中，存储空间确定单元43，可以用于：
按照数据查询策略，从所述目标存储空间中查询所述目标帧图像的解码数据；
将查询到的所述目标帧图像的解码数据所在的存储空间，确定为所述目标存储空间。
其中，所述数据查询策略包括：按照存储空间的读写速度由大至小的顺序，依次查询所述多级缓存空间包含的不同存储空间。
在一种可选的实施方式中，本公开实施例提供的该装置还可以包括：
数据转移单元，用于若所述第一存储空间中存储的解码数据被消费，则，将所述第二存储空间中存储的解码数据，按照先进先出FIFO的规则，依次转移至所述第一存储空间中经释放得到的空闲存储位置。
在一种可选的实施方式中，所述第一存储空间为专用存储空间；所述第二存储空间为共享存储空间；或者，所述第一存储空间为显存；所述第二存储空间为内存。
在一种可选的实施方式中，所述第一存储空间，包括：第一子存储空间和第二子存储空间；
当所述第一存储空间为显存时，所述第一子存储空间为专用显存，所述第二子存储空间为共享显存；
所述多级缓存策略还包括：若第一子存储空间中已缓存的帧图像的解码数据已达到所述第一子存储空间的上限，则在所述第二子存储空间中缓存所述目标帧图像的解码数据。
在一种可选的实施方式中，本公开实施例提供的该装置还可以包括：
数据清除单元，用于按照基于最近最少使用LRU算法设置的数据淘汰规则，从所述多级缓存空间中清除满足所述数据淘汰规则的解码数据。
采用本公开实施例提供的上述解码数据获取装置，由于采用至少包括第一存储空间和第二存储空间的多级缓存空间，存储对素材轨道片段中的目标帧图像进行解码生成的解码数据，基于此，响应于视频编辑轨道上触发的预览操作，在预览到素材轨道片段中目标帧图像的时间线位置的情况下，若多级缓存空间中存储有目标帧图像的解码数据，那么就可以从多级缓存空间的目标存储空间中，获取目标帧图像的解码数据进行目标帧的渲染与预览图像显示，而无需像现有技术采用实时解码视频文件的方式来获取目标帧图像的解码数据，从而提高了对于解码数据的获取效率，进而也就提高了NLE的整体效率。
出于与前文各实施例相同的发明构思，本公开实施例还提供一种电子设备，用以解决现有技术中响应于视频编辑轨道上触发的预览操作，采用实时解码素材的方式来获取与目标播放位置相匹配的图像帧的解码数据，会导致解码数据获取效率较低，进而导致NLE的整体效率较低的问题。
如图5所示，所述电子设备包括：存储器51及处理器52。存储器51可被配置为存储其它各种数据以支持在电子设备上的操作。这些数据的示例包括用于在电子设备上操作的任何应用程序或方法的指令。存储器51可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器 (EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
处理器52，与存储器51耦合，用于执行存储器51中存储的程序，以用于执行本公开实施例中所述的解码数据获取方法，或执行本公开实施例中所述的多媒体数据存储方法。
处理器52在执行存储器51中的程序时，除了上面的功能之外，还可实现其它功能，具体可参见前面各实施例的描述。
进一步的，如图5所示，电子设备还包括：显示器54、通信组件53、电源组件55、音频组件56等其它组件。图5中仅示意性给出部分组件，并不意味着电子设备只包括图5所示组件。
相应地，本公开实施例还提供一种存储有计算机程序的计算机可读存储介质，所述计算机程序被计算机执行时能够实现上述各实施例提供的方法的步骤或功能。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。
最后应说明的是：以上实施例仅用以说明本公开的技术方案，而非对其限 制；尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本公开各实施例技术方案的精神和范围。

Claims (14)

  1. 一种解码数据获取方法，包括：
    对于在视频编辑轨道上呈现的素材轨道片段，对所述素材轨道片段中的目标帧图像进行解码，生成所述目标帧图像的解码数据；
    按照多级缓存策略，将所述目标帧图像的解码数据存储到多级缓存空间；其中，所述多级缓存空间至少包括第一存储空间和第二存储空间，所述第一存储空间的读写速度大于所述第二存储空间的读写速度；所述多级缓存策略包括：若第一存储空间中已缓存的帧图像的解码数据未达到所述第一存储空间的上限则在所述第一存储空间中缓存所述目标帧图像的解码数据，若第一存储空间中已缓存的帧图像的解码数据已达到所述第一存储空间的上限则在所述第二存储空间中缓存所述目标帧图像的解码数据；
    响应于在视频编辑轨道上触发的预览操作，若预览到所述素材轨道片段中所述目标帧图像的时间线位置，从所述多级缓存空间中确定目标存储空间，所述目标存储空间用于存储所述目标帧图像的解码数据；
    若所述多级缓存空间中存储有所述目标帧图像的解码数据，从所述目标存储空间中读取所述目标帧图像的解码数据，并基于所述解码数据对所述目标帧图像进行渲染并显示预览图像。
  2. 如权利要求1所述的方法，其中，从所述多级缓存空间中确定目标存储空间，包括：
    按照数据查询策略，从所述目标存储空间中查询所述目标帧图像的解码数据；
    将查询到的所述目标帧图像的解码数据所在的存储空间，确定为所述目标存储空间；
    所述数据查询策略包括：按照存储空间的读写速度由大至小的顺序，依次查询所述多级缓存空间包含的不同存储空间。
  3. 如权利要求2所述的方法，还包括：
    若所述第一存储空间中存储的解码数据被消费，则，将所述第二存储空间中存储的解码数据，按照先进先出FIFO的规则，依次转移至所述第一存储空间中经释放得到的空闲存储位置。
  4. 如权利要求1所述的方法，其中，所述第一存储空间为专用存储空间；所述第二存储空间为共享存储空间；或者，
    所述第一存储空间为显存；所述第二存储空间为内存。
  5. 如权利要求1或4所述的方法，其中，所述第一存储空间，包括：第一子存储空间和第二子存储空间；
    当所述第一存储空间为显存时，所述第一子存储空间为专用显存，所述第二子存储空间为共享显存；
    所述多级缓存策略还包括：若第一子存储空间中已缓存的帧图像的解码数据已达到所述第一子存储空间的上限，则在所述第二子存储空间中缓存所述目标帧图像的解码数据。
  6. 如权利要求1所述的方法，还包括：
    按照基于最近最少使用LRU算法设置的数据淘汰规则，从所述多级缓存空间中清除满足所述数据淘汰规则的解码数据。
  7. 一种解码数据获取装置，其中，包括：
    解码单元，用于对于在视频编辑轨道上呈现的素材轨道片段，对所述素材轨道片段中的目标帧图像进行解码，生成所述目标帧图像的解码数据；
    存储单元，用于按照多级缓存策略，将所述目标帧图像的解码数据存储到多级缓存空间；其中，所述多级缓存空间至少包括第一存储空间和第二存储空 间，所述第一存储空间的读写速度大于所述第二存储空间的读写速度；所述多级缓存策略包括：若第一存储空间中已缓存的帧图像的解码数据未达到所述第一存储空间的上限则在所述第一存储空间中缓存所述目标帧图像的解码数据，若第一存储空间中已缓存的帧图像的解码数据已达到所述第一存储空间的上限则在所述第二存储空间中缓存所述目标帧图像的解码数据；
    存储空间确定单元，用于响应于在视频编辑轨道上触发的预览操作，若预览到所述素材轨道片段中所述目标帧图像的时间线位置，从所述多级缓存空间中确定目标存储空间，所述目标存储空间用于存储所述目标帧图像的解码数据；
    数据获取与显示单元，用于若所述多级缓存空间中存储有所述目标帧图像的解码数据，从所述目标存储空间中读取所述目标帧图像的解码数据，并基于所述解码数据对所述目标帧图像进行渲染并显示预览图像。
  8. 如权利要求7所述的装置，其中，所述存储空间确定单元，用于：
    按照数据查询策略，从所述目标存储空间中查询所述目标帧图像的解码数据；
    将查询到的所述目标帧图像的解码数据所在的存储空间，确定为所述目标存储空间；
    所述数据查询策略包括：按照存储空间的读写速度由大至小的顺序，依次查询所述多级缓存空间包含的不同存储空间。
  9. 如权利要求8所述的装置，还包括：
    数据转移单元，用于若所述第一存储空间中存储的解码数据被消费，则，将所述第二存储空间中存储的解码数据，按照先进先出FIFO的规则，依次转移至所述第一存储空间中经释放得到的空闲存储位置。
  10. 如权利要求7所述的装置，其中，所述第一存储空间为专用存储空间；所述第二存储空间为共享存储空间；或者，
    所述第一存储空间为显存；所述第二存储空间为内存。
  11. 如权利要求7或10所述的装置，其中，所述第一存储空间，包括：第一子存储空间和第二子存储空间；
    当所述第一存储空间为显存时，所述第一子存储空间为专用显存，所述第二子存储空间为共享显存；
    所述多级缓存策略还包括：若第一子存储空间中已缓存的帧图像的解码数据已达到所述第一子存储空间的上限，则在所述第二子存储空间中缓存所述目标帧图像的解码数据。
  12. 如权利要求7所述的装置，还包括：
    数据清除单元，用于按照基于最近最少使用LRU算法设置的数据淘汰规则，从所述多级缓存空间中清除满足所述数据淘汰规则的解码数据。
  13. 一种电子设备，其中，包括：存储器及处理器，其中，
    所述存储器，用于存储程序；
    所述处理器，与所述存储器耦合，用于执行所述存储器中存储的所述程序，以用于执行权利要求1～6任一权项所述的解码数据获取方法。
  14. 一种存储有计算机程序的计算机可读存储介质，所述计算机程序被计算机执行时能够实现权利要求1～6任一权项所述的解码数据获取方法。
PCT/CN2024/097854 2023-06-28 2024-06-06 解码数据获取方法、装置、设备与存储介质 Ceased WO2025001808A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310776768.7A CN119233027B (zh) 2023-06-28 2023-06-28 解码数据获取方法、装置、设备与存储介质
CN202310776768.7 2023-06-28

Publications (1)

Publication Number Publication Date
WO2025001808A1 true WO2025001808A1 (zh) 2025-01-02

Family

ID=93937332

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/097854 Ceased WO2025001808A1 (zh) 2023-06-28 2024-06-06 解码数据获取方法、装置、设备与存储介质

Country Status (2)

Country Link
CN (1) CN119233027B (zh)
WO (1) WO2025001808A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104091607A (zh) * 2014-06-13 2014-10-08 北京奇艺世纪科技有限公司 一种基于ios设备的视频编辑方法及装置
JP2015090429A (ja) * 2013-11-06 2015-05-11 キヤノン株式会社 画像形成装置およびその制御方法
CN108632624A (zh) * 2017-12-18 2018-10-09 百富计算机技术（深圳）有限公司 图像数据处理方法、装置、终端设备及可读存储介质
CN111263211A (zh) * 2018-12-03 2020-06-09 深圳市炜博科技有限公司 一种缓存视频数据的方法及终端设备
CN114827722A (zh) * 2022-04-12 2022-07-29 咪咕文化科技有限公司 视频预览方法、装置、设备及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4337033B2 (ja) * 2003-08-11 2009-09-30 ソニー株式会社 編集システム及びその制御方法
CN109685712B (zh) * 2017-10-18 2023-05-23 阿里巴巴集团控股有限公司 图像缓存和使用方法及装置、终端
CN112929755B (zh) * 2021-01-21 2022-08-16 稿定（厦门）科技有限公司 进度拖动过程中的视频文件播放方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015090429A (ja) * 2013-11-06 2015-05-11 キヤノン株式会社 画像形成装置およびその制御方法
CN104091607A (zh) * 2014-06-13 2014-10-08 北京奇艺世纪科技有限公司 一种基于ios设备的视频编辑方法及装置
CN108632624A (zh) * 2017-12-18 2018-10-09 百富计算机技术（深圳）有限公司 图像数据处理方法、装置、终端设备及可读存储介质
CN111263211A (zh) * 2018-12-03 2020-06-09 深圳市炜博科技有限公司 一种缓存视频数据的方法及终端设备
CN114827722A (zh) * 2022-04-12 2022-07-29 咪咕文化科技有限公司 视频预览方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN119233027B (zh) 2025-10-03
CN119233027A (zh) 2024-12-31

Similar Documents

Publication Publication Date Title
US7673238B2 (en) Portable media device with video acceleration capabilities
US9641638B2 (en) Application-guided bandwidth-managed caching
EP1733394A1 (en) System and method for supporting improved trick mode performance for disc-based multimedia content
CN110418186A (zh) 音视频播放方法、装置、计算机设备和存储介质
WO2013075342A1 (zh) 一种视频处理的方法及装置
CN110704184A (zh) 一种应用内存优化方法、装置和移动终端
WO2021052130A1 (zh) 视频处理方法、装置、设备及计算机可读存储介质
CN115209216A (zh) 视频的播放方法、装置及电子设备
JP6137980B2 (ja) 記録装置およびその制御方法
JP7769112B2 (ja) ビデオ処理方法、装置、電子機器及び読み取り可能な記憶媒体
KR20140004240A (ko) 이미지 검색 방법, 시스템 및 컴퓨터 기록매체
CN106940722B (zh) 一种图片显示方法及装置
WO2024146330A1 (zh) 视频存储方法和视频播放方法
WO2025001808A1 (zh) 解码数据获取方法、装置、设备与存储介质
CN116257165A (zh) 视频处理方法、装置、终端及存储介质
US20130103907A1 (en) Memory management device, memory management method, control program, and recording medium
CN111356012A (zh) 视频预览方法和装置
WO2024041514A1 (zh) 视频播放方法、装置和电子设备
US7680392B2 (en) Streaming contents recording method of mobile communication terminal
CN109996022A (zh) 监控存储扩容方法及视频监控系统
WO2020024197A1 (zh) 视频的处理方法、装置及计算机可读存储介质
US12542952B1 (en) Methods and systems for synchronizing rendering of video frames during scrubbing
CN110753265B (zh) 数据处理方法、装置和电子设备
WO2024104267A1 (zh) 一种多媒体文件的播放控制方法、装置、设备及存储介质
JP2004078704A (ja) 映像音声記録再生装置とそのファイル読み書き方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24830454

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE