WO2019015395A1 - 内存管理方法、装置、电子设备及存储介质 - Google Patents

内存管理方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2019015395A1
WO2019015395A1 PCT/CN2018/088395 CN2018088395W WO2019015395A1 WO 2019015395 A1 WO2019015395 A1 WO 2019015395A1 CN 2018088395 W CN2018088395 W CN 2018088395W WO 2019015395 A1 WO2019015395 A1 WO 2019015395A1
Authority
WO
WIPO (PCT)
Prior art keywords
memory
memory address
video frame
frame data
multiplexing queue
Prior art date
Application number
PCT/CN2018/088395
Other languages
English (en)
French (fr)
Inventor
徐峰
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Priority to EP18834482.4A priority Critical patent/EP3640807B1/en
Priority to EP22164063.4A priority patent/EP4040299B1/en
Priority to US16/631,304 priority patent/US11604726B2/en
Publication of WO2019015395A1 publication Critical patent/WO2019015395A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0207—Addressing or allocation; Relocation with multidimensional access, e.g. row/column, matrix
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
    • G06F12/0646—Configuration or reconfiguration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4334—Recording operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4335—Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1041—Resource optimization
    • G06F2212/1044—Space efficiency improvement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/25—Using a specific main memory architecture
    • G06F2212/251—Local memory within processor subsystem
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements

Definitions

  • the present application relates to the field of network technologies, and in particular, to a memory management method, apparatus, electronic device, and storage medium.
  • the terminal can play video on the webpage through the browser.
  • the terminal needs to manage the memory occupied by the browser, for example, decoding the video stream and allocating memory for the browser, and reclaiming the allocated memory.
  • the video stream is generally decoded in units of frames.
  • the terminal needs to allocate a certain amount of memory space to store the video stream.
  • Each frame of the decoded video data is used to subsequently render each frame of the decoded video data to present a video picture at the terminal.
  • the allocation of a certain size of the memory space refers to the allocation of the memory address, and the terminal periodically collects the memory address that has been allocated.
  • the embodiment of the present invention provides a memory management method, device, electronic device, and storage medium, which can solve the problem of large memory usage and low memory utilization.
  • the technical solution is as follows:
  • a memory management method comprising:
  • the first memory address is added to the memory multiplexing queue.
  • the memory multiplex queue is configured to allocate a first memory address to the video frame data, including:
  • the second memory address is obtained from the memory multiplex queue according to the data amount of the video frame data, and the memory space indicated by the second memory address is not less than The amount of data of the video frame data;
  • the method further includes:
  • the second memory address is not obtained from the memory multiplexing queue, obtain a third memory address from a memory address other than the memory multiplexing queue, where the memory space indicated by the third memory address is not less than The amount of data of the video frame data;
  • the third memory address is taken as the first memory address.
  • the memory multiplex queue is configured to allocate a first memory address to the video frame data, including:
  • the memory multiplexing queue does not include a memory address, obtaining a fourth memory address from a memory address other than the memory multiplexing queue;
  • the fourth memory address is taken as the first memory address.
  • the method further includes:
  • a step of allocating a first memory address for the video frame data based on the memory multiplexing queue is performed.
  • the method further includes:
  • the video frame data is decoded by the sub-thread.
  • the method further includes:
  • adding the first memory address to the memory multiplexing queue includes:
  • the first memory address is added to the memory multiplexing queue by the main thread.
  • a memory management device comprising:
  • An allocation module configured to allocate a first memory address to the video frame data based on the memory multiplexing queue, where the memory multiplexing queue is used to record a memory address where the video frame data that has been rendered is located;
  • a storage module configured to store the video frame data to a memory space indicated by the first memory address
  • a adding module configured to add the first memory address to the memory multiplexing queue after rendering based on the video frame data.
  • the allocating module includes:
  • a first acquiring unit configured to obtain a second memory address from the memory multiplexing queue according to the data amount of the video frame data, if the memory multiplexing queue includes a memory address, the second memory address
  • the indicated memory space is not less than the amount of data of the video frame data
  • a first allocation unit configured to: move the second memory address out of the memory multiplexing queue if the second memory address is obtained from the memory multiplexing queue, and use the second memory address as a Said first memory address.
  • the allocating module further includes:
  • a second acquiring unit configured to obtain a third memory address from a memory address other than the memory multiplexing queue if the second memory address is not obtained from the memory multiplexing queue, where the third memory address is The indicated memory space is not less than the amount of data of the video frame data;
  • a second allocation unit configured to use the third memory address as the first memory address.
  • the allocating module further includes:
  • a third obtaining unit configured to obtain a fourth memory address from a memory address other than the memory multiplexing queue if the memory multiplexing queue does not include a memory address
  • a third allocation unit configured to use the fourth memory address as the first memory address.
  • the allocating module is configured to acquire a resolution of the video stream when receiving a video stream, and perform memory-based according to a data amount of the video frame data indicated by the resolution a multiplexing queue for assigning a first memory address to video frame data;
  • the allocating module is configured to perform, during the process of decoding the received video stream, a step of allocating a first memory address to the video frame data based on the memory multiplexing queue when the video frame data is acquired.
  • the device further includes:
  • a startup module configured to start a sub-thread through a specified interface of the multimedia browsing application
  • a decoding module configured to decode the video frame data by using the sub-thread.
  • the device further includes:
  • a rendering module configured to render the video frame data by using a main thread of the multimedia browsing application
  • the adding module is configured to: pass, by the main thread, the first memory address to a child thread of the multimedia browsing application; and add, by the child thread, the first memory address to the memory In the multiplexing queue; or, the first memory address is added to the memory multiplexing queue by the main thread.
  • an electronic device in a third aspect, includes a processor and a memory, a memory for storing a computer program, and a processor for executing a computer program stored on the memory to implement any of the first aspects The method steps described in the implementation.
  • a fourth aspect provides a computer readable storage medium having stored therein a computer program, the computer program being executed by a processor to implement the method steps of any one of the implementations of the first aspect .
  • the memory allocation can be performed according to the memory multiplexing queue, so that the allocated memory address can be reused, during the process of playing the video. There is no need to open up a memory space for each video frame data, which reduces memory usage and improves memory utilization.
  • FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application.
  • FIG. 2 is a flowchart of a memory management method provided by an embodiment of the present application.
  • FIG. 3 is a flowchart of a memory management method provided by an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of a memory management provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a memory multiplexing queue provided by an embodiment of the present application.
  • FIG. 6 is a flowchart of a memory management method provided by an embodiment of the present application.
  • FIG. 7 is a flowchart of a memory management method provided by an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a memory management provided by an embodiment of the present application.
  • FIG. 9 is a flowchart of a memory management method provided by an embodiment of the present application.
  • FIG. 10 is a block diagram of a memory management apparatus according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a terminal according to an embodiment of the present disclosure.
  • FIG. 12 is a schematic structural diagram of an electronic device 600 according to an embodiment of the present application.
  • FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application, including a terminal 11, a server 12, and a monitoring device 13.
  • the terminal 11 is capable of playing a video in a webpage
  • the server 12 is configured to provide a multimedia service to the terminal 11, such as providing a video stream
  • the monitoring device 13 has a video collection function.
  • the multimedia service functionality of server 12 is integrated into monitoring device 13. After integrating the functions of the server 12, the monitoring device 12 can also provide multimedia services to the terminal 11.
  • the terminal 11 can directly interact with the monitoring device 13 via the Internet.
  • a browser is installed on the terminal 11, and the monitoring device 13 is accessed by inputting the network address of the monitoring device 13 in the browser to acquire the video stream collected by the monitoring device 13 in real time.
  • the monitoring device 13 can also be provided with a memory (such as a hard disk storage), and the monitoring device 13 can store the collected video data into the memory, so that the terminal 11 can acquire the already stored video data, thereby performing playback of the monitoring video, and the like. .
  • a memory such as a hard disk storage
  • the server 12 acts as a separate entity, and the terminal 11 and the monitoring device 13 can interact with the server 12 via the Internet, respectively.
  • the monitoring device 13 can send the collected monitoring video stream to the server 12 in real time, and the server 12 forwards the video stream to the terminal 13 as a relay to perform real-time monitoring through the terminal.
  • a browser is installed on the terminal 11, and a real-time forwarded video stream or other multimedia data, such as image data, audio and video data, etc., is acquired from the server 12 by inputting the access address of the server 12 in the browser.
  • the server 12 can also store the monitored video stream from the monitoring device 13 for storage, for example, in the hard disk storage, so that the terminal 11 can also obtain the stored monitoring video data from the server for monitoring and playback.
  • the steps performed by the servers involved in the subsequent steps may refer to the steps performed by the monitoring device 13.
  • FIG. 2 is a flowchart of a memory management method according to an embodiment of the present application.
  • the method process provided by the embodiment of the present application includes:
  • the first memory address is allocated to the video frame data based on the memory multiplexing queue, where the memory multiplexing queue is used to record the memory address of the video frame data that has been rendered.
  • the memory-multiplexed queue allocates a first memory address to the video frame data, including:
  • the second memory address is obtained from the memory multiplex queue according to the data amount of the video frame data, and the memory space indicated by the second memory address is not less than the video frame data. Amount of data;
  • the second memory address is obtained from the memory multiplexing queue, the second memory address is moved out of the memory multiplexing queue, and the second memory address is used as the first memory address.
  • the method further includes:
  • the second memory address is not obtained from the memory multiplexing queue, obtain a third memory address from a memory address other than the memory multiplexing queue, where the memory space indicated by the third memory address is not less than the video frame data.
  • the amount of data
  • the third memory address is used as the first memory address.
  • the memory-multiplexed queue allocates a first memory address to the video frame data, including:
  • the memory multiplexing queue does not include a memory address, obtaining a fourth memory address from a memory address other than the memory multiplexing queue;
  • the fourth memory address is taken as the first memory address.
  • the method further includes:
  • a step of allocating a first memory address for the video frame data based on the memory multiplexing queue is performed.
  • the method provided by the embodiment of the present application adds a memory address of the video frame data that has been rendered to the memory multiplexing queue, and then performs memory allocation according to the memory multiplexing queue, so that the allocated memory address can be duplicated.
  • the process of playing video there is no need to open up a memory space for each video frame data, which reduces the memory usage and improves the memory utilization.
  • the terminal is installed with a multimedia browsing application
  • the multimedia browsing application refers to an application that displays multimedia data in a webpage
  • the multimedia browsing application may be a browser
  • the browser may be a plug-inless browsing.
  • You can play videos on a web page without installing any video plug-ins.
  • the terminal accesses the monitoring device through the browser
  • the terminal can obtain a video decoding program through the monitoring device, and decode the received video stream by executing the video decoding program, so as to play the video in the webpage.
  • the browser can also perform video decoding by installing a video plug-in to provide the function of playing video in a webpage.
  • the terminal After the terminal obtains the video stream from the server through the browser, the terminal decodes each frame of data indicated by the video stream.
  • the decoded video data of each frame is referred to as video frame data
  • the terminal needs to allocate memory for the video frame data, that is, allocate a memory address for the video frame data to the memory indicated by the memory address.
  • the video frame data is stored in the space.
  • the process of allocating memory for the video frame data by the terminal is the process of applying for memory by the multimedia browsing application for the video frame data.
  • the terminal allocates memory, and the execution is performed.
  • the main body can replace the terminal with a multimedia browsing application.
  • the following is an example in which the multimedia browsing application is a browser.
  • the manner in which the terminal allocates memory may include the following two types according to different memory allocation timings:
  • the terminal After receiving the video stream through the browser, the terminal first decodes the video stream, and then performs memory allocation according to the decoded video frame data, and the allocated memory address may be a memory address removed from the memory multiplexing queue. After the video frame data is rendered, the allocated memory address can be added to the memory multiplexing queue to implement multiplexing of the allocated memory addresses. The process is detailed in the embodiment provided in FIG. The memory multiplexing queue is used to record the memory address of the video data that has been rendered.
  • the terminal After receiving the video stream through the browser, the terminal allocates memory in advance, and the pre-allocated memory address may be a memory address removed from the memory multiplexing queue. After decoding the video frame data, the video frame data is stored to The memory space indicated by the pre-allocated memory address may be added to the memory multiplexing queue after the video frame data is rendered, so as to implement multiplexing of the allocated memory address, the process See the embodiment provided in Figure 7 for details.
  • the following embodiment uses a decoded video frame data as an example to introduce a memory management process of the terminal, where the memory address allocated for the video frame data is simply referred to as a first memory. address.
  • FIG. 3 is a flowchart of a memory management method according to an embodiment of the present application. Referring to FIG. 3, the method process provided by the embodiment of the present application includes:
  • step 202 In the process of decoding the received video stream, when the decoded video frame data is obtained, it is detected whether a memory address is included in the memory multiplexing queue. If yes, go to step 202, otherwise go to step 205. .
  • the terminal after acquiring the video stream from the server, the terminal decodes each frame of video data indicated by the video stream.
  • the video stream includes a plurality of data packets conforming to the preset format, each data packet corresponding to the frame identifier, and the data packet having the same frame identifier constitutes one frame of video data.
  • the terminal may determine one frame of video data according to the frame identifier of the data packet, and after all the video data of the one frame is received, decode the video data of the one frame to obtain video frame data.
  • the preset format may be preset or modified according to the actual video transmission requirement, which is not limited in this embodiment.
  • the terminal can receive the video stream from the server in the main thread. Since the video decoding process is time-consuming, in order not to block the main thread, the web worker technology can be used to create an asynchronous thread different from the main thread, and the asynchronous thread executes the asynchronous thread.
  • the process of decoding which can also be called a child thread.
  • the terminal may start a sub-thread through a specified interface of the browser, and decode the video frame data through the sub-thread.
  • the specified interface may be configured or modified in advance.
  • the specified interface may be a web worker interface of the browser, which is not limited in this embodiment. Referring to FIG.
  • a flow chart of a memory management is shown, in which a terminal invokes a WebSocket interface through a main thread to create a data transmission channel, and obtains a video stream from a monitoring device or a server through the data transmission channel (FIG. 4). In real time stream). After the main thread acquires the video stream, the video stream is sent to the child thread. For example, the terminal sends the video stream to the child thread by calling a thread interaction interface (postMessage).
  • postMessage thread interaction interface
  • the memory multiplexing queue is used to record the memory address of the video data that has been rendered. It should be noted that after the video frame data is decoded, the terminal needs to render the video frame data to display a video picture corresponding to the video frame data, thereby implementing video playback. In the embodiment of the present application, the memory address occupied by the video frame data that has been rendered is added to the memory multiplexing queue, so that the memory space indicated by the memory address is subsequently reused.
  • the memory multiplexing queue may be initially empty; or one or more memory addresses capable of satisfying the data amount of the video frame data may be preset in the memory multiplexing queue, that is, the video is The memory address corresponding to the resolution of the stream.
  • the memory address in the embodiment of the present application refers to a memory address corresponding to a memory block.
  • the memory address corresponding to the memory block is the 16-byte address.
  • the logic layer can represent the allocated memory block through an array. Accordingly, the memory multiplexing queue is an array queue, and each array can be called an element of the queue.
  • the terminal may detect whether the memory multiplexing queue is empty (whether or contains a memory address), if it is not empty, determine that the memory multiplexing queue includes a memory address, and perform step 202; If it is empty, it is determined that the memory multiplex queue does not contain a memory address, and step 205 is performed.
  • the process of detecting whether the memory multiplexing queue is empty may be: obtaining a first element from a queue header of the memory multiplexing queue, and determining that the memory multiplexing queue is not empty if the first element is not empty; If the first element is empty, then the memory reuse queue is determined to be empty.
  • the memory multiplexing queue includes a memory address, obtain a second memory address from the memory multiplexing queue according to the data amount of the video frame data.
  • the terminal may determine the data amount of the video frame data. For example, the terminal may determine the frame length, the frame width, and the image format of the video frame data to determine the video frame data.
  • the frame length of the video frame data refers to the number of pixels included in each line of the image indicated by the video frame data
  • the frame width of the video frame data refers to the number of pixels included in each column of the image indicated by the video frame data. number.
  • the image format of the video frame data may be an RGB image format or a YUV image format, which is not limited in this embodiment. It should be noted that, by default, the image format of the video frame data can be set to the YUV image format.
  • the terminal can perform image format conversion to convert the video frame data from the YUV image format to the RGB image.
  • the terminal may also perform image format processing in the decoding process to directly output video frame data that satisfies the rendering required image format.
  • the frame length and the frame width of the video frame data may be carried in the video frame data
  • the image format may be a default image format of the server and the terminal, and configured by the server to the terminal in advance; of course, the image format may also be based on The actual requirement change is notified to the terminal by being carried in the video frame data, which is not limited in this embodiment.
  • the terminal may obtain the frame length, the frame width, and the image format from the video frame data, thereby determining the data amount of the video frame data.
  • the data amount of the video frame data is width ⁇ heigth ⁇ 3Bit
  • the data amount of the video frame data is width ⁇ heigth ⁇ 1.5Bit.
  • width represents the frame length of the video frame data
  • heigth represents the frame width of the video frame data.
  • the terminal may compare the memory space indicated by the memory address with the video frame data according to the data amount of the video frame data, and obtain the second memory from the memory multiplexing queue. address.
  • the memory space indicated by the second memory address should be no less than the amount of data of the video frame data.
  • the process of obtaining the second memory address from the memory multiplexing queue according to the data volume of the video frame data may be: the terminal acquires one from the memory multiplexing queue according to the order of the memory addresses in the memory multiplexing queue. a memory address, detecting whether the memory space indicated by the memory address is not less than the data amount of the video frame data, and if yes, determining the memory address as the second memory address; if not, acquiring the memory from the memory multiplexing queue A memory address until the second memory address is obtained.
  • the memory space indicated by the multiple memory addresses included in the memory multiplexing queue should be the same if the memory multiplexing queue
  • the size of the memory space indicated by the first memory address is smaller than the data volume of the video frame data, and the memory space indicated by the plurality of memory addresses acquired subsequently is likely to be smaller than the data amount of the video frame data. Therefore, in order to improve processing Efficiency, when the memory space indicated by the first memory address obtained by the terminal in the memory multiplexing queue is smaller than the data volume of the video frame data, the memory address is moved out of the memory multiplexing queue, and the terminal no longer acquires the memory multiplexing queue.
  • the next memory address in the process directly performs the process of obtaining the third memory address from the memory address other than the memory multiplexing queue. For details, see step 204.
  • the terminal When the terminal detects that the memory space indicated by the currently acquired memory address is smaller than the data amount of the video frame data, the terminal reclaims the memory address, that is, removes the memory address from the memory multiplexing queue, and clears the memory. The data stored in the memory space indicated by the address, and the memory address is released. Since the resolution of the video stream is stable within a certain period of time, when the memory space indicated by the memory address is smaller than the data amount of the video frame data, it indicates that the memory address is not used within a certain period of time, and is timely recovered. Less memory addresses reduce memory usage and increase memory utilization.
  • the second memory address is obtained from the memory multiplexing queue, move the second memory address out of the memory multiplexing queue, and use the second memory address as the first memory address allocated for the video frame data.
  • the second memory address is moved out of the memory multiplexing queue, and the second memory address is used as the first memory address allocated for the video frame data to store the video frame data.
  • removing a memory address from the memory multiplexing queue means deleting the memory address from the memory multiplexing queue.
  • the second memory address is not obtained from the memory multiplexing queue, obtain a third memory address from a memory address other than the memory multiplexing queue, and use the third memory address as the allocated first memory address.
  • the terminal may not obtain the second memory address from the memory multiplexing queue.
  • the memory space indicated by the memory address in the memory multiplexing queue is not large enough, that is, there is no memory space in the memory multiplexing queue.
  • the terminal may re-allocate a memory address for the video frame data, that is, the terminal acquires a third memory address in the unallocated memory address other than the memory multiplexing queue, and the third memory address is used as the The first memory address of the video frame data allocation.
  • the terminal can use the new ArrayBuffer() statement to create an array that satisfies the size of the video frame data, and the memory address indicated by the array is the third memory address.
  • the memory space indicated by the third memory address is not less than the data amount of the video frame data.
  • the memory multiplexing queue does not include a memory address, obtain a fourth memory address from a memory address other than the memory multiplexing queue, and use the fourth memory address as the allocated first memory address.
  • the memory frame data is reassigned a memory address, that is, the fourth memory address is obtained from an unallocated address other than the multiplexing queue.
  • the process of allocating the fourth memory address is the same as the process of allocating the third memory address in step 203, and details are not described herein.
  • the process of decoding a video stream in this embodiment the process of allocating memory after decoding, and the process of storing decoded video frame data may all be processed in a sub-thread.
  • the terminal After the terminal stores the video frame data in the allocated memory space, the terminal may render the video frame data in the main thread to display the video picture indicated by the video frame data.
  • the process of rendering the video frame data by the terminal may be: the terminal maintains a rendering queue in the main thread, and after the terminal stores the video frame data, the first memory address where the video frame data is located is sent to the main thread through the address push message. To add the first memory address to the render queue at the main thread.
  • the address push message carries the first memory address.
  • the terminal renders the video frame data through the main thread of the multimedia browsing application.
  • the terminal periodically scans the rendering queue.
  • the terminal requests video frame data (requestAnimationFrame) to the rendering queue, and sequentially obtains the video indicated by the memory address according to the order of the memory addresses in the rendering queue.
  • Frame data and render the acquired video frame data.
  • the rendering queue may be empty when the terminal reaches the scanning rendering queue cycle. The cycle continues to scan.
  • the scan queue is not empty, it is rendered according to the video frame data indicated by the memory address in the render queue.
  • the terminal can render using a preset drawing standard and display the rendered image as a container with a canvas.
  • the preset drawing standard may be a WebGL (Web Graphics Library) drawing standard, which is not limited in this embodiment.
  • the memory space occupied by the video frame data that has been rendered is reused in the embodiment of the present application, that is, the video frame data that has been rendered is located.
  • the memory address is reclaimed into the memory multiplexing queue so that the memory address can be subsequently reassigned to other unrendered video frame data.
  • the terminal may add the first memory address to the memory multiplexing queue in two ways: in an implementation manner, the terminal may pass the first memory address to the child thread of the multimedia browsing application through the main thread, and pass the The child thread adds the first memory address to the memory multiplexing queue; in another embodiment, the terminal directly adds the first memory address to the memory multiplexing queue through the main thread.
  • the terminal may send the first memory address to the sub-thread by calling a thread interaction interface (postMessage), and the first memory address is added to the memory complex by the sub-thread execution. The steps to use the queue.
  • the number of memory addresses that the terminal has been in the occupied state can reach a stable state, that is, when the terminal is always When the number of memory addresses in the occupied state reaches a certain number, the storage requirement of the video frame data that has been decoded but not yet rendered can be satisfied.
  • the number of memory addresses included in the memory multiplexing queue is also relatively stable, that is, when a memory address is removed from the memory multiplexing queue, a memory is also added to the memory multiplexing queue. address.
  • the memory multiplexing queue contains five memory addresses to meet the playback requirements. Because in the process of playing video, the terminal only needs to open a small number of memory addresses, and the video can be played in the webpage by reusing the small number of memory addresses, thereby greatly reducing the memory usage and improving the memory utilization.
  • the memory multiplexing queue includes a queue header and a queue tail, and the terminal removes the memory address from the queue header to perform Memory allocation, the process may include the processes corresponding to the above steps 201 to 205.
  • the terminal adds the memory address of the video frame data that has been rendered from the tail of the queue, and the process may include the process corresponding to step 206 to step 207.
  • the method provided by the embodiment of the present application adds a memory address of the video frame data that has been rendered to the memory multiplexing queue, and then performs memory allocation according to the memory multiplexing queue, so that the allocated memory address can be duplicated.
  • the process of playing video there is no need to open up a memory space for each video frame data, which reduces the memory usage and improves the memory utilization.
  • the memory is allocated to the video frame data, so that the terminal can dynamically allocate a suitable memory space for the video frame data according to the actual data amount of the video frame data, thereby improving The flexibility of memory allocation.
  • the host of the memory management in the embodiment of the present application is a multimedia browsing application.
  • the multimedia browsing application uses the multimedia browsing application as a browser, and the memory management method described in FIG. 3 above is used.
  • the detailed process is shown in Figure 6:
  • the browser when receiving video frame data from the server, the browser starts the sub-thread through the specified interface, and the video frame data is decoded by the sub-thread.
  • the browser acquires a video stream from the server by using the input IP address, where the video stream includes multiple video frame data, and the browser decodes each received frame data, so as to subsequently display to the user through rendering.
  • Video content
  • the browser receives the video stream from the server in the main thread. Since the video decoding process is time-consuming, in order not to block the main thread, the web worker technology can be used to create an asynchronous thread different from the main thread, and the asynchronous thread executes the asynchronous thread.
  • the process of decoding which can also be called a child thread.
  • the child thread can be started through a specified interface of the browser, and the video frame data is decoded by the child thread.
  • the specified interface may be configured or modified in advance.
  • the specified interface may be a web worker interface of the browser, which is not limited in this embodiment.
  • the flowchart of the corresponding memory management can be seen in Figure 4.
  • the video stream includes a plurality of data packets conforming to the preset format, each data packet corresponding to the frame identifier, and the data packet having the same frame identifier constitutes one frame of video data.
  • the terminal may determine one frame of video data according to the frame identifier of the data packet, and after all the video data of the one frame is received, decode the video data of the one frame to obtain video frame data.
  • the preset format may be preset or modified according to the actual video transmission requirement, which is not limited in this embodiment.
  • the browser detects whether the memory multiplex queue contains the memory address. If yes, go to step 2013; otherwise, go to step 2016.
  • the memory multiplexing queue is used to record the memory address of the video data that has been rendered. After the video frame data is decoded, the browser needs to render the video frame data to display the video picture corresponding to the video frame data, thereby implementing video playback. In the embodiment of the present application, the browser may add the memory address occupied by the already rendered video frame data to the memory multiplexing queue, so as to subsequently reuse the memory space indicated by the memory address.
  • the memory multiplexing queue may be initially empty; or one or more memory addresses capable of satisfying the data amount of the video frame data, that is, the video stream, may be preset in the memory multiplexing queue.
  • the resolution corresponds to the memory address.
  • the memory address in the embodiment of the present application refers to a memory address corresponding to a memory block.
  • the memory address corresponding to the memory block is the 16-byte address.
  • the logic layer can represent the allocated memory block through an array. Accordingly, the memory multiplexing queue is an array queue, and each array can be called an element of the queue.
  • the browser can detect whether the memory multiplexing queue is empty. If it is not empty, determine that the memory multiplexing queue contains a memory address, and perform step 2013; if it is empty, determine that the memory multiplexing queue does not contain the memory. Address, go to step 2016.
  • the process of detecting whether the memory multiplexing queue is empty may be: the browser obtains the first element from the queue header of the memory multiplexing queue, and if the first element is not empty, determining that the memory multiplexing queue is not Empty; if the first element is empty, then the memory reuse queue is determined to be empty.
  • the browser obtains the second memory address from the memory multiplexing queue according to the data amount of the video frame data.
  • the browser may determine the data amount of the video frame data. For example, the browser may determine the frame length of the video frame data, the frame width, and the image format of the video frame data to determine the video. The amount of data of the frame data.
  • the explanation of the frame length of the video frame data, the frame width, and the image format of the video frame data is the same as the corresponding explanation in step 202, and details are not described herein.
  • the frame length and frame width of the video frame data may be carried in the video frame data
  • the image format may be a default image format of the server and the browser, and configured by the server to the browser in advance; of course, the image format is also It can be notified to the browser by being carried in the video frame data according to the actual needs, which is not limited in this embodiment.
  • the browser After decoding the video stream to obtain the video frame data, the browser can obtain the frame length, the frame width, and the image format from the video frame data, thereby determining the data amount of the video frame data.
  • the data amount of the video frame data is width ⁇ heigth ⁇ 3Bit
  • the data amount of the video frame data is width ⁇ heigth ⁇ 1.5Bit.
  • width represents the frame length of the video frame data
  • heigth represents the frame width of the video frame data.
  • the browser can obtain the second memory address from the memory multiplexing queue according to the data amount of the video frame data.
  • the memory space indicated by the second memory address should be no less than the amount of data of the video frame data.
  • the process of obtaining the second memory address from the memory multiplexing queue according to the data amount of the video frame data may be: the browser multiplexes the queue from the memory according to the order of the memory addresses in the memory multiplexing queue. Obtaining a memory address, detecting whether the memory space indicated by the memory address is not less than the data amount of the video frame data, and if yes, determining the memory address as the second memory address; if not, from the memory multiplexing queue Get the next memory address until the second memory address is obtained.
  • the memory space indicated by the multiple memory addresses included in the memory multiplexing queue should be the same if the memory multiplexing queue
  • the size of the memory space indicated by the first memory address is smaller than the data volume of the video frame data, and the memory space indicated by the plurality of memory addresses acquired subsequently is likely to be smaller than the data amount of the video frame data. Therefore, in order to improve processing Efficiency, when the memory space indicated by the first memory address obtained by the browser in the memory multiplexing queue is smaller than the data volume of the video frame data, the memory address is moved out of the memory multiplexing queue, and the browser no longer obtains the memory complex.
  • the process of obtaining the third memory address from the memory address other than the memory multiplexing queue is directly executed by using the next memory address in the queue. For details, see step 2015.
  • the browser detects that the memory space indicated by the currently obtained memory address is smaller than the data amount of the video frame data, and the browser reclaims the memory address, that is, removes the memory address from the memory multiplexing queue, and clears
  • the memory address indicates the data stored in the memory space and releases the memory address. Since the resolution of the video stream is stable within a certain period of time, when the memory space indicated by the memory address is smaller than the data amount of the video frame data, it indicates that the memory address is not used within a certain period of time, and is timely recovered. Less memory addresses reduce memory usage and increase memory utilization.
  • the browser moves the second memory address out of the memory multiplexing queue, and uses the second memory address as the first memory address allocated for the video frame data.
  • the second memory address is moved out of the memory multiplexing queue, and the second memory address is used as the first memory address allocated for the video frame data to store the video frame data.
  • removing a memory address from the memory multiplexing queue means deleting the memory address from the memory multiplexing queue.
  • the browser obtains the third memory address from the memory address other than the memory multiplexing queue, and uses the third memory address as the allocated first memory address.
  • the browser may also fail to obtain the second memory address from the memory multiplexing queue.
  • the memory multiplexing queue does not have a memory address whose memory space is not less than the data amount of the video frame data.
  • the browser can reallocate a memory address for the video frame data, that is, the browser obtains the third memory address in the unallocated memory address outside the memory multiplexing queue, and the third memory address is used as the third memory address.
  • the first memory address assigned to the video frame data For example, the browser can use the new ArrayBuffer() statement to create an array that satisfies the size of the video frame data, and the memory address indicated by the array is the third memory address.
  • the memory space indicated by the third memory address is not less than the data amount of the video frame data.
  • the browser obtains the fourth memory address from the memory address other than the memory multiplexing queue, and uses the fourth memory address as the allocated first memory address.
  • the browser When the browser detects that the memory multiplexing queue does not contain a memory address, it reassigns a memory address to the video frame data, that is, obtains a fourth memory address from an unallocated address other than the multiplexing queue.
  • the process of allocating the fourth memory address is the same as the process of allocating the third memory address in step 2014, and details are not described herein.
  • the browser stores the video frame data to the memory space indicated by the first memory address.
  • the process of decoding a video stream in this embodiment the process of allocating memory after decoding, and the process of storing decoded video frame data may all be processed in a sub-thread.
  • the browser After the browser stores the video frame data in the allocated memory space, the browser can render the video frame data in the main thread to display the video picture indicated by the video frame data.
  • the browser renders the video frame data by using the main thread.
  • the process may be: the browser maintains a rendering queue in the main thread, and after the browser stores the video frame data, the first frame of the video frame data is sent by the address push message.
  • the memory address is sent to the main thread to add the first memory address to the render queue at the main thread.
  • the address push message carries the first memory address.
  • the browser periodically scans the rendering queue.
  • the browser requests video frame data (requestAnimationFrame) from the rendering queue, and sequentially obtains the memory address according to the order of the memory addresses in the rendering queue.
  • Video frame data and rendering of the acquired video frame data due to the difference between the speed at which the browser decodes the video stream and the speed of rendering, when the browser reaches the cycle of scanning the rendering queue, the rendering queue may be empty.
  • the browser does not perform the rendering process, and waits until The scanning continues in the next scanning cycle.
  • the rendering queue is not empty, the rendering is performed according to the video frame data indicated by the memory address in the rendering queue.
  • a browser can render using a preset drawing standard and display the rendered image in a canvas.
  • the preset drawing standard can be a WebGL drawing standard, which is not limited in this embodiment.
  • the browser After rendering based on the video frame data, the browser adds the first memory address to the memory multiplexing queue.
  • the browser can reuse the memory space occupied by the video frame data that has been rendered, that is, the browser has already rendered the memory.
  • the memory address of the video frame data is recycled to the memory multiplexing queue, so that the memory address is subsequently allocated to other unrendered video frame data.
  • the browser can add the first memory address to the memory multiplexing queue in two ways: in one embodiment, the browser can pass the first memory address to the child thread of the multimedia browsing application through the main thread, and The first memory address is added to the memory multiplexing queue by the sub-thread; in another embodiment, the browser directly adds the first memory address to the memory multiplexing queue through the main thread.
  • the browser may send the first memory address to the sub-thread by calling a thread interaction interface (postMessage), and add the first memory address to the memory through the sub-thread execution. The step of multiplexing the queue.
  • the number of memory addresses that the browser has been in the occupied state can reach a stable state, that is, when browsing When the number of memory addresses that have been in the occupied state reaches a certain number, the storage requirement of the video frame data that has been decoded but not yet rendered can be satisfied.
  • the number of memory addresses included in the memory multiplexing queue is also relatively stable, that is, when a memory address is removed from the memory multiplexing queue, a memory is also added to the memory multiplexing queue. address.
  • the memory multiplexing queue contains five memory addresses to meet the playback requirements. In the process of playing video, the browser only needs to open a small amount of memory address, and the video can be played in the webpage by reusing the small amount of memory address, which greatly reduces the memory usage and improves the memory utilization.
  • the principle of the memory multiplexing queue is the same as that of the memory multiplexing queue in step 207, and is not described here.
  • the browser adds the memory address of the video frame data that has been rendered to the memory multiplexing queue, and then performs memory allocation according to the memory multiplexing queue, so that the allocated memory address can be allocated. Reusable, in the process of playing video in the browser, there is no need to open a memory space for each video frame data, which reduces the memory usage of the browser and improves the memory utilization of the browser.
  • the embodiment provided in FIG. 3 is that after the video stream is decoded, the memory is allocated according to the decoded video frame data.
  • the terminal can also perform memory pre-allocation after receiving the video stream, because the video stream is decoded.
  • the size of the decoded video frame data cannot be known before, so the resolution of the video stream needs to be known in advance, and the memory pre-allocation according to the video frame data indicated by the resolution of the video stream can ensure the pre-allocation of the appropriate memory size. See the embodiment provided in Figure 7.
  • FIG. 7 is a flowchart of a memory management method according to an embodiment of the present disclosure. Referring to FIG. 7, the method process provided by the embodiment of the present application includes:
  • the resolution of the video stream may be pre-configured by the server to the terminal, or may be obtained by the terminal from the server. This embodiment does not limit this.
  • the terminal may determine one frame of video data according to the frame identifier of the data packet included in the video stream, and pre-allocate the memory for the frame of video data before decoding.
  • step 303 Detects whether a memory address is included in the memory multiplexing queue. If yes, go to step 303. Otherwise, go to step 305.
  • This step is the same as the process of detecting whether the memory address is included in the memory multiplexing queue in step 201, and details are not described herein.
  • the memory multiplexing queue includes a memory address, obtain a second memory address from the memory multiplexing queue according to the data amount of the video frame data indicated by the resolution.
  • the terminal can calculate the data amount of the video frame data according to the resolution of the video stream. For example, the terminal determines the frame length and the frame width of the video frame data according to the resolution. For example, when the resolution is 1080 ⁇ 720, the corresponding video frame data has a frame length of 1080 and a frame width of 720. The amount of data of the video frame data is determined according to the frame length and frame width of the determined video frame data, and the image format of the video frame data. The process is the same as the method for determining the data amount of the video frame data in the above step 202, and details are not described herein.
  • the second memory address is obtained from the memory multiplexing queue, move the second memory address out of the memory multiplexing queue, and use the second memory address as the first memory address allocated for the video frame data.
  • This step is the same as step 203 above, and will not be described here.
  • the second memory address is not obtained from the memory multiplexing queue, obtain a third memory address from a memory address other than the memory multiplexing queue, and use the third memory address as the allocated first memory address.
  • This step is the same as step 204 above, and will not be described here.
  • the memory multiplexing queue does not include a memory address, obtain a fourth memory address from a memory address other than the memory multiplexing queue according to the data amount of the video frame data indicated by the resolution, and use the fourth memory address as the fourth memory address.
  • This step is the same as the above step 205, and will not be described herein.
  • Steps 301 to 306 are performed on a main thread of a terminal, and after the terminal performs pre-allocation of memory for video frame data, the terminal may The pre-allocated first memory address is delivered to the sub-thread by an address delivery message (postMessage), and after the sub-thread decodes the video frame data, the video frame data is stored to the memory space indicated by the first memory address.
  • postMessage address delivery message
  • the terminal also renders the decoded video frame data, and the process of the rendering is the same as the process described in the foregoing step 206, and details are not described herein.
  • This step is the same as step 207 above, and will not be described here.
  • the resolution of the video stream may change during the video playback process.
  • the resolution of the video stream is modified by the user through the terminal, and the modified resolution is higher than that before the modification.
  • the first memory address allocated by the terminal for the decoded video frame data may not be sufficient to store the video frame data.
  • the data amount of the video frame data decoded by the terminal is greater than that indicated by the first memory address.
  • the terminal re-allocates a piece of memory for the video frame data according to the actual data amount of the video frame data, that is, obtains from the memory reuse queue and the memory address other than the first memory address.
  • a memory address is assigned to the video frame data.
  • the terminal sends the actual resolution corresponding to the video frame data to the main thread through the sub-thread, and the resolution of the video stream is modified in the main thread, and then the memory is allocated based on the modified resolution.
  • the allocated memory address may also be sent to the main thread to add the allocated memory address to the rendering queue through the main thread.
  • the method provided by the embodiment of the present application adds a memory address of the video frame data that has been rendered to the memory multiplexing queue, and then performs memory allocation according to the memory multiplexing queue, so that the allocated memory address can be duplicated.
  • the process of playing video there is no need to open up a memory space for each video frame data, which reduces the memory usage and improves the memory utilization.
  • the host of the memory management in the embodiment of the present application is a multimedia browsing application.
  • the multimedia browsing application uses the multimedia browsing application as a browser, and the memory management method described in FIG. 7 above is used.
  • the detailed process is shown in Figure 9:
  • the browser obtains the resolution of the video stream.
  • the resolution of the video stream may be pre-configured by the server to the browser, or may be obtained by the browser from the server. This embodiment does not limit this.
  • the browser may determine one frame of video data according to the frame identifier of the data packet included in the video stream, and pre-allocate the memory for the frame of video data before decoding.
  • the browser detects whether the memory multiplex queue includes a memory address. If yes, go to step 3013. Otherwise, go to step 3015.
  • This step is the same as the process of detecting whether the memory multiplex queue contains a memory address in step 2012, and will not be described here.
  • the browser obtains the second memory address from the memory multiplexing queue according to the data amount of the video frame data indicated by the resolution.
  • the browser can calculate the amount of data of the video frame data according to the resolution of the video stream. For example, the browser determines the frame length and frame width of the video frame data according to the resolution. For example, when the resolution is 1080 ⁇ 720, the corresponding video frame data has a frame length of 1080 and a frame width of 720. The amount of data of the video frame data is determined according to the frame length and frame width of the determined video frame data, and the image format of the video frame data. This process is the same as the method of determining the data amount of the video frame data in the above step 2013, and details are not described herein.
  • the browser moves the second memory address out of the memory multiplexing queue and uses the second memory address as the first memory address allocated for the video frame data.
  • This step is the same as the above step 2014, and will not be described here.
  • the browser obtains the third memory address from the memory address other than the memory multiplexing queue, and uses the third memory address as the allocated first memory address.
  • This step is the same as the above step 2015, and will not be described here.
  • the browser obtains the fourth memory address from the memory address other than the memory multiplexing queue according to the data amount of the video frame data indicated by the resolution, and the fourth memory is used.
  • the address is the first memory address assigned.
  • This step is the same as the above step 2016, and will not be described here.
  • the browser stores the video frame data to a memory space indicated by the first memory address.
  • the browser also renders the decoded video frame data, and the process of the rendering is the same as the process described in the above step 2017, and details are not described herein.
  • FIG. 8 a schematic flowchart of a memory management is shown.
  • the foregoing steps 3011 to 3016 are executed in a main thread of a browser, and the browser may perform pre-allocation of memory for video frame data.
  • the pre-allocated first memory address is delivered to the child thread by an address delivery message (postMessage), and the child thread stores the video frame data to a memory space indicated by the first memory address.
  • postMessage address delivery message
  • the browser After rendering according to the video frame data, the browser adds the first memory address to the memory multiplexing queue.
  • This step is the same as the above step 2018, and will not be described herein.
  • the resolution of the video stream may change during the video playback process.
  • the resolution of the video stream is modified by the user through the browser, and the modified resolution is higher than before the modification.
  • the first memory address allocated by the browser for the decoded video frame data may not be sufficient to store the video frame data.
  • the data amount of the video frame data obtained by the browser decoding is greater than the first memory address.
  • the browser allocates a piece of memory for the video frame data according to the actual data amount of the video frame data, that is, the memory is multiplexed from the memory and the memory other than the first memory address. In the address, obtain a memory address assigned to the video frame data.
  • the browser sends the actual resolution corresponding to the video frame data to the main thread through the sub-thread, and the resolution of the video stream is modified in the main thread, and then the memory is allocated based on the modified resolution.
  • the allocated memory address may also be sent to the main thread to add the allocated memory address to the rendering queue through the main thread.
  • the browser adds the memory address of the video frame data that has been rendered to the memory multiplexing queue, and then performs memory allocation according to the memory multiplexing queue, so that the allocated memory address can be allocated. Reusable, in the process of playing video, there is no need to open a memory space for each video frame data, which reduces the memory usage of the browser and improves the memory utilization of the browser.
  • FIG. 10 is a block diagram of a memory management apparatus according to an embodiment of the present application.
  • the apparatus includes an allocation module 401, a storage module 402, and an adding module 403.
  • the allocation module is configured to allocate, according to the memory multiplexing queue, a first memory address for the video frame data, where the memory multiplexing queue is used to record a memory address of the video frame data that has been rendered; and a storage module is configured to The video frame data is stored in the memory space indicated by the first memory address; the adding module is configured to add the first memory address to the memory multiplexing queue after rendering based on the video frame data.
  • the allocation module includes:
  • a first obtaining unit configured to: if the memory multiplexing queue includes a memory address, obtain a second memory address from the memory multiplexing queue according to the data amount of the video frame data, and the memory indicated by the second memory address The space is not less than the amount of data of the video frame data;
  • the first allocation unit is configured to: if the second memory address is obtained from the memory multiplexing queue, move the second memory address out of the memory multiplexing queue, and use the second memory address as the first memory address.
  • the allocating module further includes:
  • a second obtaining unit configured to obtain a third memory address from a memory address other than the memory multiplexing queue if the second memory address is not obtained from the memory multiplexing queue, and the memory space indicated by the third memory address Not less than the amount of data of the video frame data;
  • a second allocation unit configured to use the third memory address as the first memory address.
  • the allocating module further includes:
  • a third obtaining unit configured to obtain a fourth memory address from a memory address other than the memory multiplexing queue if the memory multiplexing queue does not include a memory address
  • a third allocation unit configured to use the fourth memory address as the first memory address.
  • the allocating module is configured to: when receiving the video stream, acquire a resolution of the video stream; perform a memory-based multiplexing queue according to the data amount of the video frame data indicated by the resolution , the step of allocating a first memory address to the video frame data; or,
  • the allocating module is configured to perform a step of allocating a first memory address for the video frame data based on the memory multiplexing queue when acquiring the video frame data in the process of decoding the received video stream.
  • the device further includes:
  • a startup module configured to start a sub-thread through a specified interface of the multimedia browsing application
  • a decoding module configured to decode the video frame data by using the sub-thread.
  • the device further includes:
  • a rendering module configured to render the video frame data by using a main thread of the multimedia browsing application
  • the adding module is configured to: pass the first memory address to the child thread of the multimedia browsing application through the main thread; add the first memory address to the memory multiplexing queue by using the sub-thread; or The first memory address is added to the memory multiplexing queue by the main thread.
  • the device provided by the embodiment of the present disclosure adds a memory address of the video frame data that has been rendered to the memory multiplexing queue, and then performs memory allocation according to the memory multiplexing queue, so that the allocated memory address can be duplicated.
  • the process of playing video there is no need to open up a memory space for each video frame data, which reduces the memory usage and improves the memory utilization.
  • the memory management device provided by the foregoing embodiment manages the memory
  • only the division of each of the foregoing functional modules is illustrated.
  • the foregoing functions may be allocated differently according to requirements.
  • the function module is completed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.
  • the memory management device and the touch memory management method embodiment provided by the foregoing embodiments are in the same concept, and the specific implementation process is described in detail in the method embodiment, and details are not described herein again.
  • FIG. 11 is a schematic structural diagram of a terminal according to an embodiment of the present disclosure.
  • the terminal may be used to perform the memory management method provided in the foregoing embodiments.
  • the terminal 500 includes:
  • the terminal 500 may include an RF (Radio Frequency) circuit 110, a memory 120 including one or more computer readable storage media, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, and a WiFi (Wireless Fidelity, wireless).
  • the fidelity module 170 includes a processor 180 having one or more processing cores, and a power supply 190 and the like. It will be understood by those skilled in the art that the terminal structure shown in FIG. 5 does not constitute a limitation to the terminal, and may include more or less components than those illustrated, or a combination of certain components, or different component arrangements. among them:
  • the RF circuit 110 can be used for transmitting and receiving information or during a call, and receiving and transmitting signals. Specifically, after receiving the downlink information of the base station, the downlink information is processed by one or more processors 180. In addition, the data related to the uplink is sent to the base station. .
  • the RF circuit 110 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier). , duplexer, etc.
  • RF circuitry 110 can also communicate with the network and other devices via wireless communication.
  • the wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access). , Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), e-mail, SMS (Short Messaging Service), and the like.
  • GSM Global System of Mobile communication
  • GPRS General Packet Radio Service
  • CDMA Code Division Multiple Access
  • WCDMA Wideband Code Division Multiple Access
  • LTE Long Term Evolution
  • e-mail Short Messaging Service
  • the memory 120 can be used to store software programs and modules, and the processor 180 executes various functional applications and data processing by running software programs and modules stored in the memory 120.
  • the memory 120 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may be stored according to The data created by the use of the terminal 500 (such as audio data, phone book, etc.) and the like.
  • memory 120 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, memory 120 may also include a memory controller to provide access to memory 120 by processor 180 and input unit 130.
  • the input unit 130 can be configured to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.
  • input unit 130 can include touch-sensitive surface 131 as well as other input devices 132.
  • Touch-sensitive surface 131 also referred to as a touch display or trackpad, can collect touch operations on or near the user (such as a user using a finger, stylus, etc., on any suitable object or accessory on touch-sensitive surface 131 or The operation near the touch-sensitive surface 131) and driving the corresponding connecting device according to a preset program.
  • the touch-sensitive surface 131 can include two portions of a touch detection device and a touch controller.
  • the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information.
  • the processor 180 is provided and can receive commands from the processor 180 and execute them.
  • the touch-sensitive surface 131 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves.
  • the input unit 130 can also include other input devices 132.
  • other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and the like.
  • Display unit 140 can be used to display information entered by the user or information provided to the user and various graphical user interfaces of terminal 500, which can be constructed from graphics, text, icons, video, and any combination thereof.
  • the display unit 140 may include a display panel 141.
  • the display panel 141 may be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like.
  • the touch-sensitive surface 131 may cover the display panel 141, and when the touch-sensitive surface 131 detects a touch operation thereon or nearby, it is transmitted to the processor 180 to determine the type of the touch event, and then the processor 180 according to the touch event The type provides a corresponding visual output on display panel 141.
  • touch-sensitive surface 131 and display panel 141 are implemented as two separate components to implement input and input functions, in some embodiments, touch-sensitive surface 131 can be integrated with display panel 141 for input. And output function.
  • Terminal 500 can also include at least one type of sensor 150, such as a light sensor, motion sensor, and other sensors.
  • the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 141 according to the brightness of the ambient light, and the proximity sensor may close the display panel 141 when the terminal 500 moves to the ear. / or backlight.
  • the gravity acceleration sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity.
  • the terminal 500 can also be configured with gyroscopes, barometers, hygrometers, thermometers, infrared sensors and other sensors, here Let me repeat.
  • the audio circuit 160, the speaker 161, and the microphone 162 can provide an audio interface between the user and the terminal 500.
  • the audio circuit 160 can transmit the converted electrical data of the received audio data to the speaker 161 for conversion to the sound signal output by the speaker 161; on the other hand, the microphone 162 converts the collected sound signal into an electrical signal by the audio circuit 160. After receiving, it is converted into audio data, and then processed by the audio data output processor 180, transmitted to the terminal, for example, via the RF circuit 110, or outputted to the memory 120 for further processing.
  • the audio circuit 160 may also include an earbud jack to provide communication of the peripheral earphones with the terminal 500.
  • WiFi is a short-range wireless transmission technology
  • the terminal 500 can help users to send and receive emails, browse web pages, and access streaming media through the WiFi module 170, which provides wireless broadband Internet access for users.
  • FIG. 11 shows the WiFi module 170, it can be understood that it does not belong to the essential configuration of the terminal 500, and may be omitted as needed within the scope of not changing the essence of the invention.
  • the processor 180 is the control center of the terminal 500, connecting various portions of the entire handset with various interfaces and lines, by running or executing software programs and/or modules stored in the memory 120, and recalling data stored in the memory 120, The various functions and processing data of the terminal 500 are performed to perform overall monitoring of the mobile phone.
  • the processor 180 may include one or more processing cores; preferably, the processor 180 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application, and the like.
  • the modem processor primarily handles wireless communications. It can be understood that the above modem processor may not be integrated into the processor 180.
  • the terminal 500 also includes a power source 190 (such as a battery) for powering various components.
  • a power source 190 such as a battery
  • the power source can be logically coupled to the processor 180 through a power management system to manage functions such as charging, discharging, and power management through the power management system.
  • Power supply 190 may also include any one or more of a DC or AC power source, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
  • the terminal 500 may further include a camera, a Bluetooth module, and the like, and details are not described herein again.
  • the display unit of the terminal is a touch screen display
  • the terminal further includes a memory, and one or more programs, wherein one or more programs are stored in the memory and configured to be processed by one or more Execution.
  • the one or more programs include executable instructions, and the terminal 500 is configured to execute instructions to perform the method performed by the terminal in the above described memory management method embodiment.
  • FIG. 12 is a schematic structural diagram of an electronic device 600 according to an embodiment of the present application.
  • the electronic device 600 includes a processor and a memory, and may further include a communication interface, a communication bus, an input/output interface, and a display device, wherein the processor, the memory, the input/output interface, the display device, and the communication interface are completed through a communication bus. Communication with each other.
  • the memory stores a computer program for executing a computer program stored on the memory to implement the memory management method in the above embodiment.
  • a communication bus is a circuit that connects the elements described and implements transmission between these elements.
  • the processor receives commands from other elements over the communication bus, decrypts the received commands, and performs calculations or data processing in accordance with the decrypted commands.
  • the memory may include program modules such as a kernel, middleware, an application programming interface (API), and an application.
  • the program module can be composed of software, firmware or hardware, or at least two of them.
  • the input and output interfaces forward commands or data entered by the user through input and output devices (eg, sensors, keyboards, touch screens).
  • the display device displays various information to the user.
  • the communication interface connects the electronic device 600 with other network devices, user devices, and networks.
  • the communication interface can be connected to the network by wired or wireless connection to other external network devices or user devices.
  • the wireless communication may include at least one of the following: Wireless Fidelity (WiFi), Bluetooth (BT), Near Field Communication (NFC), and Global Positioning System (GPS).
  • the wired communication may include at least one of the following: a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), and an asynchronous standard interface (Recommended Standard 232, RS-232). , and ordinary old-fashioned telephone business (Plain Old T Elephone Service, POTS).
  • the network can be a telecommunication network and a communication network.
  • the communication network can be a computer network, the Internet, an Internet of Things, a telephone network.
  • the electronic device 600 can be connected to the network through a communication interface, and the electronic device 600 and other network devices are used for communication.
  • the protocol can be supported by at least one of an application, an Application Programming Interface (API), a middleware, a kernel, and a communication interface.
  • API Application Programming Interface
  • a computer readable storage medium such as a memory storing a computer program, which when executed by a processor implements the memory management method of the above embodiment.
  • the computer readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), or a Compact Disc Read-Only Memory (CD-ROM). , tapes, floppy disks, and optical data storage devices.
  • a person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium.
  • the storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Mathematical Physics (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请公开了一种内存管理方法、装置、电子设备及存储介质，属于网络技术领域。该方法包括：基于内存复用队列，为视频帧数据分配第一内存地址，内存复用队列用于记录已进行了渲染的视频帧数据所在的内存地址；将视频帧数据存储至第一内存地址所指示的内存空间；在基于视频帧数据进行渲染后，将第一内存地址添加到内存复用队列中。通过将已经进行了渲染的视频帧数据所在的内存地址添加到内存复用队列，后续可以根据该内存复用队列进行内存分配，使得已经分配的内存地址能够得到重复利用，在播放视频的过程中，无需针对每一个视频帧数据均开辟一块内存空间，降低了内存占用率，提高了内存利用率。

Description

内存管理方法、装置、电子设备及存储介质
本申请要求于2017年07月18日提交中国国家知识产权局、申请号为201710587746.0、发明名称为“内存管理方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及网络技术领域，特别涉及一种内存管理方法、装置、电子设备及存储介质。
背景技术
终端能够通过浏览器在网页上进行视频播放，在播放视频的过程中，终端需要对浏览器占用的内存进行管理，比如为浏览器解码视频流并分配内存，以及回收分配的内存。
现有技术中，浏览器从服务器接收到视频流后，一般以帧为单位对视频流进行解码，对于解码所得到的每一帧视频数据，终端均需要分配一定大小的内存空间，来存储该解码后的每一帧视频数据，以便后续对解码后的每一帧视频数据进行渲染，从而在终端呈现视频画面。其中，分配一定大小的内存空间是指分配内存地址，对于已经分配的内存地址，终端会定期进行回收。
在实现本申请的过程中，发明人发现现有技术至少存在以下问题：
对于帧率较高的视频数据，比如帧率为25fps(帧每秒)的视频数据，1秒钟便需要进行25次内存分配，1分钟就达到1500次，导致短时间内会为浏览器分配大量的内存地址，即使对分配的内存地址进行定期回收，如此大量的内存地址也无法得到及时回收，导致内存占用量大，内存利用率低。
发明内容
本申请实施例提供了一种内存管理方法、装置、电子设备及存储介质，可以解决内存占用量大、内存利用率低的问题。所述技术方案如下：
第一方面，提供了一种内存管理方法，所述方法包括：
基于内存复用队列，为视频帧数据分配第一内存地址，所述内存复用队列用于记录已进行了渲染的视频帧数据所在的内存地址；
将所述视频帧数据存储至所述第一内存地址所指示的内存空间；
在基于所述视频帧数据进行渲染后，将所述第一内存地址添加到所述内存复用队列中。
在一种可能的实现方式中，所述基于内存复用队列，为视频帧数据分配第一内存地址，包括：
如果所述内存复用队列中包含有内存地址，根据所述视频帧数据的数据量，从所述内存复用队列中获取第二内存地址，所述第二内存地址所指示的内存空间不小于所述视频帧数据的数据量；
如果从所述内存复用队列中获取到所述第二内存地址，将所述第二内存地址移出所述 内存复用队列，并将所述第二内存地址作为所述第一内存地址。
在一种可能的实现方式中，根据所述视频帧数据的数据量，从所述内存复用队列中获取第二内存地址之后，所述方法还包括：
如果未从所述内存复用队列中获取到所述第二内存地址，从所述内存复用队列以外内存地址中获取第三内存地址，所述第三内存地址所指示的内存空间不小于所述视频帧数据的数据量；
将所述第三内存地址作为所述第一内存地址。
在一种可能的实现方式中，所述基于内存复用队列，为视频帧数据分配第一内存地址，包括：
如果所述内存复用队列中未包含有内存地址，从所述内存复用队列以外的内存地址中获取第四内存地址；
将所述第四内存地址作为所述第一内存地址。
在一种可能的实现方式中，所述方法还包括：
当接收到视频流时，获取所述视频流的分辨率；根据所述分辨率所指示的视频帧数据的数据量，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤；或，
在对接收到的视频流进行解码的过程中，当获取到所述视频帧数据时，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤。
在一种可能的实现方式中，所述方法还包括：
通过多媒体浏览应用的指定接口启动子线程；
通过所述子线程对所述视频帧数据进行解码。
在一种可能的实现方式中，所述方法还包括：
通过多媒体浏览应用的主线程对所述视频帧数据进行渲染；
相应地，所述在基于所述视频帧数据进行渲染后，将所述第一内存地址添加到所述内存复用队列中，包括：
通过所述主线程将所述第一内存地址传递给所述多媒体浏览应用的子线程；通过所述子线程将所述第一内存地址添加到所述内存复用队列中；或，
通过所述主线程将所述第一内存地址添加到所述内存复用队列中。
第二方面，提供了一种内存管理装置，所述装置包括：
分配模块，用于基于内存复用队列，为视频帧数据分配第一内存地址，所述内存复用队列用于记录已进行了渲染的视频帧数据所在的内存地址；
存储模块，用于将所述视频帧数据存储至所述第一内存地址所指示的内存空间；
添加模块，用于在基于所述视频帧数据进行渲染后，将所述第一内存地址添加到所述内存复用队列中。
在一种可能的实现方式中，所述分配模块包括：
第一获取单元，用于如果所述内存复用队列中包含有内存地址，根据所述视频帧数据的数据量，从所述内存复用队列中获取第二内存地址，所述第二内存地址所指示的内存空间不小于所述视频帧数据的数据量；
第一分配单元，用于如果从所述内存复用队列中获取到所述第二内存地址，将所述第 二内存地址移出所述内存复用队列，并将所述第二内存地址作为所述第一内存地址。
在一种可能的实现方式中，所述分配模块还包括：
第二获取单元，用于如果未从所述内存复用队列中获取到所述第二内存地址，从所述内存复用队列以外内存地址中获取第三内存地址，所述第三内存地址所指示的内存空间不小于所述视频帧数据的数据量；
第二分配单元，用于将所述第三内存地址作为所述第一内存地址。
在一种可能的实现方式中，所述分配模块还包括：
第三获取单元，用于如果所述内存复用队列中未包含有内存地址，从所述内存复用队列以外的内存地址中获取第四内存地址；
第三分配单元，用于将所述第四内存地址作为所述第一内存地址。
在一种可能的实现方式中，所述分配模块，用于当接收到视频流时，获取所述视频流的分辨率；根据所述分辨率所指示的视频帧数据的数据量，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤；或，
所述分配模块，用于在对接收到的视频流进行解码的过程中，当获取到所述视频帧数据时，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤。
在一种可能的实现方式中，所述装置还包括：
启动模块，用于通过多媒体浏览应用的指定接口启动子线程；
解码模块，用于通过所述子线程对所述视频帧数据进行解码。
在一种可能的实现方式中，所述装置还包括：
渲染模块，用于通过多媒体浏览应用的主线程对所述视频帧数据进行渲染；
相应地，所述添加模块用于：通过所述主线程将所述第一内存地址传递给所述多媒体浏览应用的子线程；通过所述子线程将所述第一内存地址添加到所述内存复用队列中；或，通过所述主线程将所述第一内存地址添加到所述内存复用队列中。
第三方面，提供了一种电子设备，所述电子设备包括处理器和存储器；存储器，用于存放计算机程序；处理器，用于执行存储器上所存放的计算机程序，实现第一方面任一种实现方式所述的方法步骤。
第四方面，提供了一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现第一方面任一种实现方式所述的方法步骤。
本申请实施例提供的技术方案带来的有益效果至少包括：
通过将已经进行了渲染的视频帧数据所在的内存地址添加到内存复用队列，后续可以根据该内存复用队列进行内存分配，使得已经分配的内存地址能够得到重复利用，在播放视频的过程中，无需针对每一个视频帧数据均开辟一块内存空间，降低了内存占用率，提高了内存利用率。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的 附图。
图1是本申请实施例提供的一种实施环境的示意图；
图2是本申请实施例提供的一种内存管理方法的流程图；
图3是本申请实施例提供的一种内存管理方法的流程图；
图4是本申请实施例提供的一种内存管理的流程图示意图；
图5是本申请实施例提供的一种内存复用队列的示意图；
图6是本申请实施例提供的一种内存管理方法的流程图；
图7是本申请实施例提供的一种内存管理方法的流程图；
图8是本申请实施例提供的一种内存管理的流程图示意图；
图9是本申请实施例提供的一种内存管理方法的流程图；
图10是本申请实施例提供的一种内存管理装置的框图；
图11是本申请实施例提供的一种终端的结构示意图；
图12是本申请实施例提供的一种电子设备600的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
图1是本申请实施例提供的实施环境示意图，包括终端11，服务器12和监控设备13。
其中，终端11能够在网页中播放视频，服务器12用于向终端11提供多媒体服务，如提供视频流，监控设备13具有视频采集功能。
在一种实施方式中，服务器12的多媒体服务功能集成在监控设备13中。集成了服务器12的功能后，监控设备12也可以向终端11提供多媒体服务。终端11可以通过互联网与监控设备13直接进行交互。例如，终端11上安装有浏览器，通过在浏览器中输入监控设备13的网络地址来访问监控设备13，以获取监控设备13实时采集的视频流。
当然，监控设备13中还可以设置有存储器(比如硬盘存储器)，监控设备13可以将采集的视频数据存储到该存储器中，以便终端11能够获取已经存储的视频数据，从而进行监控视频的回放等。
在另一种实施方式中，服务器12作为一个独立的实体，终端11和监控设备13分别可以通过互联网与服务器12进行交互。监控设备13可以将采集的监控视频流实时发送至服务器12，由服务器12作为中转将视频流转发至终端13，以通过终端进行实时监控。例如，终端11上安装有浏览器，通过在该浏览器中输入服务器12的访问地址，从服务器12获取实时转发的视频流或者其他多媒体数据，比如，图像数据，音视频数据等。
其中，服务器12还可以将从监控设备13接收到监控视频流进行存储，比如存储到硬盘存储器中，使得终端11也可以从服务器获取已存储的监控视频数据，以进行监控回放。
需要说明的是，当服务器12的多媒体服务功能集成在监控设备13中时，后续所涉及的服务器所执行的步骤均可以指监控设备13所执行的步骤。
图2是本申请实施例提供的一种内存管理方法的流程图，参见图2，本申请实施例提供的方法流程包括：
101、基于内存复用队列，为视频帧数据分配第一内存地址，该内存复用队列用于记录已进行了渲染的视频帧数据所在的内存地址。
102、将该视频帧数据存储至该第一内存地址所指示的内存空间。
103、在基于该视频帧数据进行渲染后，将该第一内存地址添加到该内存复用队列中。
在一种可能的实现方式中，该基于内存复用队列，为视频帧数据分配第一内存地址，包括：
如果该内存复用队列中包含有内存地址，根据该视频帧数据的数据量，从该内存复用队列中获取第二内存地址，该第二内存地址所指示的内存空间不小于该视频帧数据的数据量；
如果从该内存复用队列中获取到该第二内存地址，将该第二内存地址移出该内存复用队列，并将该第二内存地址作为该第一内存地址。
在一种可能的实现方式中，根据该视频帧数据的数据量，从该内存复用队列中获取第二内存地址之后，该方法还包括：
如果未从该内存复用队列中获取到该第二内存地址，从该内存复用队列以外内存地址中获取第三内存地址，该第三内存地址所指示的内存空间不小于该视频帧数据的数据量；
将该第三内存地址作为该第一内存地址。
在一种可能的实现方式中，该基于内存复用队列，为视频帧数据分配第一内存地址，包括：
如果该内存复用队列中未包含有内存地址，从该内存复用队列以外的内存地址中获取第四内存地址；
将该第四内存地址作为该第一内存地址。
在一种可能的实现方式中，该方法还包括：
当接收到视频流时，获取该视频流的分辨率；根据该分辨率所指示的视频帧数据的数据量，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤；或，
在对接收到的视频流进行解码的过程中，当获取到该视频帧数据时，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤。
本申请实施例提供的方法，通过将已经进行了渲染的视频帧数据所在的内存地址添加到内存复用队列，后续可以根据该内存复用队列进行内存分配，使得已经分配的内存地址能够得到重复利用，在播放视频的过程中，无需针对每一个视频帧数据均开辟一块内存空间，降低了内存占用率，提高了内存利用率。
在本申请实施例中，终端安装有多媒体浏览应用，该多媒体浏览应用是指具有在网页中展示多媒体数据的应用，比如该多媒体浏览应用可以为浏览器，其中，该浏览器可以为无插件浏览器，即可以不安装任何视频插件，便可以在网页中播放视频。比如，终端在通过浏览器访问监控设备时，可以通过监控设备获取视频解码程序，通过执行该视频解码程序来对接收到的视频流进行解码，从而实现在网页中播放视频。当然，该浏览器也可以通过安装视频插件来进行视频解码，以提供在网页中播放视频的功能。其中，终端通过浏览器从服务器上获取到视频流之后，对视频流所指示的每一帧数据进行解码。本实施例中将解码后的每一帧视频数据称为视频帧数据，终端需要为该视频帧数据分配内存，也即是， 为该视频帧数据分配内存地址，以在内存地址所指示的内存空间中存储该视频帧数据。
其中，需要说明的是，终端为该视频帧数据分配内存的过程也即是多媒体浏览应用为该视频帧数据申请内存的过程，下述实施例中以终端进行内存分配的角度进行说明，其中执行主体均可以将终端替换为多媒体浏览应用。下面以多媒体浏览应用是浏览器为例，本申请实施例中，根据内存分配时机的不同，终端分配内存的方式可以包括以下两种：
(1)终端在通过浏览器接收到视频流之后，先对该视频流进行解码，然后根据解码得到的视频帧数据进行内存分配，分配的内存地址可以为从内存复用队列中移出的内存地址，在对该视频帧数据进行渲染之后，可以将该分配的内存地址添加到该内存复用队列中，以实现已分配内存地址的复用，该过程详见图3所提供的实施例。其中，内存复用队列用于记录已经进行了渲染的视频数据所在的内存地址。
(2)终端在通过浏览器接收到视频流之后，预先分配内存，预分配的内存地址可以为从内存复用队列中移出的内存地址，在解码得到视频帧数据后，将视频帧数据存储至预分配的内存地址所指示的内存空间，在对该视频帧数据进行渲染之后，也可以将该分配的内存地址添加到该内存复用队列中，以实现已分配内存地址的复用，该过程详见图7所提供的实施例。
需要说明的是，为了便于说明，下述实施例以解码得到的一帧视频帧数据为例，对终端的内存管理过程进行介绍，其中，为该视频帧数据分配的内存地址简称为第一内存地址。
图3是本申请实施例提供的一种内存管理方法的流程图，参见图3，本申请实施例提供的方法流程包括：
201、在对接收到的视频流进行解码的过程中，当获取到解码得到的视频帧数据时，检测内存复用队列中是否包含有内存地址，如果包含有，执行步骤202，否则执行步骤205。
在本申请实施例中，终端在从服务器获取到视频流之后，会对该视频流所指示的每一帧视频数据进行解码。其中，视频流中包括符合预设格式的多个数据包，每个数据包对应有帧标识，具有相同帧标识的数据包构成一帧视频数据。终端可以根据数据包的帧标识来确定一帧视频数据，并在全部接收到该一帧视频数据之后，对该一帧视频数据进行解码，得到视频帧数据。其中，预设格式可以根据实际的视频传输需求进行预先设定或修改，本实施例对此不作限定。
其中，终端可以在主线程中从服务器中接收视频流，由于视频解码过程较为耗时，为了不堵塞主线程，可以采用web worker技术创建不同于主线程的异步线程，通过该异步线程来执行该解码的过程，该异步线程也可以称为子线程。实施时，终端可以通过浏览器的指定接口启动子线程，通过该子线程对视频帧数据进行解码。其中，指定接口可以预先进行设定或修改，例如该指定接口可以为浏览器的web worker接口，本实施例对此不作限定。参见图4，其示出了一种内存管理的流程示意图，其中，终端通过主线程调用WebSocket接口，以创建一个数据传输通道，通过该数据传输通道从监控设备或服务器获取视频流(如图4中的实时流)。在主线程获取到视频流之后，将该视频流发送至子线程，例如，终端通过调用线程交互接口(postMessage)将该视频流发送至子线程。
其中，内存复用队列用于记录已经进行了渲染的视频数据所在的内存地址。需要说明的是，在解码得到视频帧数据之后，终端需要对该视频帧数据进行渲染，以显示该视频帧数据对应的视频画面，从而实现视频播放。本申请实施例中，将已经渲染过的视频帧数据 所占用的内存地址添加到内存复用队列中，以便后续对该内存地址所指示的内存空间进行重复利用。
在本申请实施例中，该内存复用队列初始时可以为空；也可以在该内存复用队列中预先设置一个或多个能够满足视频帧数据的数据量的内存地址，也即是与视频流的分辨率对应的内存地址。
需要说明的是，本申请实施例中的内存地址是指一个内存块所对应的内存地址。例如，当内存以字节为单位进行编址时，假设内存块包括16个字节，则该内存块所对应的内存地址便是该16个字节的地址。在实施过程中，逻辑层可以通过数组来表示分配的内存块，相应地，内存复用队列为一个数组队列，每个数组可以称为队列的一个元素。
在本申请实施例中，终端可以检测内存复用队列是否为空(是否存在或包含有内存地址)，如果不为空，则确定该内存复用队列中包含有内存地址，执行步骤202；如果为空，则确定该内存复用队列中未包含有内存地址，执行步骤205。其中，终端检测内存复用队列是否为空的过程可以为：从内存复用队列的队列头，获取第一个元素，如果第一个元素不为空，则确定内存复用队列不为空；如果第一个元素为空，则确定内存复用队列为空。
202、如果内存复用队列中包含有内存地址，根据该视频帧数据的数据量，从内存复用队列中获取第二内存地址。
终端在解码得到该视频数据后，可以确定该视频帧数据的数据量，比如，终端可以确定该视频帧数据的帧长度、帧宽度，以及该视频帧数据的图像格式，来确定该视频帧数据的数据量。其中，视频帧数据的帧长度是指视频帧数据所指示图像中每一行所包含像素点的个数，视频帧数据的帧宽度是指视频帧数据所指示图像中每一列所包含像素点的个数。其中，视频帧数据的图像格式可以为RGB图像格式或者YUV图像格式等，本实施例对此不作限定。需要说明的是，在默认情况下视频帧数据的图像格式可以设置为YUV图像格式，如果渲染需要RGB图像格式，则终端可以进行图像格式转换，以将视频帧数据由YUV图像格式转换为RGB图像格式；另外，终端还可以在解码过程中进行图像格式处理，直接输出满足渲染需求图像格式的视频帧数据。
需要说明的是，视频帧数据的帧长度、帧宽度可以携带在视频帧数据中，该图像格式可以为服务器和终端默认的图像格式，由服务器提前配置到终端；当然，该图像格式也可以根据实际需求变化，通过携带在视频帧数据中通知给终端，本实施例对此不作限定。在对视频流解码得到视频帧数据后，终端可以从视频帧数据中获取到帧长度、帧宽度和图像格式，从而确定该视频帧数据的数据量。例如，对于图像格式为RGB24的视频帧数据，该视频帧数据的数据量为width×heigth×3Bit，对于图像格式为YUV420的视频帧数据，该视频帧数据的数据量为width×heigth×1.5Bit。其中，width表示视频帧数据的帧长度，heigth表示视频帧数据的帧宽度。
当内存复用队列中包含有内存地址时，终端可以根据该视频帧数据的数据量，将该内存地址所指示的内存空间和该视频帧数据比较，从该内存复用队列中获取第二内存地址。该第二内存地址所指示的内存空间应当不小于该视频帧数据的数据量。
其中，终端根据该视频帧数据的数据量，从该内存复用队列中获取第二内存地址的过程可以为：终端按照内存地址在内存复用队列中的排列顺序，从内存复用队列获取一个内存地址，检测该内存地址所指示的内存空间是否不小于视频帧数据的数据量，如果是，则 将该内存地址确定为第二内存地址；如果否，则从该内存复用队列中获取下一个内存地址，直至获取到第二内存地址为止。
在另一种实施方式中，由于视频流的分辨率在一段时间内是相对稳定的，内存复用队列中所包含的多个内存地址所指示的内存空间应该是相同的，如果内存复用队列中第一个内存地址所指示内存空间的大小小于视频帧数据的数据量，那么后续获取的多个内存地址所指示的内存空间也很有可能小于视频帧数据的数据量，因此，为了提高处理效率，当终端在内存复用队列中获取的第一个内存地址所指示的内存空间小于视频帧数据的数据量时，将该内存地址移出该内存复用队列，终端不再获取内存复用队列中的下一个内存地址，直接执行从内存复用队列以外的内存地址中获取第三内存地址的过程，该过程详见步骤204。
其中，当终端检测到当前获取的内存地址所指示的内存空间小于视频帧数据的数据量，终端将该内存地址进行回收，也即是，从内存复用队列中移出该内存地址，清空该内存地址所指示内存空间所存储的数据，并释放该内存地址。由于视频流的分辨率在一定时间以内是稳定的，当该内存地址所指示的内存空间小于视频帧数据的数据量时，说明该内存地址在一定时间以内是用不到的，通过及时回收用不到的内存地址，降低了内存占用率，提高了内存利用率。
203、如果从内存复用队列中获取到该第二内存地址，将所述第二内存地址移出内存复用队列，并将第二内存地址作为为该视频帧数据分配的第一内存地址。
当终端获取到第二内存地址时，将该第二内存地址移出该内存复用队列，并将该第二内存地址作为为视频帧数据分配的第一内存地址，以存储该视频帧数据。其中，从内存复用队列中移出一个内存地址是指从内存复用队列中删除该内存地址。
204、如果未从内存复用队列中获取到该第二内存地址，从内存复用队列以外内存地址中获取第三内存地址，将该第三内存地址作为分配的第一内存地址。
终端还有可能从内存复用队列中获取不到该第二内存地址，比如内存复用队列中的内存地址所指示的内存空间不够大，也即是，内存复用队列中不存在内存空间不小于视频帧数据的数据量的内存地址。此时，终端可以为该视频帧数据重新分配一个内存地址，也即是，终端在该内存复用队列以外的未分配的内存地址中获取第三内存地址，将该第三内存地址作为为该视频帧数据分配的第一内存地址。例如，终端可以采用new ArrayBuffer()语句创建一个满足视频帧数据大小的数组，将该数组所指示的内存地址即为第三内存地址。当然，该第三内存地址所指示内存空间不小于该视频帧数据的数据量。
205、如果内存复用队列中未包含有内存地址，从内存复用队列以外的内存地址中获取第四内存地址，将该第四内存地址作为分配的第一内存地址。
当终端检测到内存复用队列未包含有内存地址时，会为该视频帧数据重新分配一个内存地址，也即是，从复用队列以外的未分配的地址中获取第四内存地址。该分配第四内存地址的过程与步骤203中分配第三内存地址的过程同理，在此不做赘述。
206、将视频帧数据存储至第一内存地址所指示的内存空间。
参见图4，本实施例中对视频流进行解码的过程，解码后分配内存的过程以及存储解码得到的视频帧数据的过程均可以在子线程中处理。终端在将视频帧数据存储到分配的内存空间之后，终端可以在主线程对该视频帧数据进行渲染，以显示该视频帧数据所指示的视 频画面。
其中，终端对视频帧数据进行渲染的过程可以为：终端在主线程维护一个渲染队列，终端存储视频帧数据之后，通过地址推送消息将该视频帧数据所在的第一内存地址发送至主线程，以在主线程将该第一内存地址添加到该渲染队列。其中，地址推送消息携带该第一内存地址。实施时，终端通过多媒体浏览应用的主线程对该视频帧数据进行渲染。
在渲染过程中，终端会定时扫描该渲染队列，如图4所示，终端向渲染队列请求视频帧数据(requestAnimationFrame)，按照该渲染队列中内存地址的排列顺序，依次获取内存地址所指示的视频帧数据，并对获取的视频帧数据进行渲染。其中，由于终端对视频流解码的速度和渲染的速度之间存在差异，终端到达扫描渲染队列的周期时，该渲染队列可能为空，此时，终端便不执行渲染过程，待到下一个扫描周期继续进行扫描，待扫描到渲染队列不为空时，根据渲染队列中内存地址所指示的视频帧数据进行渲染。例如，终端可以采用预设绘图标准进行渲染，并以画布(canvas)为容器显示渲染后的图像。该预设绘图标准可以为WebGL(Web Graphics Library，web图形库)绘图标准，本实施例对此不作限定。
207、基于视频帧数据进行渲染后，将第一内存地址添加到内存复用队列中。
为了避免不断分配新的内存造成内存占用量迅速上升，本申请实施例中，将已经渲染过的视频帧数据所占用的内存空间进行重复利用，也即是，将已经渲染过的视频帧数据所在的内存地址回收到内存复用队列中，以便后续将该内存地址再次分配给其他未渲染的视频帧数据。
终端可以通过两种方式将第一内存地址添加到该内存复用队列中：在一种实施方式中，终端可以通过主线程将第一内存地址传递给该多媒体浏览应用的子线程，并通过该子线程将该第一内存地址添加到该内存复用队列中；在另一种实施方式中，终端直接通过主线程将该第一内存地址添加到该内存复用队列中。在实施过程中，终端在主线程对视频帧数据进行渲染后，可以通过调用线程交互接口(postMessage)将该第一内存地址发送给子线程，通过子线程执行将第一内存地址添加到内存复用队列的步骤。
需要说明的是，由于对视频流进行解码的速度和渲染的速度在一段时间内是相对稳定的，终端一直处于占用状态的内存地址的数目可以达到一个稳定的状态，也即是，当终端一直处于占用状态的内存地址的数目达到一定数目时，便能够满足已经解码但是还未进行渲染的视频帧数据的存储需求。此时，内存复用队列中所包含的内存地址的数目也处于相对稳定的状态，也即是，在从内存复用队列中移出一个内存地址时，也会向内存复用队列中添加一个内存地址。通过对播放高清监控视频的测试，在对视频播放过程中，内存复用队列中包含5个内存地址时便可以满足播放需求。由于在播放视频的过程中，终端仅需开辟少量的内存地址，通过对该少量的内存地址的重复利用便可实现在网页中播放视频，大大降低了内存占用量，提高了内存利用率。
为了更加清楚的说明内存复用队列的原理，下面结合图5所示的内存复用队列示意图进行解释说明，图中内存复用队列包括队列头和队列尾，终端从队列头移出内存地址以进行内存分配，该过程可以包括上述步骤201至步骤205所对应的过程。终端从队列尾添加已经进行了渲染的视频帧数据所在的内存地址，该过程可以包括步骤206至步骤207所对应的过程。
本申请实施例提供的方法，通过将已经进行了渲染的视频帧数据所在的内存地址添加 到内存复用队列，后续可以根据该内存复用队列进行内存分配，使得已经分配的内存地址能够得到重复利用，在播放视频的过程中，无需针对每一个视频帧数据均开辟一块内存空间，降低了内存占用率，提高了内存利用率。
另外，在该实施例中，通过在解码得到视频帧数据后，再为视频帧数据分配内存，使得终端可以根据视频帧数据的实际数据量，动态地为视频帧数据分配合适的内存空间，提高了内存分配的灵活性。
由于本申请实施例中承载内存管理的主体是多媒体浏览应用，为了更加清楚的对内存管理方法流程进行解释说明，下面以多媒体浏览应用是浏览器为例，对上述图3所述的内存管理方法进行介绍，详细流程参见图6：
2011、当从服务器接收到视频帧数据时，浏览器通过指定接口启动子线程，通过该子线程对该视频帧数据进行解码。
在本申请实施例中，浏览器通过输入的IP地址从服务器获取视频流，该视频流包含多个视频帧数据，浏览器对接收到的每一帧数据进行解码，以便后续通过渲染向用户展示视频内容。
其中，浏览器在主线程中从服务器中接收视频流，由于视频解码过程较为耗时，为了不堵塞主线程，可以采用web worker技术创建不同于主线程的异步线程，通过该异步线程来执行该解码的过程，该异步线程也可以称为子线程。实施时，可以通过浏览器的指定接口启动子线程，通过该子线程对视频帧数据进行解码。该指定接口可以预先进行设定或修改，例如该指定接口可以为浏览器的web worker接口，本实施例对此不作限定。其中，相应内存管理的流程示意图可以参见图4。
其中，视频流中包括符合预设格式的多个数据包，每个数据包对应有帧标识，具有相同帧标识的数据包构成一帧视频数据。终端可以根据数据包的帧标识来确定一帧视频数据，并在全部接收到该一帧视频数据之后，对该一帧视频数据进行解码，得到视频帧数据。其中，预设格式可以根据实际的视频传输需求进行预先设定或修改，本实施例对此不作限定。
2012、当获取到解码得到的视频帧数据时，浏览器检测内存复用队列中是否包含有内存地址，如果包含有，执行步骤2013，否则执行步骤2016。
其中，内存复用队列用于记录已经进行了渲染的视频数据所在的内存地址。在解码得到视频帧数据之后，浏览器需要对该视频帧数据进行渲染，以显示该视频帧数据对应的视频画面，从而实现视频播放。本申请实施例中，浏览器可以将已经渲染过的视频帧数据所占用的内存地址添加到内存复用队列中，以便后续对该内存地址所指示的内存空间进行重复利用。
在本申请实施例中，该内存复用队列初始时可以为空；也可以在该内存复用队列中预先设置一个或多个能够满足视频帧数据的数据量的内存地址，也即与视频流的分辨率对应的内存地址。
需要说明的是，本申请实施例中的内存地址是指一个内存块所对应的内存地址。例如，当内存以字节为单位进行编址时，假设内存块包括16个字节，则该内存块所对应的内存地址便是该16个字节的地址。在实施过程中，逻辑层可以通过数组来表示分配的内存块，相应地，内存复用队列为一个数组队列，每个数组可以称为队列的一个元素。
浏览器可以检测内存复用队列是否为空，如果不为空，则确定该内存复用队列中包含有内存地址，执行步骤2013；如果为空，则确定该内存复用队列中未包含有内存地址，执行步骤2016。其中，浏览器检测内存复用队列是否为空的过程可以为：浏览器从内存复用队列的队列头，获取第一个元素，如果第一个元素不为空，则确定内存复用队列不为空；如果第一个元素为空，则确定内存复用队列为空。
2013、如果内存复用队列中包含有内存地址，浏览器根据该视频帧数据的数据量，从内存复用队列中获取第二内存地址。
浏览器在解码得到该视频数据后，可以确定该视频帧数据的数据量，比如，浏览器可以确定该视频帧数据的帧长度、帧宽度，以及该视频帧数据的图像格式，来确定该视频帧数据的数据量。其中，视频帧数据的帧长度、帧宽度和视频帧数据的图像格式的解释与步骤202中相应的解释同理，在此不做赘述。
需要说明的是，视频帧数据的帧长度、帧宽度可以携带在视频帧数据中，该图像格式可以为服务器和浏览器默认的图像格式，由服务器提前配置到浏览器；当然，该图像格式也可以根据实际需求变化，通过携带在视频帧数据中通知给浏览器，本实施例对此不作限定。在对视频流解码得到视频帧数据后，浏览器可以从视频帧数据中获取到帧长度、帧宽度和图像格式，从而确定该视频帧数据的数据量。例如，对于图像格式为RGB24的视频帧数据，该视频帧数据的数据量为width×heigth×3Bit，对于图像格式为YUV420的视频帧数据，该视频帧数据的数据量为width×heigth×1.5Bit。其中，width表示视频帧数据的帧长度，heigth表示视频帧数据的帧宽度。
当内存复用队列中包含有内存地址时，浏览器可以根据该视频帧数据的数据量，从该内存复用队列中获取第二内存地址。该第二内存地址所指示的内存空间应当不小于该视频帧数据的数据量。
其中，浏览器根据该视频帧数据的数据量，从该内存复用队列中获取第二内存地址的过程可以为：浏览器按照内存地址在内存复用队列中的排列顺序，从内存复用队列获取一个内存地址，检测该内存地址所指示的内存空间是否不小于视频帧数据的数据量，如果是，则将该内存地址确定为第二内存地址；如果否，则从该内存复用队列中获取下一个内存地址，直至获取到第二内存地址为止。
在另一种实施方式中，由于视频流的分辨率在一段时间内是相对稳定的，内存复用队列中所包含的多个内存地址所指示的内存空间应该是相同的，如果内存复用队列中第一个内存地址所指示内存空间的大小小于视频帧数据的数据量，那么后续获取的多个内存地址所指示的内存空间也很有可能小于视频帧数据的数据量，因此，为了提高处理效率，当浏览器在内存复用队列中获取的第一个内存地址所指示的内存空间小于视频帧数据的数据量时，将该内存地址移出该内存复用队列，浏览器不再获取内存复用队列中的下一个内存地址，直接执行从内存复用队列以外的内存地址中获取第三内存地址的过程，该过程详见步骤2015。
其中，当浏览器检测到当前获取的内存地址所指示的内存空间小于视频帧数据的数据量，浏览器将该内存地址进行回收，也即是，从内存复用队列中移出该内存地址，清空该内存地址所指示内存空间所存储的数据，并释放该内存地址。由于视频流的分辨率在一定时间以内是稳定的，当该内存地址所指示的内存空间小于视频帧数据的数据量时，说明该 内存地址在一定时间以内是用不到的，通过及时回收用不到的内存地址，降低了内存占用率，提高了内存利用率。
2014、如果从内存复用队列中获取到该第二内存地址，浏览器将该第二内存地址移出内存复用队列，并将第二内存地址作为为该视频帧数据分配的第一内存地址。
当浏览器获取到第二内存地址时，将该第二内存地址移出该内存复用队列，并将该第二内存地址作为为视频帧数据分配的第一内存地址，以存储该视频帧数据。其中，从内存复用队列中移出一个内存地址是指从内存复用队列中删除该内存地址。
2015、如果未从内存复用队列中获取到该第二内存地址，浏览器从内存复用队列以外内存地址中获取第三内存地址，将该第三内存地址作为分配的第一内存地址。
浏览器还有可能从内存复用队列中获取不到该第二内存地址，比如内存复用队列中不存在内存空间不小于视频帧数据的数据量的内存地址。此时，浏览器可以为该视频帧数据重新分配一个内存地址，也即是，浏览器在该内存复用队列以外的未分配的内存地址中获取第三内存地址，将该第三内存地址作为为该视频帧数据分配的第一内存地址。例如，浏览器可以采用new ArrayBuffer()语句创建一个满足视频帧数据大小的数组，将该数组所指示的内存地址即为第三内存地址。当然，该第三内存地址所指示内存空间不小于该视频帧数据的数据量。
2016、如果内存复用队列中未包含有内存地址，浏览器从内存复用队列以外的内存地址中获取第四内存地址，将该第四内存地址作为分配的第一内存地址。
当浏览器检测到内存复用队列未包含有内存地址时，会为该视频帧数据重新分配一个内存地址，也即是，从复用队列以外的未分配的地址中获取第四内存地址。该分配第四内存地址的过程与步骤2014中分配第三内存地址的过程同理，在此不做赘述。
2017、浏览器将视频帧数据存储至第一内存地址所指示的内存空间。
参见图4，本实施例中对视频流进行解码的过程，解码后分配内存的过程以及存储解码得到的视频帧数据的过程均可以在子线程中处理。浏览器在将视频帧数据存储到分配的内存空间之后，浏览器可以在主线程对该视频帧数据进行渲染，以显示该视频帧数据所指示的视频画面。
其中，浏览器通过主线程对视频帧数据进行渲染，该过程可以为：浏览器在主线程维护一个渲染队列，浏览器存储视频帧数据之后，通过地址推送消息将该视频帧数据所在的第一内存地址发送至主线程，以在主线程将该第一内存地址添加到该渲染队列。其中，地址推送消息携带该第一内存地址。
在渲染过程中，浏览器会定时扫描该渲染队列，如图4所示，浏览器向渲染队列请求视频帧数据(requestAnimationFrame)，按照该渲染队列中内存地址的排列顺序，依次获取内存地址所指示的视频帧数据，并对获取的视频帧数据进行渲染。其中，由于浏览器对视频流解码的速度和渲染的速度之间存在差异，浏览器到达扫描渲染队列的周期时，该渲染队列可能为空，此时，浏览器便不执行渲染过程，待到下一个扫描周期继续进行扫描，待扫描到渲染队列不为空时，根据渲染队列中内存地址所指示的视频帧数据进行渲染。例如，浏览器可以采用预设绘图标准进行渲染，并以画布(canvas)为容器显示渲染后的图像。该预设绘图标准可以为WebGL绘图标准，本实施例对此不作限定。
2018、基于视频帧数据进行渲染后，浏览器将第一内存地址添加到内存复用队列中。
为了避免不断分配新的内存造成内存占用量迅速上升，本申请实施例中，浏览器可以将已经渲染过的视频帧数据所占用的内存空间进行重复利用，也即是，浏览器将已经渲染过的视频帧数据所在的内存地址回收到内存复用队列中，以便后续将该内存地址再次分配给其他未渲染的视频帧数据。
浏览器可以通过两种方式将第一内存地址添加到该内存复用队列中：在一种实施方式中，浏览器可以通过主线程将第一内存地址传递给该多媒体浏览应用的子线程，并通过该子线程将该第一内存地址添加到该内存复用队列中；在另一种实施方式中，浏览器直接通过主线程将该第一内存地址添加到该内存复用队列中。在实施过程中，浏览器在主线程对视频帧数据进行渲染后，可以通过调用线程交互接口(postMessage)将该第一内存地址发送给子线程，通过子线程执行将第一内存地址添加到内存复用队列的步骤。
需要说明的是，由于对视频流进行解码的速度和渲染的速度在一段时间内是相对稳定的，浏览器一直处于占用状态的内存地址的数目可以达到一个稳定的状态，也即是，当浏览器一直处于占用状态的内存地址的数目达到一定数目时，便能够满足已经解码但是还未进行渲染的视频帧数据的存储需求。此时，内存复用队列中所包含的内存地址的数目也处于相对稳定的状态，也即是，在从内存复用队列中移出一个内存地址时，也会向内存复用队列中添加一个内存地址。通过对播放高清监控视频的测试，在对视频播放过程中，内存复用队列中包含5个内存地址时便可以满足播放需求。由于在播放视频的过程中，浏览器仅需开辟少量的内存地址，通过对该少量的内存地址的重复利用便可实现在网页中播放视频，大大降低了内存占用量，提高了内存利用率。
其中，内存复用队列的原理与步骤207所述内存复用队列的原理相同，在此不做赘述。
本申请实施例提供的方法，浏览器通过将已经进行了渲染的视频帧数据所在的内存地址添加到内存复用队列，后续可以根据该内存复用队列进行内存分配，使得已经分配的内存地址能够得到重复利用，在浏览器播放视频的过程中，无需针对每一个视频帧数据均开辟一块内存空间，降低了浏览器的内存占用率，提高了浏览器的内存利用率。
上图3所提供的实施例是在对视频流进行解码后，再根据解码得到的视频帧数据进行内存分配，此外，终端还可以在接收到视频流后，进行内存预分配，由于视频流解码前无法得知解码后的视频帧数据大小，因此需要预先知道视频流的分辨率，根据视频流的分辨率所指示的视频帧数据进行内存预分配，才能保证预分配合适的内存大小，详细过程参见图7所提供的实施例。
图7是本申请实施例提供的一种内存管理方法的流程图，参见图7，本申请实施例提供的方法流程包括：
301、当接收到视频流时，获取视频流的分辨率。
在该实施例中，视频流的分辨率可以由服务器预先配置到终端，也可以由终端主动从服务器获取，本实施例对此不作限定。终端在接收到视频流之后，可以根据视频流所包含的数据包的帧标识，来确定一帧视频数据，并在解码前为该一帧视频数据进行内存的预分配。
302、检测内存复用队列中是否包含有内存地址，如果包含有，执行步骤303，否则执行步骤305。
该步骤与步骤201中检测内存复用队列中是否包含有内存地址的过程同理，在此不做赘述。
303、如果内存复用队列中包含有内存地址，根据分辨率所指示的视频帧数据的数据量，从内存复用队列中获取第二内存地址。
终端可以根据视频流的分辨率，计算视频帧数据的数据量。例如，终端根据该分辨率确定视频帧数据的帧长度和帧宽度，例如，当分辨率为1080×720时，对应的视频帧数据的帧长度为1080，帧宽度为720。根据确定的视频帧数据的帧长度和帧宽度，以及视频帧数据的图像格式，确定视频帧数据的数据量。该过程与上述步骤202中确定视频帧数据的数据量的方式相同，在此不做赘述。
304、如果从内存复用队列中获取到该第二内存地址，将第二内存地址移出内存复用队列，并将第二内存地址作为为该视频帧数据分配的第一内存地址。
该步骤与上述步骤203同理，在此不做赘述。
305、如果未从内存复用队列中获取到该第二内存地址，从内存复用队列以外内存地址中获取第三内存地址，将该第三内存地址作为分配的第一内存地址。
该步骤与上述步骤204同理，在此不做赘述。
306、如果内存复用队列中未包含有内存地址，根据该分辨率所指示的视频帧数据的数据量，从内存复用队列以外的内存地址中获取第四内存地址，将第四内存地址作为分配的第一内存地址。
该步骤与上述步骤205同理，在此不做赘述。
307、将视频帧数据存储至第一内存地址所指示的内存空间。
需要说明的是，参见图8，其示出了一种内存管理的流程示意图，上述步骤301至步骤306在终端的主线程执行，终端在为视频帧数据进行内存的预分配之后，可以将该预分配的第一内存地址通过地址传递消息(postMessage)传递给子线程，在子线程解码得到该视频帧数据后，将该视频帧数据存储至该第一内存地址所指示的内存空间。
另外，本实施例中，终端也会对解码得到的视频帧数据进行渲染，该渲染的过程与上述步骤206中所述的过程同理，在此不做赘述。
308、基于所述视频帧数据进行渲染后，将第一内存地址添加到内存复用队列中。
该步骤与上述步骤207同理，在此不做赘述。
需要说明的是，由于在视频播放的过程中，视频流的分辨率还有可能发生变化，比如，用户通过终端对视频流的分辨率进行了修改，当修改后的分辨率高于修改前的分辨率时，终端为解码得到的视频帧数据分配的第一内存地址可能不足以存储该视频帧数据，此时，当终端解码得到的视频帧数据的数据量大于该第一内存地址所指示的内存空间时，终端会在子线程根据该视频帧数据的实际数据量，为该视频帧数据重新分配一块内存，也即是，从内存复用队列和第一内存地址以外的内存地址中，获取一个内存地址分配给该视频帧数据。另外，终端将该视频帧数据所对应的实际分辨率通过子线程发送至主线程，在主线程对视频流的分辨率进行修改，后续基于修改后的分辨率进行内存分配。其中，在向主线程发送实际分辨率的同时，还可以将分配的内存地址发送到主线程，以通过主线程将该分配的内存地址添加到渲染队列中。
本申请实施例提供的方法，通过将已经进行了渲染的视频帧数据所在的内存地址添加 到内存复用队列，后续可以根据该内存复用队列进行内存分配，使得已经分配的内存地址能够得到重复利用，在播放视频的过程中，无需针对每一个视频帧数据均开辟一块内存空间，降低了内存占用率，提高了内存利用率。
由于本申请实施例中承载内存管理的主体是多媒体浏览应用，为了更加清楚的对内存管理方法流程进行解释说明，下面以多媒体浏览应用是浏览器为例，对上述图7所述的内存管理方法进行介绍，详细流程参见图9：
3011、当接收到视频流时，浏览器获取视频流的分辨率。
在该实施例中，视频流的分辨率可以由服务器预先配置到浏览器，也可以由浏览器主动从服务器获取，本实施例对此不作限定。浏览器在接收到视频流之后，可以根据视频流所包含的数据包的帧标识，来确定一帧视频数据，并在解码前为该一帧视频数据进行内存的预分配。
3012、浏览器检测内存复用队列中是否包含有内存地址，如果包含有，执行步骤3013，否则执行步骤3015。
该步骤与步骤2012中浏览器检测内存复用队列中是否包含有内存地址的过程同理，在此不做赘述。
3013、如果内存复用队列中包含有内存地址，浏览器根据分辨率所指示的视频帧数据的数据量，从内存复用队列中获取第二内存地址。
浏览器可以根据视频流的分辨率，计算视频帧数据的数据量。例如，浏览器根据该分辨率确定视频帧数据的帧长度和帧宽度，例如，当分辨率为1080×720时，对应的视频帧数据的帧长度为1080，帧宽度为720。根据确定的视频帧数据的帧长度和帧宽度，以及视频帧数据的图像格式，确定视频帧数据的数据量。该过程与上述步骤2013中确定视频帧数据的数据量的方式相同，在此不做赘述。
3014、如果从内存复用队列中获取到该第二内存地址，浏览器将第二内存地址移出内存复用队列，并将第二内存地址作为为该视频帧数据分配的第一内存地址。
该步骤与上述步骤2014同理，在此不做赘述。
3015、如果未从内存复用队列中获取到该第二内存地址，浏览器从内存复用队列以外内存地址中获取第三内存地址，将该第三内存地址作为分配的第一内存地址。
该步骤与上述步骤2015同理，在此不做赘述。
3016、如果内存复用队列中未包含有内存地址，浏览器根据该分辨率所指示的视频帧数据的数据量，从内存复用队列以外的内存地址中获取第四内存地址，将第四内存地址作为分配的第一内存地址。
该步骤与上述步骤2016同理，在此不做赘述。
3017、浏览器将视频帧数据存储至第一内存地址所指示的内存空间。
本实施例中，浏览器也会对解码得到的视频帧数据进行渲染，该渲染的过程与上述步骤2017中所述的过程同理，在此不做赘述。
需要说明的是，参见图8，其示出了一种内存管理的流程示意图，上述步骤3011至步骤3016在浏览器的主线程执行，浏览器在为视频帧数据进行内存的预分配之后，可以将该预分配的第一内存地址通过地址传递消息(postMessage)传递给子线程，在子线程将该视 频帧数据存储至该第一内存地址所指示的内存空间。
3018、基于所述视频帧数据进行渲染后，浏览器将第一内存地址添加到内存复用队列中。
该步骤与上述步骤2018同理，在此不做赘述。
需要说明的是，由于在视频播放的过程中，视频流的分辨率还有可能发生变化，比如，用户通过浏览器对视频流的分辨率进行了修改，当修改后的分辨率高于修改前的分辨率时，浏览器为解码得到的视频帧数据分配的第一内存地址可能不足以存储该视频帧数据，此时，当浏览器解码得到的视频帧数据的数据量大于该第一内存地址所指示的内存空间时，浏览器会在子线程根据该视频帧数据的实际数据量，为该视频帧数据重新分配一块内存，也即是，从内存复用队列和第一内存地址以外的内存地址中，获取一个内存地址分配给该视频帧数据。另外，浏览器将该视频帧数据所对应的实际分辨率通过子线程发送至主线程，在主线程对视频流的分辨率进行修改，后续基于修改后的分辨率进行内存分配。其中，在向主线程发送实际分辨率的同时，还可以将分配的内存地址发送到主线程，以通过主线程将该分配的内存地址添加到渲染队列中。
本申请实施例提供的方法，浏览器通过将已经进行了渲染的视频帧数据所在的内存地址添加到内存复用队列，后续可以根据该内存复用队列进行内存分配，使得已经分配的内存地址能够得到重复利用，在播放视频的过程中，无需针对每一个视频帧数据均开辟一块内存空间，降低了浏览器的内存占用率，提高了浏览器内存利用率。
图10是本申请实施例提供的一种内存管理装置的框图。参照图10，该装置包括分配模块401，存储模块402和添加模块403。
其中，分配模块，用于基于内存复用队列，为视频帧数据分配第一内存地址，该内存复用队列用于记录已进行了渲染的视频帧数据所在的内存地址；存储模块，用于将该视频帧数据存储至该第一内存地址所指示的内存空间；添加模块，用于在基于该视频帧数据进行渲染后，将该第一内存地址添加到该内存复用队列中。
在一种可能的实现方式中，该分配模块包括：
第一获取单元，用于如果该内存复用队列中包含有内存地址，根据该视频帧数据的数据量，从该内存复用队列中获取第二内存地址，该第二内存地址所指示的内存空间不小于该视频帧数据的数据量；
第一分配单元，用于如果从该内存复用队列中获取到该第二内存地址，将该第二内存地址移出该内存复用队列，并将该第二内存地址作为该第一内存地址。
在一种可能的实现方式中，该分配模块还包括：
第二获取单元，用于如果未从该内存复用队列中获取到该第二内存地址，从该内存复用队列以外内存地址中获取第三内存地址，该第三内存地址所指示的内存空间不小于该视频帧数据的数据量；
第二分配单元，用于将该第三内存地址作为该第一内存地址。
在一种可能的实现方式中，该分配模块还包括：
第三获取单元，用于如果该内存复用队列中未包含有内存地址，从该内存复用队列以外的内存地址中获取第四内存地址；
第三分配单元，用于将该第四内存地址作为该第一内存地址。
在一种可能的实现方式中，该分配模块，用于当接收到视频流时，获取该视频流的分辨率；根据该分辨率所指示的视频帧数据的数据量，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤；或，
该分配模块，用于在对接收到的视频流进行解码的过程中，当获取到该视频帧数据时，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤。
在一种可能的实现方式中，该装置还包括：
启动模块，用于通过多媒体浏览应用的指定接口启动子线程；
解码模块，用于通过该子线程对该视频帧数据进行解码。
在一种可能的实现方式中，该装置还包括：
渲染模块，用于通过多媒体浏览应用的主线程对该视频帧数据进行渲染；
相应地，该添加模块用于：通过该主线程将该第一内存地址传递给该多媒体浏览应用的子线程；通过该子线程将该第一内存地址添加到该内存复用队列中；或，通过该主线程将该第一内存地址添加到该内存复用队列中。
本公开实施例提供的装置，通过将已经进行了渲染的视频帧数据所在的内存地址添加到内存复用队列，后续可以根据该内存复用队列进行内存分配，使得已经分配的内存地址能够得到重复利用，在播放视频的过程中，无需针对每一个视频帧数据均开辟一块内存空间，降低了内存占用率，提高了内存利用率。
需要说明的是：上述实施例提供的内存管理装置在管理内存时，仅以上述各功能模块的划分进行举例说明，在一种可能的实现方式中，可以根据需要而将上述功能分配由不同的功能模块完成，即将设备的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的内存管理装置与触内存管理方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
图11是本申请实施例提供的一种终端的结构示意图，该终端可以用于执行上述各个实施例中提供的内存管理方法。参见图11，该终端500包括：
终端500可以包括RF(Radio Frequency，射频)电路110、包括有一个或一个以上计算机可读存储介质的存储器120、输入单元130、显示单元140、传感器150、音频电路160、WiFi(Wireless Fidelity，无线保真)模块170、包括有一个或者一个以上处理核心的处理器180、以及电源190等部件。本领域技术人员可以理解，图5中示出的终端结构并不构成对终端的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。其中：
RF电路110可用于收发信息或通话过程中，信号的接收和发送，特别地，将基站的下行信息接收后，交由一个或者一个以上处理器180处理；另外，将涉及上行的数据发送给基站。通常，RF电路110包括但不限于天线、至少一个放大器、调谐器、一个或多个振荡器、用户身份模块(SIM)卡、收发信机、耦合器、LNA(Low Noise Amplifier，低噪声放大器)、双工器等。此外，RF电路110还可以通过无线通信与网络和其他设备通信。所述无线通信可以使用任一通信标准或协议，包括但不限于GSM(Global System of Mobile communication，全球移动通讯系统)、GPRS(General Packet Radio Service，通用分组无线服 务)、CDMA(Code Division Multiple Access，码分多址)、WCDMA(Wideband Code Division Multiple Access,宽带码分多址)、LTE(Long Term Evolution,长期演进)、电子邮件、SMS(Short Messaging Service，短消息服务)等。
存储器120可用于存储软件程序以及模块，处理器180通过运行存储在存储器120的软件程序以及模块，从而执行各种功能应用以及数据处理。存储器120可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据终端500的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器120可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。相应地，存储器120还可以包括存储器控制器，以提供处理器180和输入单元130对存储器120的访问。
输入单元130可用于接收输入的数字或字符信息，以及产生与用户设置以及功能控制有关的键盘、鼠标、操作杆、光学或者轨迹球信号输入。具体地，输入单元130可包括触敏表面131以及其他输入设备132。触敏表面131，也称为触摸显示屏或者触控板，可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触敏表面131上或在触敏表面131附近的操作)，并根据预先设定的程式驱动相应的连接装置。可选的，触敏表面131可包括触摸检测装置和触摸控制器两个部分。其中，触摸检测装置检测用户的触摸方位，并检测触摸操作带来的信号，将信号传送给触摸控制器；触摸控制器从触摸检测装置上接收触摸信息，并将它转换成触点坐标，再送给处理器180，并能接收处理器180发来的命令并加以执行。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触敏表面131。除了触敏表面131，输入单元130还可以包括其他输入设备132。具体地，其他输入设备132可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆等中的一种或多种。
显示单元140可用于显示由用户输入的信息或提供给用户的信息以及终端500的各种图形用户接口，这些图形用户接口可以由图形、文本、图标、视频和其任意组合来构成。显示单元140可包括显示面板141，可选的，可以采用LCD(Liquid Crystal Display，液晶显示器)、OLED(Organic Light-Emitting Diode,有机发光二极管)等形式来配置显示面板141。进一步的，触敏表面131可覆盖显示面板141，当触敏表面131检测到在其上或附近的触摸操作后，传送给处理器180以确定触摸事件的类型，随后处理器180根据触摸事件的类型在显示面板141上提供相应的视觉输出。虽然在图11中，触敏表面131与显示面板141是作为两个独立的部件来实现输入和输入功能，但是在某些实施例中，可以将触敏表面131与显示面板141集成而实现输入和输出功能。
终端500还可包括至少一种传感器150，比如光传感器、运动传感器以及其他传感器。具体地，光传感器可包括环境光传感器及接近传感器，其中，环境光传感器可根据环境光线的明暗来调节显示面板141的亮度，接近传感器可在终端500移动到耳边时，关闭显示面板141和/或背光。作为运动传感器的一种，重力加速度传感器可检测各个方向上(一般为三轴)加速度的大小，静止时可检测出重力的大小及方向，可用于识别手机姿态的应用(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等；至于终端500还可配置的陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感 器，在此不再赘述。
音频电路160、扬声器161，传声器162可提供用户与终端500之间的音频接口。音频电路160可将接收到的音频数据转换后的电信号，传输到扬声器161，由扬声器161转换为声音信号输出；另一方面，传声器162将收集的声音信号转换为电信号，由音频电路160接收后转换为音频数据，再将音频数据输出处理器180处理后，经RF电路110以发送给比如另一终端，或者将音频数据输出至存储器120以便进一步处理。音频电路160还可能包括耳塞插孔，以提供外设耳机与终端500的通信。
WiFi属于短距离无线传输技术，终端500通过WiFi模块170可以帮助用户收发电子邮件、浏览网页和访问流式媒体等，它为用户提供了无线的宽带互联网访问。虽然图11示出了WiFi模块170，但是可以理解的是，其并不属于终端500的必须构成，完全可以根据需要在不改变发明的本质的范围内而省略。
处理器180是终端500的控制中心，利用各种接口和线路连接整个手机的各个部分，通过运行或执行存储在存储器120内的软件程序和/或模块，以及调用存储在存储器120内的数据，执行终端500的各种功能和处理数据，从而对手机进行整体监控。可选的，处理器180可包括一个或多个处理核心；优选的，处理器180可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器180中。
终端500还包括给各个部件供电的电源190(比如电池)，优选的，电源可以通过电源管理系统与处理器180逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。电源190还可以包括一个或一个以上的直流或交流电源、再充电系统、电源故障检测电路、电源转换器或者逆变器、电源状态指示器等任意组件。
尽管未示出，终端500还可以包括摄像头、蓝牙模块等，在此不再赘述。具体在本实施例中，终端的显示单元是触摸屏显示器，终端还包括有存储器，以及一个或者一个以上的程序，其中一个或者一个以上程序存储于存储器中，且经配置以由一个或者一个以上处理器执行。所述一个或者一个以上程序包含可执行指令，终端500被配置为执行指令，以执行上述内存管理方法实施例中终端所执行的方法。
图12是本申请实施例提供的一种电子设备600的结构示意图。参见图12，该电子设备600包括处理器和存储器，还可以包括通信接口、通信总线、输入输出接口和显示设备，其中，处理器、存储器、输入输出接口、显示设备和通信接口通过通信总线完成相互间的通信。该存储器存储有计算机程序，该处理器用于执行存储器上所存放的计算机程序，实现上述实施例中的内存管理方法。
通信总线是连接所描述的元素的电路并且在这些元素之间实现传输。例如，处理器通过通信总线从其它元素接收到命令，解密接收到的命令，根据解密的命令执行计算或数据处理。存储器可以包括程序模块，例如内核(kernel)，中间件(middleware)，应用程序编程接口(Application Programming Interface，API)和应用。该程序模块可以是有软件、固件或硬件、或其中的至少两种组成。输入输出接口转发用户通过输入输出设备(例如感应器、键盘、触摸屏)输入的命令或数据。显示设备显示各种信息给用户。通信接口将该电子设备600与其它网络设备、用户设备、网络进行连接。例如，通信接口可以通过有线或无线 连接到网络以连接到外部其它的网络设备或用户设备。无线通信可以包括以下至少一种：无线保真(Wireless Fidelity，WiFi)，蓝牙(Bluetooth，BT)，近距离无线通信技术(Near Field Communication，NFC)，全球卫星定位系统(Global Positioning System，GPS)和蜂窝通信(cellular communication)(例如，长期演进技术(Long Term Evolution，LTE)，长期演进技术的后续演进(Long Term Evolution–Advanced，LTE-A)，码分多址(Code Division Multiple Access，CDMA)，宽带码分多址(Wideband CDMA，WCDMA)，通用移动通信系统(Universal Mobile Telecommunication System，UMTS)，无线宽带接入(Wireless Broadband，WiBro)和全球移动通讯系统(Global System for Mobile communication，GSM)。有线通信可以包括以下至少一种：通用串行总线(Universal Serial Bus，USB)，高清晰度多媒体接口(High Definition Multimedia Interface，HDMI)，异步传输标准接口(Recommended Standard 232，RS-232)，和普通老式电话业务(Plain Old Telephone Service，POTS)。网络可以是电信网络和通信网络。通信网络可以为计算机网络、因特网、物联网、电话网络。电子设备600可以通过通信接口连接网络，电子设备600和其它网络设备通信所用的协议可以被应用、应用程序编程接口(Application Programming Interface，API)、中间件、内核和通信接口至少一个支持。
在示例性实施例中，还提供了一种计算机可读存储介质，例如存储有计算机程序的存储器，上述计算机程序被处理器执行时实现上述实施例中内存管理方法。例如，所述计算机可读存储介质可以是只读内存(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory，RAM)、只读光盘(Compact Disc Read-Only Memory，CD-ROM)、磁带、软盘和光数据存储设备等。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上所述仅为本申请的较佳实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (16)

  1. 一种内存管理方法，其特征在于，所述方法包括：
    基于内存复用队列，为视频帧数据分配第一内存地址，所述内存复用队列用于记录已进行了渲染的视频帧数据所在的内存地址；
    将所述视频帧数据存储至所述第一内存地址所指示的内存空间；
    在基于所述视频帧数据进行渲染后，将所述第一内存地址添加到所述内存复用队列中。
  2. 根据权利要求1所述的方法，其特征在于，所述基于内存复用队列，为视频帧数据分配第一内存地址，包括：
    如果所述内存复用队列中包含有内存地址，根据所述视频帧数据的数据量，从所述内存复用队列中获取第二内存地址，所述第二内存地址所指示的内存空间不小于所述视频帧数据的数据量；
    如果从所述内存复用队列中获取到所述第二内存地址，将所述第二内存地址移出所述内存复用队列，并将所述第二内存地址作为所述第一内存地址。
  3. 根据权利要求2所述的方法，其特征在于，所述根据所述视频帧数据的数据量，从所述内存复用队列中获取第二内存地址之后，所述方法还包括：
    如果未从所述内存复用队列中获取到所述第二内存地址，从所述内存复用队列以外内存地址中获取第三内存地址，所述第三内存地址所指示的内存空间不小于所述视频帧数据的数据量；
    将所述第三内存地址作为所述第一内存地址。
  4. 根据权利要求1所述的方法，其特征在于，所述基于内存复用队列，为视频帧数据分配第一内存地址，包括：
    如果所述内存复用队列中未包含有内存地址，从所述内存复用队列以外的内存地址中获取第四内存地址；
    将所述第四内存地址作为所述第一内存地址。
  5. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    当接收到视频流时，获取所述视频流的分辨率；根据所述分辨率所指示的视频帧数据的数据量，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤；或，
    在对接收到的视频流进行解码的过程中，当获取到所述视频帧数据时，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤。
  6. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    通过多媒体浏览应用的指定接口启动子线程；
    通过所述子线程对所述视频帧数据进行解码。
  7. 根据权利要求1或6所述的方法，其特征在于，所述方法还包括：
    通过多媒体浏览应用的主线程对所述视频帧数据进行渲染；
    相应地，所述在基于所述视频帧数据进行渲染后，将所述第一内存地址添加到所述内存复用队列中，包括：
    通过所述主线程将所述第一内存地址传递给所述多媒体浏览应用的子线程；通过所述子线程将所述第一内存地址添加到所述内存复用队列中；或，
    通过所述主线程将所述第一内存地址添加到所述内存复用队列中。
  8. 一种内存管理装置，其特征在于，所述装置包括：
    分配模块，用于基于内存复用队列，为视频帧数据分配第一内存地址，所述内存复用队列用于记录已进行了渲染的视频帧数据所在的内存地址；
    存储模块，用于将所述视频帧数据存储至所述第一内存地址所指示的内存空间；
    添加模块，用于在基于所述视频帧数据进行渲染后，将所述第一内存地址添加到所述内存复用队列中。
  9. 根据权利要求8所述的装置，其特征在于，所述分配模块包括：
    第一获取单元，用于如果所述内存复用队列中包含有内存地址，根据所述视频帧数据的数据量，从所述内存复用队列中获取第二内存地址，所述第二内存地址所指示的内存空间不小于所述视频帧数据的数据量；
    第一分配单元，用于如果从所述内存复用队列中获取到所述第二内存地址，将所述第二内存地址移出所述内存复用队列，并将所述第二内存地址作为所述第一内存地址。
  10. 根据权利要求9所述的装置，其特征在于，所述分配模块还包括：
    第二获取单元，用于如果未从所述内存复用队列中获取到所述第二内存地址，从所述内存复用队列以外内存地址中获取第三内存地址，所述第三内存地址所指示的内存空间不小于所述视频帧数据的数据量；
    第二分配单元，用于将所述第三内存地址作为所述第一内存地址。
  11. 根据权利要求8所述的装置，其特征在于，所述分配模块还包括：
    第三获取单元，用于如果所述内存复用队列中未包含有内存地址，从所述内存复用队列以外的内存地址中获取第四内存地址；
    第三分配单元，用于将所述第四内存地址作为所述第一内存地址。
  12. 根据权利要求8所述的装置，其特征在于，所述分配模块，用于当接收到视频流时，获取所述视频流的分辨率；根据所述分辨率所指示的视频帧数据的数据量，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤；或，
    所述分配模块，用于在对接收到的视频流进行解码的过程中，当获取到所述视频帧数据时，执行基于内存复用队列，为视频帧数据分配第一内存地址的步骤。
  13. 根据权利要求8所述的装置，其特征在于，所述装置还包括：
    启动模块，用于通过多媒体浏览应用的指定接口启动子线程；
    解码模块，用于通过所述子线程对所述视频帧数据进行解码。
  14. 根据权利要求8或13所述的装置，其特征在于，所述装置还包括：
    渲染模块，用于通过多媒体浏览应用的主线程对所述视频帧数据进行渲染；
    相应地，所述添加模块用于：通过所述主线程将所述第一内存地址传递给所述多媒体浏览应用的子线程；通过所述子线程将所述第一内存地址添加到所述内存复用队列中；或，通过所述主线程将所述第一内存地址添加到所述内存复用队列中。
  15. 一种电子设备，其特征在于，所述电子设备包括处理器和存储器；存储器，用于存放计算机程序；处理器，用于执行存储器上所存放的计算机程序，实现权利要求1-7任一项所述的方法步骤。
  16. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1-7任一项所述的方法步骤。
PCT/CN2018/088395 2017-07-18 2018-05-25 内存管理方法、装置、电子设备及存储介质 WO2019015395A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP18834482.4A EP3640807B1 (en) 2017-07-18 2018-05-25 Memory management method and apparatus, and electronic device and storage medium
EP22164063.4A EP4040299B1 (en) 2017-07-18 2018-05-25 Memory management method and apparatus, electronic device and storage medium
US16/631,304 US11604726B2 (en) 2017-07-18 2018-05-25 Memory management method, electronic device and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710587746.0 2017-07-18
CN201710587746.0A CN109271327B (zh) 2017-07-18 2017-07-18 内存管理方法及装置

Publications (1)

Publication Number Publication Date
WO2019015395A1 true WO2019015395A1 (zh) 2019-01-24

Family

ID=65015043

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/088395 WO2019015395A1 (zh) 2017-07-18 2018-05-25 内存管理方法、装置、电子设备及存储介质

Country Status (4)

Country Link
US (1) US11604726B2 (zh)
EP (2) EP4040299B1 (zh)
CN (1) CN109271327B (zh)
WO (1) WO2019015395A1 (zh)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11025982B2 (en) * 2019-03-29 2021-06-01 Twizted Design, Inc. System and method for synchronizing content and data for customized display
CN110708609A (zh) * 2019-08-05 2020-01-17 青岛海信传媒网络技术有限公司 一种视频播放方法及装置
CN110730364B (zh) * 2019-10-14 2021-08-31 广州微算互联信息技术有限公司 一种云手机数据传输方法、装置及存储介质
CN110727520B (zh) * 2019-10-23 2022-05-03 四川长虹电器股份有限公司 一种优化Android帧动画的实现方法
CN111736998A (zh) * 2020-06-18 2020-10-02 上海商汤智能科技有限公司 内存管理方法和相关产品
CN114071224B (zh) * 2020-07-31 2023-08-25 腾讯科技（深圳）有限公司 视频数据处理方法、装置、计算机设备及存储介质
US11645107B2 (en) * 2021-01-13 2023-05-09 Adobe Inc. Processing multi-frame tasks in a multi-threaded digital design system
CN112882682A (zh) * 2021-02-25 2021-06-01 广州趣丸网络科技有限公司 一种音频录制设备中的内存复用方法、装置、设备及介质
CN114820272A (zh) * 2022-03-18 2022-07-29 维塔科技(北京)有限公司 数据交互的方法、装置、存储介质及电子设备
CN115134661A (zh) * 2022-06-28 2022-09-30 龙芯中科(合肥)技术有限公司 视频处理方法及视频处理应用
CN115484403B (zh) * 2022-08-08 2023-10-24 荣耀终端有限公司 录像方法和相关装置
CN117251292B (zh) * 2023-11-13 2024-03-29 山东泽赢信息科技服务有限公司 内存管理方法、系统、终端及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101320351A (zh) * 2008-06-27 2008-12-10 华中科技大学 内存的分配、清理和释放方法及内存管理的装置
US20090257508A1 (en) * 2008-04-10 2009-10-15 Gaurav Aggarwal Method and system for enabling video trick modes
CN101984417A (zh) * 2010-11-01 2011-03-09 中兴通讯股份有限公司 内存管理方法及装置
CN103246567A (zh) * 2013-03-26 2013-08-14 中国科学院电子学研究所 用于目标跟踪内存管理的队列方法
CN105245912A (zh) * 2015-10-21 2016-01-13 东方网力科技股份有限公司 一种缓存视频数据及读取视频数据的方法及装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6732249B1 (en) * 2000-06-09 2004-05-04 3Com Corporation Host computer virtual memory within a network interface adapter
JP2014531142A (ja) * 2011-08-16 2014-11-20 デスティニーソフトウェアプロダクションズ インク スクリプトをベースとするビデオ・レンダリング
US8627036B2 (en) 2011-09-12 2014-01-07 Microsoft Corporation Memory management techniques

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090257508A1 (en) * 2008-04-10 2009-10-15 Gaurav Aggarwal Method and system for enabling video trick modes
CN101320351A (zh) * 2008-06-27 2008-12-10 华中科技大学 内存的分配、清理和释放方法及内存管理的装置
CN101984417A (zh) * 2010-11-01 2011-03-09 中兴通讯股份有限公司 内存管理方法及装置
CN103246567A (zh) * 2013-03-26 2013-08-14 中国科学院电子学研究所 用于目标跟踪内存管理的队列方法
CN105245912A (zh) * 2015-10-21 2016-01-13 东方网力科技股份有限公司 一种缓存视频数据及读取视频数据的方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3640807A4 *

Also Published As

Publication number Publication date
EP3640807A4 (en) 2020-07-15
EP4040299B1 (en) 2024-03-06
EP3640807A1 (en) 2020-04-22
US11604726B2 (en) 2023-03-14
EP4040299A1 (en) 2022-08-10
EP3640807B1 (en) 2023-03-29
CN109271327A (zh) 2019-01-25
US20200210329A1 (en) 2020-07-02
CN109271327B (zh) 2020-11-27

Similar Documents

Publication Publication Date Title
US11604726B2 (en) Memory management method, electronic device and storage medium
WO2016090902A1 (zh) 一种操作系统热切换的方法、装置及移动终端
WO2018161534A1 (zh) 一种显示图像的方法、双屏终端和计算机可读的非易失性存储介质
CN108595203B (zh) 应用功能执行方法、移动终端和计算机可读存储介质
WO2015058617A1 (zh) 视频帧存储管理方法、视频服务器及通信系统
CN106658064B (zh) 虚拟礼物展示方法及装置
WO2017096909A1 (zh) 建立数据连接的方法及装置
CN104519404A (zh) 图像互换格式文件的播放方法及装置
CN104869465A (zh) 视频播放控制方法和装置
WO2021128929A1 (zh) 一种全景应用中图像渲染的方法及终端设备
CN109144723B (zh) 一种分配存储空间的方法和终端
US10136115B2 (en) Video shooting method and apparatus
CN109992192B (zh) 一种界面显示方法及终端设备
US10158896B2 (en) Video channel allocation management method and related device, and communication system
CN109725819B (zh) 界面显示方法、装置、双屏双系统终端及可读存储介质
CN110187934B (zh) 应用控制方法、终端及计算机可读存储介质
CN110209434B (zh) 一种内存管理方法、装置及计算机可读存储介质
CN113612957B (zh) 通话方法以及相关设备
CN105282628B (zh) 一种视频播放的方法和装置
CN115914719A (zh) 投屏显示方法、智能终端及存储介质
CN112887776B (zh) 一种降低音频延时的方法、设备及计算机可读存储介质
CN114253687A (zh) 一种多窗口泄露处理方法、设备及计算机可读存储介质
CN109710576B (zh) 文件展示的方法、终端及存储介质
CN110162349B (zh) 桌面启动器加速启动的方法、终端以及存储介质
CN116248792A (zh) 通知方法、智能终端及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18834482

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018834482

Country of ref document: EP

Effective date: 20200114